Dynamic Matrices in GitHub Actions

...from JSON payloads that you send! — This was a recent rabbit hole 🐰🕳 that took me 2 days to figure out. I couldn't find a quick and clear answer on Google so I figured I'd write about it.

matrix diagram
matrix diagram

I recently needed to understand GitHub actions' matrix strategy for work.

...A matrix allows you to create multiple jobs by performing variable substitution in a single job definition.

For example, you can use a matrix to create jobs for more than one supported version of a programming language, operating system, or tool...

Unfortunately, the docs didn't cover my specific use case and the examples had hardcoded matrix values:

I wanted to figure out how to create dynamic matrices... and more specifically, from JSON payloads that I'd be sending via repository_dispatch events.

So, how do you send dynamic matrix values from repository dispatch payloads?

Trial and error

My dev process was pretty clunky and was not particularly pleasant either. At the time of development, I hadn't yet tried nekos/act, which could've helped speed up parts of my process.

Set up a repo with a workflow

I first created a repo — well, shoved actions into an existing repo — and created a workflow that listened for repository_dispatch events, and specific event_types.

.github/workflows/dispatch_listener.yaml

POST /repos/{owner}/{repo}/dispatches

The next thing I did was call the GitHub REST API to trigger the repository's workflow manually. I did so via Postman, but in actual implementation, I'd probably be calling it via another workflow with something like HTTP Request Action or from server-side code using Octokit and authenticating as a GitHub application.

curl

I debugged the live workflow runs via the GitHub UI's "Actions" tab, which was not the worst given the simplicity of my workflow, but it was certainly not fun. The lack of IDE linting and type-system made it feel like I was developing in the dark.

91 workflow runs 😵‍💫
91 workflow runs 😵‍💫

Success

Skipping over a boring streak of unsuccessful workflows, here's the yaml for a workflow that finally worked.

Success: This workflow file allows you to:
  1. Post variable JSON arrays in a `client_payload`
  2. Pass them from 1 job run to a following job
  3. Trigger a matrix
.github/workflows/dispatch_listener.yaml
16 parallel matrix jobs
16 parallel matrix jobs

Notes

You could pass the client_payload keys directly to a matrix value

Or you can set them as outputs from prior jobs, leading up to a job with a matrix.

The key here was the single quotes '' wrapping the toJSON function. Without it, the outputs were set to either Array or Object — the types of the underlying JSON — or [ or { — the first line of pretty-printed JSON.

nektos/act

After finding my solution, I went back to try act, a tool for running GitHub actions/workflows locally. Given the same JSON payload I was sending via Postman,

event.json

I run act repository_dispatch -e event.json and can see the various steps and jobs of my workflow being executed.

act repository_dispatch -e event.json
act repository_dispatch -e event.json
Warning: At the time of writing this post, act doesn't handle matrices yet.

Other

Some complex workflows that people dropped in some threads that I encountered.