Probe Workflow

Probe real-host behavior with a temporary workflow

A probe is a throwaway GitHub Actions workflow that answers one question about the real world: what a third-party CLI actually prints, how a privilege boundary actually behaves, what a container or another OS actually ships. It exists because a unit test asserts what you believe, while a probe measures what is true. The lifecycle is fixed: write, push, read, iterate, then move each finding where it belongs and delete the probe. A probe that lingers becomes CI cost with no question left to answer.

Reach for one when:

• Code must match a tool's real output or error wordings, and the tool does not run on the development machine (another OS, another distro, a musl container, a privileged path).
• A feature's flagship scenario has only ever run against mocks or a stand-in host.
• A CI failure depends on runner-image state that local reproduction cannot fake (preinstalled tools, filesystem ACLs, service accounts).

Do not reach for one when a local test, a container run on the development machine, or reading the tool's source answers the question faster.

Invocation

The loop commits and pushes on every iteration. Get the user's explicit go-ahead for autonomous commit/push/run/cancel cycles before starting, or run under --dangerously-skip-permissions in a trusted checkout. Confirm which branch to push to: these repositories key workflow concurrency on the ref, so pushing to the default branch supersedes cleanly while a side branch queues alongside it. A probe living on a side branch cannot be started with gh workflow run, which only dispatches a workflow file present on the default branch (HTTP 404: workflow ... not found on the default branch): give it a push trigger on its own branch instead, and let each push start it.

Ground rules

• Mark it temporary in line one. The workflow's first comment says TEMPORARY probe workflow, to be deleted once <question> is answered. Name the file <subject>-probe.yaml.
• Trigger on push with a paths: filter covering the workflow file AND the code under test. A probe validating mymodule.py must re-run when mymodule.py changes, or code iterations silently test nothing.
• Cooldown rules apply. Installs from live registries carry the repository's cooldown (UV_EXCLUDE_NEWER, NPM_CONFIG_MIN_RELEASE_AGE in the workflow env:); distro archives (apt-get, apk) are out of scope by design. Pin actions by SHA and tools by version, copied from an existing workflow.
• Hard gates, not eyeballs. Every claim becomes a grep -q (or test -e, exit-code check) against captured output: command > out.txt 2>&1 || true then grep. A step that only prints is a diagnostic, not a validation, and belongs in the run only while a question is open.
• Negative gates too. Assert the behavior does NOT fire where it must not (if grep -q ...; then exit 1; fi), or the probe proves half the contract.

Measure before asserting

When an assertion fails and more than one theory explains it, do not fix the theory: add a measurement step and push again. Print the state the theories disagree about (ls -la the directory, run the raw command as each user, cat the config), read the numbers, then write the fix. Guessing costs a full runner round-trip per guess; measuring costs one round-trip total. Record each lesson as a comment beside the step that hit it, so retirement can route every one to its lasting home.

Two shell traps recur in measurement steps:

• GitHub's default shell is bash -e: a step written to capture a failure dies at the failing command instead. Set shell: bash {0} on diagnostic steps, or suffix || true on every command whose non-zero exit is the datum.
• Steps run as root in container jobs. Behavior gated on privilege (escalation, ownership, per-user config) needs a created unprivileged user, with scripts handed over via su <user> -s /bin/sh /path/script.sh and HOME set explicitly inside the script.