Crabbox Remote Test Runner

Crabbox

Remote and clean-machine proof. Packages. Docker. Live providers. Desktop. Cross-OS. The consumer repository owns when its validation needs a remote environment; Crabbox availability alone is not a reason to offload local work.

Backends:

• blacksmith-testbox: trusted maintainer source. Prepared CI. tbx_....
• aws: direct brokered Crabbox. Fresh PRs. Custom sync/env/capture. cbx_....
• local-container: Docker fallback. Not remote proof.
• ssh: existing operator host. macOS/Windows/WSL2.

Always report provider, id, run URL, command, result. Never call Testbox “AWS Crabbox.”

Repository Contract

This canonical skill owns portable Crabbox policy and CLI operations only. Consumer-specific setup belongs in that repository's AGENTS.md, package scripts, hydration workflow, or another file outside the synchronized skill.

Resolve these placeholders from trusted repository instructions before running an example:

• <check-command>: the repository's focused or broad validation command.
• <install-and-check-command>: its clean-container install plus validation.
• <trusted-bootstrap-script>: a maintainer-reviewed untrusted-PR bootstrap stored outside the untrusted checkout.
• <container-image> and <owner/repo#number>: the consumer's runtime and PR.

Never invent a missing command or copy a command from another consumer.

Authorization and Isolation

Routine use of the configured Crabbox/Testbox environment is part of completing the requested task; do not ask for separate approval. This includes creating, reusing and stopping task-owned leases, temporary state, and clean checkouts or worktrees needed for proof or a task-required Crabbox repair.

A dirty, missing or occupied checkout is a reason to use a clean task-owned checkout or worktree, not a permission blocker. Preserve existing checkouts, branches and unrelated edits. Keep source-trust, credential, production-access, budget and publication boundaries; routine-use approval does not waive them.

Route First

Source trust determines which providers are allowed. It does not select one.

• Trusted development tests/checks/builds: follow the consumer's local-first policy; use remote when its environment is needed or explicitly requested.
• Trusted + remote proof: inspect and preserve the resolved provider.
• Blacksmith Testbox: use when already resolved or explicitly requested.
• Direct AWS: use when AWS semantics are required or explicitly requested.
• Untrusted contributor/fork: secretless fork CI or sanitized direct AWS.
• Never untrusted code on credential-hydrated Testbox.
• Never run untrusted repo wrapper/config locally.
• No speculative warmup. Acquire when first heavy command ready. Reuse id. Stop.

Test size, expected duration, and hydration failure do not authorize a provider override. Omit --provider for normal work. Add it only when the user requests that backend or the proof specifically tests its semantics.

Preflight

Run from repo root.

command -v crabbox
crabbox --version
crabbox config show --json | jq '{provider, profile, target}'
crabbox run --help | sed -n '1,100p'
command -v blacksmith
blacksmith --version

Set the checked installed binary once. A consumer may document a different trusted wrapper, but the shared skill never assumes a sibling checkout or repository-specific script.

export CRABBOX="$(command -v crabbox)"
test -n "$CRABBOX"
"$CRABBOX" --version
"$CRABBOX" config show --json | jq '{provider, profile, target}'

Read .crabbox.yaml and config show; the resolved provider can also come from user or environment configuration. If the binary is missing, follow the consumer's trusted install instructions. For a source build or repair, verify the canonical upstream and use a clean task-owned checkout or worktree. Never assume a sibling checkout is trusted or overwrite its unrelated work. Keep task-specific builds separate from the operator's installed binary.

Trusted Testbox

Use this section only when config show resolves blacksmith-testbox or the user explicitly requested Testbox. These provider-neutral commands preserve the resolved configuration; add --provider blacksmith-testbox only for that explicit override.

One-shot heavy gate:

"$CRABBOX" run --timing-json -- CI=1 <check-command>

Several commands: warm once, save id, reuse, stop.

"$CRABBOX" warmup --keep --timing-json
"$CRABBOX" run --id <tbx_id> --timing-json -- <check-command>
"$CRABBOX" stop <tbx_id>

Rules:

• One lease, one active command. No sync/reclaim during run.
• Native Testbox runs own sync, including reused --id runs. Never rely on --no-sync to preserve a remote baseline: Blacksmith has no native bypass, and released Crabbox versions can silently ignore the flag. An unchanged intentional rerun is not a Testbox exception.
• --reclaim only deliberate checkout-path ownership transfer.
• Base/head change: stop. Rewarm. No stale-lease override.
• Raw SHA unreliable for warmup --ref; use branch/tag.
• blacksmith testbox list hides states. Use list --all or status --id <tbx_id>.
• Testbox status/stop: --id. No status --json.
• Delegated provider rejects --fresh-pr, --full-resync, --script*, --env-helper, capture/download flags.

Verify Synced Inputs

Before interpreting a missing-module or missing-fixture failure, compare required inputs with the frozen source. A successful sync can omit tracked files matched by its exclusions. Inspect the installed sync rules, including Git-ignore negations, rather than assuming Git and rsync select identical files.

Record each required missing path and its exact frozen Git blob. Restore only those reviewed inputs in the task-owned remote checkout, then verify their bytes. Do not regenerate a baseline, copy unrelated local state, disable exclusions, or treat every missing file as acceptable. Account for changed and untracked inputs separately from committed blobs. Recheck required inputs after each native sync before running the affected proof.

A timeout during sync is not a test result. Establish whether payload dispatch occurred before retrying or reporting a command failure. If dispatch is unknown, inspect the same lease and invocation first. Preserve completed proof against its source identity, and continue only the missing verification.

Untrusted AWS

Clean trusted default-branch checkout. Installed trusted Crabbox binary. Fresh lease per reviewed full head SHA. No instance role. No Tailscale. No hydration. Only CI forwarded. Trusted bootstrap uploaded beside --fresh-pr.

cd <clean-trusted-default-branch-checkout>
env -u CRABBOX_AWS_INSTANCE_PROFILE \
  "$CRABBOX" config show --json | \
  jq -e '.aws.instanceProfile == ""' >/dev/null

env -u CRABBOX_AWS_INSTANCE_PROFILE \
  -u CRABBOX_TAILSCALE \
  -u CRABBOX_TAILSCALE_AUTH_KEY \
  -u CRABBOX_TAILSCALE_AUTH_KEY_ENV \
  -u CRABBOX_TAILSCALE_EXIT_NODE \
  -u CRABBOX_TAILSCALE_EXIT_NODE_ALLOW_LAN_ACCESS \
  -u CRABBOX_TAILSCALE_HOSTNAME_TEMPLATE \
  -u CRABBOX_TAILSCALE_TAGS \
  "$CRABBOX" warmup \
  --provider aws --network public --tailscale=false \
  --tailscale-exit-node= \
  --tailscale-exit-node-allow-lan-access=false \
  --keep --timing-json

"$CRABBOX" inspect --provider aws --id <cbx_id> --json | \
  jq -e '.network == "public" and .tailscale == null' >/dev/null

env -u CRABBOX_AWS_INSTANCE_PROFILE \
  CRABBOX_ENV_ALLOW=CI \
  "$CRABBOX" run \
  --provider aws --id <cbx_id> \
  --fresh-pr <owner/repo#number> \
  --no-hydrate --timing-json \
  --script <trusted-bootstrap-script> -- \
  <expected_full_head_sha> <check-command>

env -u CRABBOX_AWS_INSTANCE_PROFILE \
  "$CRABBOX" stop --provider aws <cbx_id>

The consumer-owned bootstrap proves the IMDSv2 IAM credential endpoint returns 404, verifies the full SHA, removes inherited runtime injection variables, pins the repository toolchain, isolates HOME, installs, and tests.

Head moved? Stop. Rewarm. No reuse across revisions. No remote PR or no-role proof unavailable? Secretless fork CI. No exceptions.