Test Install

Verify that requirements/install.sh actually works for an embodied model/env: build its venv, confirm the e2e config's model paths exist, then run the matching CI e2e test against that venv. The harness is .agents/skills/test-install/driver.py — it reads the install command, env vars, and test config straight out of .github/workflows/embodied-e2e-tests.yml, so it never drifts from CI. Drive everything through that script.

All paths below are relative to the repo root (the dir with requirements/install.sh).

Prerequisites

This runs on the embodied CI runner (or an equivalent box): NVIDIA GPUs, the shared /workspace/dataset/ tree (models, LIBERO, etc.), uv, and a python3 that has PyYAML. No apt-get needed — the driver only orchestrates. Quick check:

nvidia-smi -L | head -1
ls -d /workspace/dataset >/dev/null && python3 -c "import yaml" && echo "env OK"

If python3 lacks PyYAML, the driver auto-reexecs under uv run --with pyyaml, so it works regardless.

Run (agent path)

The driver has seven subcommands. Start with the read-only ones (list, resolve, check-paths) — they're instant and tell you exactly what CI does before you spend an hour on an install.

# What model/env combos does CI cover, and which configs do they run?
python3 .agents/skills/test-install/driver.py list

# Show the exact install command + env vars + test configs for one combo:
python3 .agents/skills/test-install/driver.py resolve gr00t_n1d6 maniskill_libero

# Do the model/checkpoint paths an e2e config needs actually exist on disk?
python3 .agents/skills/test-install/driver.py check-paths libero_spatial_ppo_gr00t_n1d6

# Same check across every e2e config at once (great pre-flight / PR check):
python3 .agents/skills/test-install/driver.py check-all

check-paths classifies every absolute path in the config: [model/input] (must exist — a missing one fails the run before training and returns exit 1), [output dir] (created by the run, may be missing), [path] (informational).

The full pipeline

run does install → check-paths (per config) → test, stopping a test whose required model paths are missing. Always preview with --dry-run first — it prints the exact shell (install line + CI test step) without executing:

python3 .agents/skills/test-install/driver.py run gr00t_n1d6 maniskill_libero \
    --venv /workspace/test-venvs/gr00t_n1d6 --dry-run

Drop --dry-run to actually build and test. Or drive the two halves separately (faster iteration — install once, test many):

# Preview the install (env vars + install.sh line, with --venv and --use-mirror
# injected). Drop --dry-run to build the venv; add --no-mirror to skip mirrors:
python3 .agents/skills/test-install/driver.py install gr00t_n1d6 maniskill_libero \
    --venv /workspace/test-venvs/gr00t_n1d6 --dry-run

# Run the matching e2e test against any venv (verbatim CI test step, venv path
# swapped in). This one is cheap — dummy SAC, 2 epochs — so run it for real:
python3 .agents/skills/test-install/driver.py test realworld_dummy_sac_cnn \
    --venv /opt/venv/openvla

A full model install (e.g. gr00t_n1d6) is heavy — it clones repos and builds flash-attn — so preview with --dry-run, then run it where you can afford the time. The dummy-SAC test above completes in a few minutes against the prebuilt /opt/venv/openvla.

A real test run spins up Ray, the env/rollout/policy workers, and trains for the config's (deliberately tiny) epoch count. The dummy-SAC smoke above finishes in a few minutes and writes TensorBoard output under the config's log_path (/workspace/results/<config>/tensorboard/) — that directory appearing with fresh files is your "it worked" signal.

Clean up the venv when you're done

A test venv is heavy (e.g. gr00t_n1d6 is ~600M) and is throwaway — it exists only to prove the install + e2e work. After the test finishes, rm -rf the --venv path you built unless the user asked to keep it for reuse:

rm -rf /workspace/test-venvs/<model>

Don't touch the shared caches (/workspace/dataset/.uv, .uv_cache) or the prebuilt /opt/venv/* — only the per-test venv you created. Cleaning up keeps /workspace/test-venvs/ from accumulating stale multi-hundred-MB trees across runs. (Cleanup is for the venv only; the /workspace/results/<config>/ outputs are your proof the test ran — leave them or mention them.)

When there's no matching test

If a model/env has an install but no e2e job in the workflow, run installs and then tells you there's nothing to run — ask the user what to run rather than guessing. If you have a config name that isn't wired into CI, test <config> --runner run|run_async|run_offline runs it directly (pick the runner; run is the default).

Verifying a brand-new model/env (the common case)

When someone adds install_<model>_model() + an e2e job (see the add-install-docker-ci-e2e and install-check skills), confirm it end to end:

# 1. CI parsed it correctly and the install command looks right:
python3 .agents/skills/test-install/driver.py resolve <model> <env>
# 2. The SFT checkpoint the e2e config points at is actually on this box:
python3 .agents/skills/test-install/driver.py check-paths <its_config>
# 3. Full build + test (preview with --dry-run, then drop it to run for real):
python3 .agents/skills/test-install/driver.py run <model> <env> \
    --venv /workspace/test-venvs/<model> --dry-run

If step 2 reports MISS [model/input], the install can be perfect and the e2e will still fail — the dataset/checkpoint just isn't staged on this runner. Surface that to the user; it's not an install bug.