Compass

You orchestrate the COMPASS robot navigation training pipeline — from scene search to trained policy. Adapt to context: interactive and explanatory during setup, autonomous and efficient for repeated operations like training runs.

COMPASS trains cross-embodiment navigation policies using residual RL on top of X-Mobility (a pretrained VLA). SAGE-10k provides 10,000 pre-generated indoor scenes across 50 room types.

Skill Scripts

This skill bundles two helper scripts in its own directory:

• scripts/sage10k_search.py — search SAGE-10k dataset by text query
• scripts/sage10k_to_usd.py — convert SAGE-10k scenes to USD (uses only Isaac Sim native APIs, no extra deps)

Use the full path when invoking them:

<SKILL_BASE_DIR>/scripts/sage10k_search.py
<SKILL_BASE_DIR>/scripts/sage10k_to_usd.py

Workflow Routing

If the user gives a scene description like "a cluttered warehouse" or "bedroom", treat it as Full Pipeline using SAGE-10k search. Use local SAGE only when the user explicitly asks for it.

Before any of Train, Evaluate, or Full Pipeline, run the Prerequisites Check first. The cost of a stale check is a confusing CUDA error 30 minutes into a training run.

Mandatory visual-verification gates

The Full Pipeline has two visual checkpoints that you MUST NOT skip, even when the user has told you to work without clarifying questions:

1. USD viewer check after conversion (see Step 5: Verify USD in Isaac Sim viewer) — open the headed Isaac Sim viewer on the converted USD and wait for the user to confirm in writing ("looks ok" / "looks good" / a description of any issue) before doing scene registration.
2. --viz kit preview training (see Train → Step 1: Preview training) — launch with num_envs=1 --viz kit so the Kit window shows the robot in the scene, and wait for the user's confirmation before launching the full-scale headless run.

"Work without clarifying questions" applies to ambiguities about what to do (scene choice, embodiment, num_envs). It does not override these two human-eye verification handoffs. SAGE conversions and Isaac Lab spawn settings can produce silently-broken scenes (rooms missing 3 of 4 walls, robot spawned outside the room, scene upside-down from wrong quaternion) that no programmatic check catches — only a human watching the viewer does. Skipping either step usually costs hours of wasted GPU on a broken scene.

Specialty skills

Some user intents are better handled by sibling skills. When the user's task fits one of these, say so to the user and recommend the matching specialty — don't try to handle it inside compass:

Tell the user the matching specialty and suggest they invoke it (or rephrase their ask so the auto-router picks it up). Don't programmatically invoke the sibling via the Skill tool — that adds latency and removes the user's ability to redirect.

Prerequisites Check

Run this before any operation. The skill assumes the user has run source ./docker/activate (the docker-as-venv shim from the docker/ subdir). Run all checks in parallel where possible. Report what's ready and what's missing.

# Container running?
./docker/run.sh status

# Activated shell? `deactivate` is a shell function defined by ./docker/activate
command -v deactivate >/dev/null && echo "shell: activated" || echo "shell: NOT activated — run: source ./docker/activate"

# GPU (use dangerouslyDisableSandbox: true)
nvidia-smi --query-gpu=name,memory.total,memory.used --format=csv,noheader

# Assets present
test -f ./assets/x_mobility.ckpt && echo "x_mobility ckpt: OK ($(du -h ./assets/x_mobility.ckpt | cut -f1))"
ls ./assets/usd/ 2>/dev/null | head -5

If anything fails, route to Setup COMPASS. If the user reports a vague "training won't start" issue, that's compass-doctor's job — recommend it.

Execution Environment Rules

These rules apply to every command. Each one exists for a specific reason; the explanation matters more than the rule itself, because edge cases need judgment.

1. Activated-shell rule. Every Python invocation runs inside a shell where the user has already done source ./docker/activate. Inside that shell, python, pip, tensorboard, etc. are shims that route into the COMPASS container automatically. The right command is just python run.py … — no Isaac Lab launcher prefix, no conda env wrapper. If command -v deactivate returns nothing, the shell isn't activated — pause and ask the user to source the activate script before continuing.

2. GPU access needs dangerouslyDisableSandbox: true. The Claude Code sandbox blocks NVIDIA driver access — without this flag, nvidia-smi fails and Isaac Sim can't find CUDA. This is a Claude Code concern, not a container concern, so it applies even though the GPU work happens inside the container.

3. Background execution for long jobs. Training and evaluation can run for hours. Use run_in_background: true so the user isn't blocked. To check progress, look at log files instead of stdout (the shim forwards stdout but it's easy to miss when the run is backgrounded):

   # Find latest training log
   ls -t /tmp/isaaclab/logs/ | head -1
   # Or check the Kit log for errors
   find ~/.local/share/ov/pkg/isaac-sim-* -name "kit_*.log" 2>/dev/null | head -1 | xargs tail -100

4. Killing processes. kill -9 <PID> of the python process. The shim forwards SIGTERM cleanly into the container, no orphan-conda-wrapper issues to worry about:

   ps aux | grep "run.py.*<scene>" | grep -v grep | awk '{print $2}' | xargs kill -9

Setup COMPASS

The new dev environment uses Docker as a venv: build the image once, download assets once, then activate. Three commands:

export HF_TOKEN=hf_xxx                  # https://huggingface.co/settings/tokens
./docker/run.sh build                   # ~10 min cold build of compass-rl image
./docker/run.sh assets                  # ~5 min: downloads compass_usds + x_mobility ckpt
source ./docker/activate                # prompt becomes (compass-rl)

After this, python run.py … (and pip, tensorboard, etc.) Just Work — they route to the container via docker exec. No conda env, no manual pip-install of requirements, no Isaac Lab clone. Assets land at ./assets/usd/ (built-in scenes) and ./assets/x_mobility.ckpt (base policy), bind-mounted into the container at the same paths under /workspace/COMPASS/.

Full reference: docs/handbook/installation/docker.md. If the user wants the bare-metal install (no Docker), point them at docs/handbook/installation/bare-metal.md — supported but slower to set up and not the recommended default.

Search SAGE-10k (recommended for new scenes)

The SAGE-10k dataset (nvidia/SAGE-10k) has 10,000 pre-generated indoor scenes across 50 room types. No SAGE installation needed.

Step 1: Search for matching scenes

python <SKILL_BASE_DIR>/scripts/sage10k_search.py "<USER_PROMPT>" --top 5 --sample-size 50

First run builds a scene index from the HF API (cached afterward). Results show room type, style, object count, and description.

Step 2: Present options to user

Show top results. Let user pick one.

Step 3: Download the selected scene

huggingface-cli download nvidia/SAGE-10k --repo-type dataset \
  --include "scenes/<zip_filename>" --local-dir ./sage_10k_cache

mkdir -p ./sage_10k_scenes/<scene_name>
unzip ./sage_10k_cache/scenes/<zip_filename> -d ./sage_10k_scenes/<scene_name>/

Step 4: Convert to USD

The bundled converter only needs Isaac Sim native APIs, no extra deps:

# dangerouslyDisableSandbox: true (Isaac Sim needs GPU)
python <SKILL_BASE_DIR>/scripts/sage10k_to_usd.py \
  ./sage_10k_scenes/<scene_name>/<layout_json> \
  ./compass/rl_env/exts/mobility_es/mobility_es/usd/<scene_name>/<scene_name>.usd

The output USD lives under the mobility_es extension directory because that's where environments.py expects to find it (the registered USD_PATHS dict resolves ../usd/<scene>/... relative to the config/ subdir). Built-in COMPASS scenes from ./assets/usd/ are referenced separately; user-added SAGE scenes go here.

The script:

• Initializes Isaac Sim headless for pxr access
• Parses PLY meshes with proper binary format (separate vertex/texcoord elements)
• Scales objects to match layout dimensions
• Creates walls and floors from room geometry
• Applies textures with UV mapping
• Sets all objects as static collision (appropriate for navigation training)

Step 5: Verify USD in Isaac Sim viewer (MANDATORY GATE)

This step is not optional and cannot be skipped, including under "no clarifying questions" mode — see Mandatory visual-verification gates.

Launch the Isaac Sim viewer on the converted USD so the user can verify the scene looks correct (geometry, collisions, textures). Use dangerouslyDisableSandbox: true and run_in_background: true:

python -c "
from isaacsim import SimulationApp
app = SimulationApp({'headless': False, 'width': 1280, 'height': 720})
import omni
omni.usd.get_context().open_stage('./compass/rl_env/exts/mobility_es/mobility_es/usd/<scene_name>/<scene_name>.usd')
while app.is_running():
    app.update()
app.close()
" &

Then:

1. Tell the user the viewer is open on display :1 and to inspect it.
2. Suggest enabling Show → Physics → Colliders to verify collision meshes.
3. Stop and wait for the user's explicit confirmation. Do not proceed to Step 6 until they reply with "looks ok"/"looks good" (or describe an issue to fix).

If they describe an issue, diagnose it (re-convert, adjust converter, etc.) and re-open the viewer for another round of verification. Repeat until they confirm.