Tao Run Deft Aoi

Skill: tao-run-deft-aoi

Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery).

Execution Contract

Treat this as a disk-backed state machine, not as a prose recipe.

1. Preserve every explicit user value. epoch 1 means num_epochs=1 and iteration 1 means max_iterations=1; a heuristic or spec default applies only when the user did not supply that parameter. Show the source of every run parameter (user, spec, or default) in the Pre-Flight Summary. Preserve the customer's metric name, operator, target, unit, evaluator, and constraints. The approved metric_contract is the source of truth for evaluation, checkpoint selection, completion, and reporting.
2. After the user approves the Summary, set PYTHON=$(bash scripts/deft_python.sh), then initialize deft_state.json once with "$PYTHON" scripts/init_deft_state.py, passing Preflight's exact GPU model/memory, resolved --network-mode, activation source, and selected absolute Python. The resulting execution_policy is immutable run state. Never hand-author or reinitialize it on resume.
3. On startup, after context compaction, before every stage, and before any completion claim, run "$PYTHON" scripts/deft_context.py --state ... --stage .... Use its durable next_stage plus the state file's status, current_iteration, iterations.*.status, stage_completed, and latest events entry to resume. Do not infer progress from assistant prose or from an artifact that is not recorded in state.
4. Invoke the mapped underlying skill after reading the DEFT overlay. Do not replace a missing/unread stage reference or a failed skill call with guessed shell commands, inline Python, a different output tree, or data fabricated from the KPI set.
5. After initialization, run install/fetch/login/container commands through "$PYTHON" scripts/deft_exec.py --state ... -- <command>. Air-gap mode rejects egress and installs, injects offline flags, and enforces no-pull. Selected platforms must enforce the equivalent policy.
6. Commit every stage with "$PYTHON" scripts/commit_stage.py; it verifies the stage's required inputs and atomically updates both the resume snapshot and ordered events array inside deft_state.json. Never edit the state file with inline Python, jq, heredocs, or an editor. Fix rejected evidence; never fabricate state. For evaluate, pass the metric result, checkpoint, inference CSV, and threshold directly to commit_stage.py. Pass positive measured --duration-sec from backend elapsed time or a host timer for executed stages. A documented --skip may record 0; negative durations are always rejected.
7. Claim the loop complete only after "$PYTHON" scripts/finalize_run.py creates the handoff artifacts, successfully commits loop_stop, and a fresh read of deft_state.json shows status == "complete", iterations.baseline.status == "complete", and the final iteration's status == "complete". A checkpoint, inference CSV, report, or assistant message is not completion evidence by itself.

Context Discipline

• Load references just in time. Re-read state, then read only the current stage's named section and act. Never preload/cat every reference or underlying skill, recursively list the skill tree, or re-read a reference already present in the current context.
• Redirect verbose train, inference, Docker, and SDG output to files. Inspect at most the final 40 lines or a one-line status/artifact check; never print a full spec, state file, loop log, or generated script into the conversation.
• A Skill-tool call loads stage instructions; it does not start a background orchestrator. Continue the documented stage in the parent immediately after it returns. Never sleep or poll waiting for a Skill-tool process. For actual background Docker work, save the PID and poll at intervals no longer than 30s.
• At the start of Pre-Flight, resolve network mode before dependencies. Read exactly one branch: references/air-gap.md for air-gap mode or references/network-bootstrap.md for network-enabled mode. Never load the network bootstrap in an air-gapped run.

When to Use This Skill

Use this skill when the user wants an agent to run the full DEFT AOI improvement loop for an NVIDIA TAO VisualChangeNet / ChangeNet PCB inspection model: baseline evaluation, RCA, synthetic defect generation, data mining, retraining, and deployment gating until a KPI target is met.

• "Run the DEFT loop"
• "Fine-tune until the configured quality metric meets its target"
• "Optimize a customer-defined metric while preserving its constraints"
• "Improve my AOI ChangeNet model using RCA and synthetic defects"
• "Iterate training until the deployment KPI meets the target"

Do not use this skill for a single standalone TAO training run, one-off inference, generic anomaly generation, or RCA-only analysis. Use the relevant agent directly when the user asks for only that step.

Base Model

The loop uses NVIDIA TAO Visual ChangeNet classify with either end-to-end C-RADIOv2-B or a frozen DINOv3 backbone. specs/baseline_spec.yaml defines the architecture. Backbone variants, staging, HF_TOKEN, and mount rules are owned by references/visual-changenet.md; the spec always points to a local mounted file. NGC_KEY gates container pulls. SigLIP mining is owned by references/tao-mine-aoi-images.md; AnomalyGen assets and network/air-gap rules are owned by references/tao-generate-anomalies.md and references/air-gap.md. The container owns its base-asset list; this workflow keeps bootstrap on Text2Image 2B by passing --model_sizes 2B explicitly.

Train AutoML Policy

DEFT AOI owns the iterative data-improvement loop, retraining cadence, and KPI checkpoint selection. For this workflow only, bypass model-level AutoML even when the underlying Visual ChangeNet model metadata has automl_enabled: true.

automl_policy: off is a workflow argument to the Visual ChangeNet skill invocation (the value the parent passes when calling tao-skill-bank:tao-train-visual-changenet via the Skill tool), not a TAO spec field. Two cases:

• Direct docker run visual_changenet train -e <spec> (the path this workflow actually uses inline): no action needed. The TAO entrypoint is plain training by default; AutoML lives behind a different code path that the SDK orchestrates. Effectively, every direct docker run is already automl_policy: off.
• SDK-orchestrated dispatch (Brev/SLURM/k8s with the SDK building the command): pass automl_policy: off to VisualChangeNetSDK.train(...) or the equivalent runner argument. The SDK uses it to pick the plain-train command instead of the AutoML wrapper.

Never add automl_policy or a workflow key to the spec YAML. TAO's Hydra ExperimentConfig schema does not recognize these keys and the train job fails at config-merge time with Error merging '<spec>.yaml' with schema: Key 'workflow' not in 'ExperimentConfig'. This is a workflow-level override only; do not change model metadata, and do not apply this policy to other workflows.

Launch Intake

After the user confirms they want to run this workflow, ask which supported platform they intend to run on. Discover the execution platforms from the installed platform skills (tao-run-on-docker / -slurm / -kubernetes / -brev, plus any external one); on a runtime that surfaces only the core router skills, read skills/platform/tao-run-on-*/SKILL.md frontmatter.

After platform selection, read the chosen platform skill's ## Credentials section and references/skill_info.yaml (required_credentials / credential_groups).

Never ask for or print credential values. Check only whether the variable is set ([ -n "$VAR" ] && echo SET || echo UNSET); if unset, name it so the user can export it or put it in a user-approved env file (~/.tao/secrets.env, ~/.config/tao/.env, or one they point at), loaded with set -a; source /path/to/.env; set +a. The run never creates or writes that file.

Agent Behavior

There is exactly one user gate: pre-flight confirmation. Print the Pre-Flight Summary (see references/preflight.md → Pre-Flight Summary), then STOP and wait for the user to type "go", "yes", "looks good", or similar explicit approval. Do not launch any side-effecting step (docker run, training, SDG, mutations under ${RESULTS_DIR}/) before that approval — reading specs, listing files, docker image inspect, and populating the summary table are fine. "Autonomous" describes behavior after this gate, not before it. Do not skip the gate even if the user's original prompt sounded urgent ("just run it", "go ahead") — the summary itself is the artifact they need to see before approving.

After the gate, the skill is fully autonomous. Run the entire loop without asking for confirmation. Do not pause between steps. Do not ask "want me to continue?" — just continue. Only stop if a step fails with an unrecoverable error or a hard-stop gate fires. Print a one-line status update at each step milestone so the user can follow progress.

Auto-mode required. The post-gate loop fires constant side-effecting calls (docker run, ${RESULTS_DIR}/ writes); without auto-accept / bypass-permissions mode it stalls on the first prompt. Remind the user at the Pre-Flight Summary to enable auto-mode (shift+tab) before approving.

Blocker recovery. Before the user gate, select a complete installed host interpreter through deft_python.sh. If none exists, follow only the already-selected network-mode reference. Air-gap mode hard-stops without a package-manager command; network-enabled bootstrap is isolated in references/network-bootstrap.md. Apply the network-mode branches in references/air-gap.md; record permitted fetches and directory creation as post-approval work, or validate staged assets in air-gap mode. After approval, fix recoverable blockers yourself, then resume the Pre-Flight step you were on (<blocker> cleared → resuming step N). Halt only for what you cannot fix (missing workspace/specs/CSVs/credentials, empty pool, leakage). A fix is not another user gate.

Non-zero command rule. Never repeat an unchanged failed command and never switch to an undocumented CLI/module path by trial and error. Read the final error block (not only the container banner), map it to the loaded stage reference/underlying skill, make one evidence-based correction, and rerun its documented verification. If the reference does not cover the failure, commit status=error and halt instead of improvising a reduced workflow.

Revised plan. If any run parameter changes after the original summary was shown (user imposes a time limit, overrides epochs, changes max_iterations, etc.), always re-run Pre-Flight and show an updated summary before proceeding.