Tao Train Visual Changenet

Visual ChangeNet

Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery).

Visual ChangeNet is a TAO Toolkit model for visual inspection and defect detection. It supports two tasks:

• Classify — Binary image classification using a siamese-style architecture with a shared backbone (C-RADIO ViT) and a learnable difference module. Compares image pairs to classify defects as PASS/NO_PASS.
• Segment — Pixel-level change segmentation using a ViT-Large NVDINOv2 backbone. Compares before/after image pairs to produce a binary change mask.

Classify supports the public C-RADIOv2-B backbone and six frozen DINOv3 variants. Read references/dinov3-backbones.md before selecting DINOv3; it contains the exact variant map, freeze requirement, Hugging Face access rules, and local-staging overlay. For C-RADIO, use the bundled scripts/stage_backbone.py and the mount in references/local-docker.md.

Segment specs use model.backbone.type: vit_large_nvdinov2 and the NVDINOv2 checkpoint family. Keep the checkpoint architecture aligned with the backbone type: NV_DINOV2_518_16_256.ckpt is compatible with the packaged segment templates, but it must not be used with fan_small_12_p4_hybrid. If you switch to a different segment backbone, use a matching checkpoint or leave model.backbone.pretrained_backbone_path empty for default initialization.

Dataclass Schemas

Generated TAO Core schemas are packaged in schemas/<action>.schema.json, with schemas/manifest.json listing available actions. Each generated schema also emits references/spec_template_<action>.yaml from the schema top-level default field. AutoML enablement is declared at the model layer in references/skill_info.yaml via automl_enabled. Runnable AutoML still requires schemas/train.schema.json and references/spec_template_train.yaml to exist and parse. Use the packaged train schema for automl_default_parameters, automl_disabled_parameters, defaults, min/max bounds, enums, option weights, math conditions, dependencies, and popular parameters. Do not expect ~/tao-core at runtime; maintainers regenerate schemas/templates before packaging the skill bank.

Train Action Policy

This model is AutoML-enabled at the model layer. Before handling any train-stage request, read references/skill_info.yaml and resolve the run override from either an explicit automl_policy value or the user's workflow request. Use automl_policy: on by default and only expose on / off in new launch prompts. Treat phrases like "turn off AutoML", "disable AutoML", "no HPO", or "plain training" as automl_policy: off for this run only. When automl_policy: on, automl_enabled: true, and both schemas/train.schema.json and references/spec_template_train.yaml are packaged, route the train action through tao-skill-bank:tao-run-automl by default with this model's skill_dir. Preserve workflow/application overrides for datasets, specs, output directories, GPU/platform settings, parent checkpoints, and automl_policy. Use direct model training only when automl_policy: off or the packaged train schema/template is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.

Checkpoint retention is an orchestration policy, not an HPO parameter. Both packaged train templates default train.checkpointer.enable_topk and train.checkpointer.replace_periodic to false, preserving periodic saves controlled by train.checkpoint_interval. When AutoML checkpoint retention is enabled, the AutoML runner sets both flags to true, monitors val_loss in min mode, and uses save_top_k: 1; this replaces the periodic series with the single best checkpoint. When AutoML checkpoint retention is disabled, leave the bounded-retention overrides unset so periodic checkpoint behavior remains.

Non-train actions declared by this model skill (evaluate, inference, export, quantize, segment_evaluate, and segment_inference) stay in this model skill. Do not present segment_export or segment_quantize as runnable parent-skill actions until matching entries are packaged in schemas/manifest.json. Prune and retrain are not declared in the current parent references/skill_info.yaml; do not present them as runnable parent-skill actions unless the metadata is extended with matching action wiring and schemas. The per-run automl_policy override does not change model metadata.

For TAO Deploy TensorRT actions (gen_trt_engine, TensorRT evaluate, and TensorRT inference for classify and segment variants), read references/tao-deploy-visual-changenet.md first. Deploy spec templates live in this skill's references/ folder with the spec_template_deploy_*.yaml prefix. Deploy requires an exported ONNX artifact as parent_model. If no ONNX artifact exists and the main skill does not expose an export action, report deploy as blocked instead of inventing an artifact.

Training Requirements

Visual ChangeNet has two separate task modes with different dataset types and data source structures.

Classify

• Dataset type: visual_changenet_classify
• Formats: default
• Accepted dataset intents: training, evaluation, testing, calibration
• Monitoring metric: val_loss
• Standalone evaluate metric: test_acc (with test_fpr, test_fnr, and defect_acc also emitted). AutoML must rank recommendations by the training val_loss; use test_acc only to verify that the selected checkpoint loads and evaluates successfully. Do not expect the evaluate action to emit val_loss.

Per-Action Dataset Requirements (Classify)

The quantize and gen_trt_engine rows below describe TAO spec data requirements only. They are not parent-skill actions unless the corresponding action is declared in references/skill_info.yaml or deploy/skill_info.yaml.

Segment

• Dataset type: visual_changenet_segment
• Formats: default
• Accepted dataset intents: training, calibration
• Monitoring metric: val_loss

Segment uses a paired directory structure (A/, B/, list/, label/) instead of CSV + images. The root_dir spec key points to the top-level directory containing all four subdirectories.

Required files per dataset: A.tar.gz, B.tar.gz, list.tar.gz, label.tar.gz

Per-Action Dataset Requirements (Segment)

The quantize and gen_trt_engine rows below describe TAO spec data requirements only. They are not parent-skill actions unless the corresponding action is declared in references/skill_info.yaml or deploy/skill_info.yaml.