Official agent skill

Tao Train Sparse4d

Sparse4D for multi-camera temporal 3D object detection and tracking.

OfficialApache-2.0Auto-check: notesAI & LLM Engineering

Install Tao Train Sparse4d

skills CLI

$ npx skills add NVIDIA/skills --skill tao-train-sparse4d -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI

$ gh skill install NVIDIA/skills tao-train-sparse4d --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/tao-train-sparse4d .claude/skills/tao-train-sparse4d && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Paste into Claude Code

Install the "tao-train-sparse4d" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/tao-train-sparse4d into .claude/skills/tao-train-sparse4d/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "tao-train-sparse4d", then confirm the skill loads.

Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

$skill-installer

$skill-installer install https://github.com/NVIDIA/skills/tree/main/skills/tao-train-sparse4d

Type this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.

skills CLI

$ npx skills add NVIDIA/skills --skill tao-train-sparse4d -a codex

Project install goes to .agents/skills/; add -g for ~/.codex/skills/.

GitHub CLI

$ gh skill install NVIDIA/skills tao-train-sparse4d --agent codex

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/tao-train-sparse4d .agents/skills/tao-train-sparse4d && rm -rf skills-src

Use ~/.agents/skills/ instead of .agents/skills for a personal install.

Codex skills documentation · loads skills from .agents/skills/

Paste into Codex

Install the "tao-train-sparse4d" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/tao-train-sparse4d into .agents/skills/tao-train-sparse4d/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "tao-train-sparse4d", then confirm the skill loads.

Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add NVIDIA/skills --skill tao-train-sparse4d -a cursor

Project install goes to .agents/skills/; add -g for ~/.cursor/skills/.

GitHub CLI

$ gh skill install NVIDIA/skills tao-train-sparse4d --agent cursor

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/tao-train-sparse4d .cursor/skills/tao-train-sparse4d && rm -rf skills-src

Use ~/.cursor/skills/ instead of .cursor/skills for a personal install.

Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/

Paste into Cursor

Install the "tao-train-sparse4d" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/tao-train-sparse4d into .cursor/skills/tao-train-sparse4d/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "tao-train-sparse4d", then confirm the skill loads.

Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

gemini skills install

$ gemini skills install https://github.com/NVIDIA/skills.git --path skills/tao-train-sparse4d

--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.

skills CLI

$ npx skills add NVIDIA/skills --skill tao-train-sparse4d -a gemini-cli

Project install goes to .agents/skills/; add -g for ~/.gemini/skills/.

GitHub CLI

$ gh skill install NVIDIA/skills tao-train-sparse4d --agent gemini-cli

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/tao-train-sparse4d .gemini/skills/tao-train-sparse4d && rm -rf skills-src

Use ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.

Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/

Paste into Gemini CLI

Install the "tao-train-sparse4d" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/tao-train-sparse4d into .gemini/skills/tao-train-sparse4d/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "tao-train-sparse4d", then confirm the skill loads.

Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

GitHub CLI

$ gh skill install NVIDIA/skills tao-train-sparse4d

Installs for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).

skills CLI

$ npx skills add NVIDIA/skills --skill tao-train-sparse4d -a github-copilot

Project install goes to .agents/skills/; add -g for ~/.copilot/skills/.

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/tao-train-sparse4d .github/skills/tao-train-sparse4d && rm -rf skills-src

Use ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.

GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/

Paste into GitHub Copilot

Install the "tao-train-sparse4d" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/tao-train-sparse4d into .github/skills/tao-train-sparse4d/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "tao-train-sparse4d", then confirm the skill loads.

GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add NVIDIA/skills --skill tao-train-sparse4d -a opencode

OpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.

GitHub CLI

$ gh skill install NVIDIA/skills tao-train-sparse4d --agent opencode

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/tao-train-sparse4d .opencode/skills/tao-train-sparse4d && rm -rf skills-src

Use ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.

OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/

Paste into OpenCode

Install the "tao-train-sparse4d" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/tao-train-sparse4d into .opencode/skills/tao-train-sparse4d/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "tao-train-sparse4d", then confirm the skill loads.

OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

Facts

Skill name: tao-train-sparse4d
GitHub stars: 3.5k
Token cost: ~3.8k tokens
SKILL.md length: 1,238 words
Files: 24 (incl. scripts, references)
Skills in repo: 380
Repo updated: Oct 7, 2026
First seen: Oct 7, 2026
Licence: Apache-2.0

At a glance

Sparse4D for multi-camera temporal 3D object detection and tracking.

Running inference for a TAO Sparse4D model
SKILL.md covers Dataclass Schemas, Train Action Policy, Training Requirements and Eval Dataset, plus 5 more sections
Phrases include train Sparse4D
Multi-camera 3D detection

What it does

Tao Train Sparse4d is an agent skill from NVIDIA/skills, published by the product's own GitHub organization. Sparse4D for multi-camera temporal 3D object detection and tracking. Uses sparse queries with deformable attention across camera views and time for end-to-end 3D perception, with an instance bank for temporal tracking. Use when training, evaluating, exporting, quantizing, or running inference for a TAO Sparse4D model. Trigger phrases include "train Sparse4D", "multi-camera 3D detection", "temporal 3D tracker", "sparse query 3D perception".

Its SKILL.md is about 3.8k tokens, which your agent loads only when the skill is triggered. The skill folder holds 27 other files, including scripts and reference files (for example `BENCHMARK.md`, `config/skillspector-baseline.yaml` and `evals/evals.json`). Compatibility notes: Requires docker + nvidia-container-toolkit.

It sits in AI & LLM Engineering, covering Computer vision. The repository describes itself as: Agent Skills for NVIDIA products — install into Claude Code, Codex, and other coding agents to run Physical AI, robotics, simulation, CUDA, and RAG workflows end to end. The licence is Apache-2.0.

When your agent uses it

Running inference for a TAO Sparse4D model
Phrases include train Sparse4D
Multi-camera 3D detection
Temporal 3D tracker

Example prompts

“train Sparse4D”
“multi-camera 3D detection”
“temporal 3D tracker”
“/tao-train-sparse4d”

Requirements

Python 3
Docker
Compatibility (from SKILL.md): Requires docker + nvidia-container-toolkit.
Pre-approved tools (allowed-tools): Read, Bash

What it can do on your machine

Read from SKILL.md and the folder at commit 67a13c0. It shows what the files ask for, not the result of running them.

Tool permissions
Pre-approves these tools, so the agent can use them without asking each time:
- Read
- Bash
From allowed-tools in the SKILL.md frontmatter.
Runs code
Ships 1 file in scripts/, which the agent can run.
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Compatibility
Requires docker + nvidia-container-toolkit.
From compatibility in the SKILL.md frontmatter.

Context cost

Tao Train Sparse4d loads about 3.8k tokens when it runs, and up to ~16k if it reads all its reference files. Until then it costs about 116 tokens; SKILL.md has 1,238 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it: ~116
When it runs · the whole SKILL.md, loaded when a task matches: ~3.8k
With references · SKILL.md plus every file in references/, read only if the agent opens them: ~16k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check: notes

The automated check noted patterns worth knowing about, such as sudo or a known installer.

NotePre-approves every shell command (allowed-tools: Bash)SKILL.md
```
allowed-tools: Read, Bash
```

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from NVIDIA/skills at commit 67a13c0, republished under its Apache-2.0 licence (© NVIDIA). 1,238 words, ~3,762 tokens.

Download SKILL.mdSave it as .claude/skills/tao-train-sparse4d/SKILL.md (or your agent's skills folder). This skill also uses 23 other files; get the full folder from GitHub.

name: tao-train-sparse4d
description: Sparse4D for multi-camera temporal 3D object detection and tracking. Uses sparse queries with deformable attention across camera views and time for end-to-end 3D perception, with an instance bank for temporal tracking. Use when training, evaluating, exporting, quantizing, or running inference for a TAO Sparse4D model. Trigger phrases include "train Sparse4D", "multi-camera 3D detection", "temporal 3D tracker", "sparse query 3D perception".
allowed-tools: Read, Bash
compatibility: Requires docker + nvidia-container-toolkit.
license: Apache-2.0
metadata.version: 0.1.0
metadata.author: NVIDIA Corporation
tags: temporal, 3d, detection, tracking

Sparse4D

Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery).

Sparse4D for multi-camera temporal 3D object detection and tracking. Uses sparse queries with deformable attention across camera views and time for end-to-end 3D perception. Includes instance bank for temporal tracking.

Use a pretrained ResNet-101 backbone when one is available by setting train.pretrained_model_path. For local smoke validation, Sparse4D training can run with an empty train.pretrained_model_path, but production runs should still use a compatible PTM.

Dataclass Schemas

Generated TAO Core schemas are packaged in schemas/<action>.schema.json, with schemas/manifest.json listing available actions. Each generated schema also emits references/spec_template_<action>.yaml from the schema top-level default field. AutoML enablement is declared at the model layer in references/skill_info.yaml via automl_enabled. Runnable AutoML for an action requires schemas/<action>.schema.json and references/spec_template_<action>.yaml to exist and parse. Use the packaged selected-action schema for automl_default_parameters, automl_disabled_parameters, defaults, min/max bounds, enums, option weights, math conditions, dependencies, and popular parameters. Do not expect ~/tao-core at runtime; maintainers regenerate schemas/templates before packaging the skill bank.

Train Action Policy

This model is AutoML-enabled at the model layer. Before handling any train-stage request, read references/skill_info.yaml and resolve the run override from either an explicit automl_policy value or the user's workflow request. Use automl_policy: on by default and only expose on / off in new launch prompts. Treat phrases like "turn off AutoML", "disable AutoML", "no HPO", or "plain training" as automl_policy: off for this run only. When automl_policy: on, automl_enabled: true, and both schemas/train.schema.json and references/spec_template_train.yaml are packaged, route the train action through tao-skill-bank:tao-run-automl by default with this model's skill_dir. Preserve workflow/application overrides for datasets, specs, output directories, GPU/platform settings, parent checkpoints, and automl_policy. Use direct model training only when automl_policy: off or the packaged train schema/template is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.

Non-train actions such as evaluate, inference, export, and deploy flows stay in this model skill. The per-run automl_policy override does not change model metadata.

Training Requirements

Dataset type: sparse4d
Formats: ovpkl
Monitoring metric: val_mAP
Current TAO Sparse4D training emits this value in status/logs as img_bbox_NuScenes/mAP and mAP; AutoML metric extractors should treat those emitted keys as aliases for val_mAP. Multi-fidelity AutoML algorithms such as Hyperband, ASHA, and BOHB may promote a checkpoint to a resume job that completes without emitting a fresh val_mAP alias. In that case, compare AutoML's carried metric to the source rung job that emitted img_bbox_NuScenes/mAP or mAP, while still verifying that the promoted job resumed from the explicit epoch/step checkpoint, produced a real checkpoint, and is usable for evaluate/inference.

Per-Action Dataset Requirements

Action	Spec Key	Source	Files	List?
dataset_convert	aicity.root	id		No
evaluate	dataset.data_root	eval_dataset	(from convert job, spec: aicity.split)	No
evaluate	model.head.instance_bank.anchor	train_datasets	/results/{dataset_convert_job_id}/anchor_init.npy	No
evaluate	dataset.train_dataset.ann_file	train_datasets	(from convert job, spec: aicity.split)	No
evaluate	dataset.val_dataset.ann_file	eval_dataset	(from convert job, spec: aicity.split)	No
evaluate	dataset.test_dataset.ann_file	inference_dataset	(from convert job, spec: aicity.split)	No
export	model.head.instance_bank.anchor	train_datasets	/results/{dataset_convert_job_id}/anchor_init.npy	No
inference	dataset.data_root	inference_dataset	(from convert job, spec: aicity.split)	No
inference	model.head.instance_bank.anchor	train_datasets	/results/{dataset_convert_job_id}/anchor_init.npy	No
inference	dataset.train_dataset.ann_file	train_datasets	(from convert job, spec: aicity.split)	No
inference	dataset.val_dataset.ann_file	eval_dataset	(from convert job, spec: aicity.split)	No
inference	dataset.test_dataset.ann_file	inference_dataset	(from convert job, spec: aicity.split)	No
quantize	dataset.data_root	train_datasets	(from convert job, spec: aicity.split)	No
quantize	model.head.instance_bank.anchor	train_datasets	/results/{dataset_convert_job_id}/anchor_init.npy	No
quantize	dataset.train_dataset.ann_file	train_datasets	(from convert job, spec: aicity.split)	No
quantize	dataset.val_dataset.ann_file	eval_dataset	(from convert job, spec: aicity.split)	No
quantize	dataset.test_dataset.ann_file	inference_dataset	(from convert job, spec: aicity.split)	No
quantize	dataset.quant_calibration_dataset.images_dir	train_datasets		No
train	dataset.data_root	train_datasets	(from convert job, spec: aicity.split)	No
train	model.head.instance_bank.anchor	train_datasets	/results/{dataset_convert_job_id}/anchor_init.npy	No
train	dataset.train_dataset.ann_file	train_datasets	(from convert job, spec: aicity.split)	No
train	dataset.val_dataset.ann_file	eval_dataset	(from convert job, spec: aicity.split)	No
train	dataset.test_dataset.ann_file	inference_dataset	(from convert job, spec: aicity.split)	No

Typical Spec Overrides

Data source overrides are mandatory for every action — the agent MUST construct data source paths from the Per-Action Dataset Requirements table above and include them in spec_overrides.

python

S3_TRAIN = "s3://bucket/data/train"
S3_EVAL = "s3://bucket/data/eval"
CONVERTED_SCENE = "<scene-from-converter>"  # e.g. "subsetscene+bev-sensor-random-0"

train (mandatory data sources):

python

CONVERTED = "s3://bucket/results/<dataset_convert_job_id>"
{
    "train.num_epochs": 30,
    "train.checkpoint_interval": 10,
    "train.validation_interval": 10,
    "train.num_gpus": 1,
    "dataset.sequences.split_num": 90,
    "dataset.train_dataset.sequences_split_num": 90,
    "dataset.data_root": f"{S3_TRAIN}/train",
    "model.head.instance_bank.anchor": f"{CONVERTED}/anchor_init.npy",
    "dataset.train_dataset.ann_file": f"{CONVERTED}/train/{CONVERTED_SCENE}_infos_train.pkl",
    "dataset.val_dataset.ann_file": f"{CONVERTED}/val/{CONVERTED_SCENE}_infos_val.pkl",
    "dataset.test_dataset.ann_file": f"{CONVERTED}/test/{CONVERTED_SCENE}_infos_test.pkl",
}

evaluate (mandatory data sources):

python

CONVERTED = "s3://bucket/results/<dataset_convert_job_id>"
{
    "dataset.data_root": f"{S3_EVAL}/val",
    "model.head.instance_bank.anchor": f"{CONVERTED}/anchor_init.npy",
    "dataset.train_dataset.ann_file": f"{CONVERTED}/train/{CONVERTED_SCENE}_infos_train.pkl",
    "dataset.val_dataset.ann_file": f"{CONVERTED}/val/{CONVERTED_SCENE}_infos_val.pkl",
    "dataset.test_dataset.ann_file": f"{CONVERTED}/test/{CONVERTED_SCENE}_infos_test.pkl",
}

export (mandatory data sources):

python

CONVERTED = "s3://bucket/results/<dataset_convert_job_id>"
{
    "model.head.instance_bank.anchor": f"{CONVERTED}/anchor_init.npy",
}

inference (mandatory data sources):

python

CONVERTED = "s3://bucket/results/<dataset_convert_job_id>"
{
    "dataset.data_root": f"{S3_EVAL}/test",
    "model.head.instance_bank.anchor": f"{CONVERTED}/anchor_init.npy",
    "dataset.train_dataset.ann_file": f"{CONVERTED}/train/{CONVERTED_SCENE}_infos_train.pkl",
    "dataset.val_dataset.ann_file": f"{CONVERTED}/val/{CONVERTED_SCENE}_infos_val.pkl",
    "dataset.test_dataset.ann_file": f"{CONVERTED}/test/{CONVERTED_SCENE}_infos_test.pkl",
}

quantize (mandatory data sources):

python

CONVERTED = "s3://bucket/results/<dataset_convert_job_id>"
{
    "dataset.data_root": f"{S3_TRAIN}/train",
    "model.head.instance_bank.anchor": f"{CONVERTED}/anchor_init.npy",
    "dataset.train_dataset.ann_file": f"{CONVERTED}/train/{CONVERTED_SCENE}_infos_train.pkl",
    "dataset.val_dataset.ann_file": f"{CONVERTED}/val/{CONVERTED_SCENE}_infos_val.pkl",
    "dataset.test_dataset.ann_file": f"{CONVERTED}/test/{CONVERTED_SCENE}_infos_test.pkl",
    "dataset.quant_calibration_dataset.images_dir": f"{S3_TRAIN}",
}

See references/local_docker_conversion.md for local-docker conversion roots and mounts, H5 depth-path normalization, converted annotation filenames, smoke-run max_num_cams/anchor contracts for export compatibility, and converted-artifact verification before train/evaluate/inference.

Eval Dataset

Optional. Val/test splits configured via dataset ann_file paths.

Show full SKILL.md (530 more words)Show less

Important Parameters

model.backbone: Backbone. Default resnet_101.
model.neck.out_channels: FPN output channels. Default 256. num_outs=4.
model.input_shape: Input image shape [W, H]. Default [1408, 512].
model.head.num_output: Number of detection output queries. Default 300.
model.head.num_decoder: Number of decoder layers. Default 6.
model.head.temporal: Enable temporal reasoning. Default True.
model.head.instance_bank.num_anchor: Instance bank anchors. Default 900.
model.head.instance_bank.num_temp_instances: Temporal instance count. Default 600.
model.depth_branch.loss_weight: Depth supervision loss weight. Default 0.2.
dataset.batch_size: Per-GPU batch size. Default 2.
dataset.num_frames: Sequence length. Default 200.
dataset.classes: Detection classes. Default [person, gr1_t2, agility_digit, nova_carter]. num_ids=70 for tracking.
train.optim.lr: Learning rate. Default 5e-5. img_backbone lr_mult=0.2.
train.lr_scheduler: Cosine scheduler with linear warmup (500 iters, ratio 0.333).
train.grad_clip.max_norm: Gradient clipping. Default 25.
train.precision: Options: bf16, fp16, fp32. Default bf16.
evaluate.metrics: Eval metrics. Default ["detection"]. Optional tracking evaluation.
evaluate.tracking.enabled: Enable tracking evaluation. tracking_threshold=0.2.

Multi-GPU / Multi-Node

Launch method: Lightning-managed (single python process, Lightning spawns workers).

Spec Key	Description	Default
`train.num_gpus`	Number of GPUs	1
`train.gpu_ids`	GPU device indices	[0]
`train.num_nodes`	Number of nodes	1

Multi-GPU strategy: ddp_find_unused_parameters_true (no fsdp support)
sync_batchnorm is always enabled (True)
Iterations per epoch computed as: num_frames * num_bev_groups / (num_nodes * num_gpus * batch_size)
Scaling: When increasing GPUs, effective batch size grows and iterations-per-epoch shrinks proportionally

Multi-node env vars (set by orchestrator): WORLD_SIZE, NODE_RANK, MASTER_ADDR, MASTER_PORT, NUM_GPU_PER_NODE.

Hardware

Minimum 2 GPU(s), recommended 8 GPU(s). 40GB+ (A100 recommended) VRAM per GPU. Multi-camera temporal model is memory intensive. bf16 required for practical training. Multi-GPU strongly recommended. Instance bank requires substantial memory for temporal reasoning.

Error Patterns

dataset_convert required: Must run dataset_convert first to produce annotation pickles and anchor_init.npy.

dataset_convert container/command: Sparse4D conversion is an AICity to OVPKL annotations conversion. Launch dataset_convert with the action-level tao_toolkit.data_services image and annotations convert -e {config_path}; do not use the PyTorch sparse4d CLI for conversion. Train/evaluate/export/ inference still use the model-level PyTorch image.

Stable raw-data path: The AICity to OVPKL converter writes image paths into the generated pickle files. Keep aicity.root at /data/aicity_root during conversion, then point dataset.data_root at the split folder, for example /data/aicity_root/train for training or /data/aicity_root/val for evaluation. This preserves the converter's absolute RGB paths and relative depth paths.

H5 depth tuple mismatch: If training fails with an H5 path error where the trainer tries to open a camera directory such as /data/aicity_root/train/<scene>/Camera, run models/tao-train-sparse4d/scripts/normalize_depth_paths.py --data-root <host-aicity-root>/train <converted-ann-dir> after dataset_convert and before train/evaluate/inference. The helper rewrites converted depth_map_path tuples to point at <scene>/depth_maps/<camera>.h5 with the H5 dataset key basename.

Missing anchor file: Set model.head.instance_bank.anchor to the anchor_init.npy path from dataset_convert results.

Temporal OOM: Reduce dataset.num_frames or dataset.batch_size if running out of memory during temporal training.

Quantize image compatibility: The model-skill wiring should pass quantize.model_path through the parent-model resolver, and checkpoint handoff should select the exact epoch/step checkpoint just like evaluate, inference, export, and resume. TorchAO checkpoint quantization passes in the validation-fixes-20260525 PyT image and writes quantized_model_torchao.pth. Older 7.0.0-rc PyT images may fail inside the Sparse4D quantize entrypoint or lack ONNX quantization dependencies; do not remove or skip the advertised quantize action if that occurs. Report the container/image failure and keep the exact checkpoint path visible.

Spec Param / Parent Model Inference

See references/spec_param_inference.md for the model-specific inference mappings from TAO Core sparse4d.config.json (the per-action spec-field to inference-function table) and the parent_model/parent_job_id checkpoint-resolution rules that generated runners apply with SDK helpers before create_job().

Files

SKILL.md and 23 other files (scripts, references) in skills/tao-train-sparse4d of NVIDIA/skills.

SKILL.md
BENCHMARK.md
config/skillspector-baseline.yaml
evals/evals.json
references/local_docker_conversion.md
references/skill_info.yaml
references/spec_param_inference.md
references/spec_template_dataset_convert.yaml
references/spec_template_evaluate.yaml
references/spec_template_export.yaml
references/spec_template_inference.yaml
references/spec_template_quantize.yaml
references/spec_template_train.yaml
schemas/dataset_convert.schema.json
schemas/evaluate.schema.json
schemas/export.schema.json
schemas/inference.schema.json
… and 7 more

Open the folder on GitHubat commit 67a13c0

Compare with similar skills

Tao Train Sparse4d next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Tao Train Sparse4d compared with similar skills
Skill	Stars	Used in	Tokens	Auto-check	Licence	Repo updated
Tao Train Sparse4d this skillNVIDIA/skills	3.5k	—	~3.8k	Automated safety check: Notes	Apache-2.0	yesterday
Segment Anything Model GuideOrchestra-Research/AI-Research-SKILLs	13k	9 repos	~3.3k	Automated safety check: Pass	MIT	3 mo ago
CLIP Image-Text MatchingOrchestra-Research/AI-Research-SKILLs	13k	8 repos	~1.7k	Automated safety check: Pass	MIT	3 mo ago
Yolo Master AgentTencent/YOLO-Master	742	—	~755	Automated safety check: Pass	AGPL-3.0	today
Video Understandjjyaoao/HelloAgents	3.2k	1 repos	~6.2k	Automated safety check: Pass	MIT	2 days ago
Motioneyes Visual Analysisedwardsanchez/MotionEyes	229	—	~2k	Automated safety check: Pass	None	6 mo ago

Similar skills

Segment Anything Model Guide
Orchestra-Research/AI-Research-SKILLs
Guide to using Meta's Segment Anything Model for zero-shot image segmentation with point, box or mask prompts, or automatic mask generation.
13k GitHub starsUsed in 9 repos~3.3k tokens
AI & LLM EngineeringAuto-check passed
CLIP Image-Text Matching
Orchestra-Research/AI-Research-SKILLs
Explains OpenAI's CLIP model for zero-shot image classification, image-text similarity, semantic image search and content moderation, with install steps and code patterns.
13k GitHub starsUsed in 8 repos~1.7k tokens
AI & LLM EngineeringAuto-check passed
Yolo Master Agent
Tencent/YOLO-Master
A skill your agent uses when the user wants to run a YOLO-Master task (train/val/predict/track/export/benchmark) or use the Agent Skill dispatcher.
742 GitHub stars~755 tokensUpdated today
AI & LLM EngineeringAuto-check passed
Video Understand
jjyaoao/HelloAgents
Implement specialized video understanding capabilities using the z-ai-web-dev-sdk.
3.2k GitHub starsUsed in 1 repo~6.2k tokens
AI & LLM EngineeringAuto-check passed
Motioneyes Visual Analysis
edwardsanchez/MotionEyes
Pixel-based motion and UI change analysis from frame sequences or screenshots using computer vision and visual comparison.
229 GitHub stars~2k tokensUpdated 6 mo ago
AI & LLM EngineeringAuto-check passed
LLaVA Vision-Language Model
Orchestra-Research/AI-Research-SKILLs
Guide to LLaVA for image chat, visual question answering and captioning, with model sizes, CLI and Gradio usage and multi-turn conversation code.
13k GitHub starsUsed in 7 repos~2k tokens
AI & LLM EngineeringAuto-check passed

More from NVIDIA/skills

All 380 skills in this repo

Vss Deploy Detection Tracking 2D
NVIDIA/skills
Official
A skill your agent uses when the user wants to deploy, run, debug, tear down, or call the REST API of the RTVI-CV 2D detection / tracking microservice.
3.5k GitHub starsUsed in 1 repo~4.5k tokens
Auto-check passed
HSB IP Def File Tool
NVIDIA/skills
Official
Generates, validates, compares and explains HOLOLINK_def.svh macro files for the HSB IP, using bundled Python scripts and asking before it writes anything.
3.5k GitHub stars~2.9k tokensUpdated yesterday
Auto-check passed
Isaac Sim Mission Control Showcase
NVIDIA/skills
Official
Runs and validates an end-to-end Mission Control demo in a locally installed Isaac Sim, with a Nova Carter robot driven through a Python server.
3.5k GitHub stars~4.8k tokensUpdated yesterday
Auto-check passed
Defect Image Generation with Cosmos AnomalyGen
NVIDIA/skills
Official
Orchestrates defect image generation for PCBA, metal surface and glass inspection with NVIDIA Cosmos AnomalyGen on OSMO, from cold-start Day 0 to real-photo Day 1 labeling.
3.5k GitHub stars~5k tokensUpdated yesterday
Auto-check: notes
Physical AI Video Augmentation on OSMO
NVIDIA/skills
Official
Orchestrates video data augmentation and auto-labeling workflows on OSMO, from flow selection and preflight checks to submission, monitoring and output download.
3.5k GitHub stars~4.7k tokensUpdated yesterday
Auto-check: notes
TAO Detection KPI Analysis
NVIDIA/skills
Official
Runs NVIDIA TAO Data Services KPI analysis on object detection results, comparing predictions to ground truth and writing per-class precision, recall and AP to a CSV.
3.5k GitHub stars~2.7k tokensUpdated yesterday
Auto-check: notes

Questions about Tao Train Sparse4d

What does Tao Train Sparse4d do?

Sparse4D for multi-camera temporal 3D object detection and tracking. Tao Train Sparse4d is an agent skill from NVIDIA/skills, published by the product's own GitHub organization. Sparse4D for multi-camera temporal 3D object detection and tracking.

When should I use Tao Train Sparse4d?

Tao Train Sparse4d fits situations like: running inference for a TAO Sparse4D model; phrases include train Sparse4D; multi-camera 3D detection; temporal 3D tracker.

How do I install Tao Train Sparse4d in Claude Code?

Run `npx skills add NVIDIA/skills --skill tao-train-sparse4d -a claude-code`. Or copy the skill folder (skills/tao-train-sparse4d in NVIDIA/skills) into .claude/skills/tao-train-sparse4d in your project. Claude Code loads it when a task matches its description.

How do I install Tao Train Sparse4d in Codex?

Run `npx skills add NVIDIA/skills --skill tao-train-sparse4d -a codex`. Or copy the skill folder (skills/tao-train-sparse4d in NVIDIA/skills) into .agents/skills/tao-train-sparse4d in your project. Codex loads it when a task matches its description.

Can I use Tao Train Sparse4d in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add NVIDIA/skills --skill tao-train-sparse4d -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/tao-train-sparse4d, .gemini/skills/tao-train-sparse4d, .github/skills/tao-train-sparse4d and .opencode/skills/tao-train-sparse4d in your project.

What does Tao Train Sparse4d need to run?

SKILL.md names no scripts, command-line tools or credentials: Tao Train Sparse4d is instructions for the agent only. Our summary lists: Python 3; Docker. Its frontmatter pre-approves these tools: Read, Bash. Compatibility (from SKILL.md): Requires docker + nvidia-container-toolkit..

Does Tao Train Sparse4d access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Tao Train Sparse4d safe to install?

Our automated static check of SKILL.md found notes only (pre-approves every shell command (allowed-tools: bash)), nothing it rates as a warning. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.

What licence does Tao Train Sparse4d use?

Tao Train Sparse4d is published under the Apache-2.0 licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Tao Train Sparse4d use?

About 3.8k tokens (SKILL.md is roughly 15k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 13k tokens, read only when the agent opens those files.

What are the alternatives to Tao Train Sparse4d?

Skills that share tags, products or a category with Tao Train Sparse4d: Segment Anything Model Guide (Orchestra-Research/AI-Research-SKILLs, 13k stars), CLIP Image-Text Matching (Orchestra-Research/AI-Research-SKILLs, 13k stars), Yolo Master Agent (Tencent/YOLO-Master, 742 stars) and Video Understand (jjyaoao/HelloAgents, 3.2k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Tao Train Sparse4d?

NVIDIA (a GitHub organization, an official publisher) maintains it in NVIDIA/skills, which has 3,539 GitHub stars. The repository holds 380 skills in this directory. The repository was last updated on October 7, 2026.

Source: NVIDIA/skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.

Install Tao Train Sparse4d

Facts

At a glance

What it does

When your agent uses it

Example prompts

Requirements

What it can do on your machine

Tool permissions

Runs code

Network

Credentials

Compatibility

Context cost

Safety

SKILL.md

Sparse4D

Dataclass Schemas

Train Action Policy

Training Requirements

Per-Action Dataset Requirements

Typical Spec Overrides

Eval Dataset

Important Parameters

Multi-GPU / Multi-Node

Hardware

Error Patterns

Spec Param / Parent Model Inference

Files

Compare with similar skills

Similar skills

More from NVIDIA/skills

Related

Questions about Tao Train Sparse4d