TAO Image Embeddings

What it does

When your agent uses it

• Embedding a folder of images before nearest-neighbor mining
• Generating SigLIP or CLIP embeddings for a dataset listed in a parquet file
• Checking that a model and model path pair match before launching a run
• Producing embeddings from a TAO checkpoint

Example prompts

• “Embed the images listed in data/images.parquet with SigLIP and write data/embeddings.parquet.”
• “Generate CLIP embeddings for my training set and keep the label column.”
• “The GPU ran out of memory on the embedding run, so lower the batch size and retry.”
• “Check my image_embeddings.yaml for a model and model_path mismatch.”

Requirements

• Docker with the NVIDIA container toolkit
• One or more CUDA GPUs
• The TAO data-services container pinned in versions.yaml
• Compatibility (from SKILL.md): Requires docker, nvidia-container-toolkit, one or more CUDA GPUs, and the TAO data-services container pinned in versions.yaml.
• Pre-approved tools (allowed-tools): Read, Bash

Workflow steps

4 steps, taken from the first numbered list in SKILL.md.

1. 1Verify Docker and GPU access
2. 2Resolve and pull the data-services image if needed
3. 3Validate the spec
4. 4Confirm RUN_ROOT contains the spec, the input parquet, the image files its filepath column points at, and the output directory. Mount…

What it can do on your machine

Read from SKILL.md and the folder at commit dfdd080. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves these tools, so the agent can use them without asking each time:

  • Read
  • Bash

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Ships 1 file in scripts/ (Python), which the agent can run.

  Shell commands in SKILL.md call:

  • docker
  • python3

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md. Its commands use docker, which can reach the network depending on how they are called.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

• Compatibility

  Requires docker, nvidia-container-toolkit, one or more CUDA GPUs, and the TAO data-services container pinned in versions.yaml.

  From compatibility in the SKILL.md frontmatter.

Context cost

TAO Image Embeddings loads about 2k tokens when it runs, and up to ~2.1k if it reads all its reference files. Until then it costs about 90 tokens; SKILL.md has 742 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~90

When it runs · the whole SKILL.md, loaded when a task matches

~2k

With references · SKILL.md plus every file in references/, read only if the agent opens them

~2.1k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

• NotePre-approves every shell command (allowed-tools: Bash)SKILL.md

  allowed-tools: Read, Bash

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from NVIDIA/skills at commit dfdd080, republished under its Apache-2.0 licence (© NVIDIA). 742 words, ~1,972 tokens.

name

tao-generate-image-embeddings

description

Run TAO Data Services image embedding to turn a parquet of image filepaths into an embedding parquet using CLIP, SigLIP, or a TAO checkpoint. Use when a workflow needs embeddings before nearest-neighbor or unique-neighbor mining, or when the user asks to "embed images", "compute image embeddings", or "generate SigLIP embeddings".

allowed-tools

Read, Bash

compatibility

Requires docker, nvidia-container-toolkit, one or more CUDA GPUs, and the TAO data-services container pinned in versions.yaml.

license

Apache-2.0

metadata.author

NVIDIA Corporation

metadata.version

0.1.0

tags

tao, data, embeddings, mining, siglip, clip

embedding image_embeddings -e /absolute/path/to/image_embeddings.yaml

OUT_DIR=/absolute/path/for/this/run              # where output_parquet is written
SPEC="$OUT_DIR/image_embeddings.yaml"            # spec lives beside its output
RUN_ROOT=/absolute/path/that/contains/parquets/images/and/results
GPU_COUNT=1

python3 skills/data/tao-generate-image-embeddings/scripts/verify_image_embeddings_spec.py \
  --spec "$SPEC"

DS_IMAGE=nvcr.io/nvidia/tao/tao-toolkit:7.2.0-data-services  # versions-key: images.tao_toolkit.data_services

docker run --rm --gpus "$GPU_COUNT" --shm-size=8g --network=host \
  -v "$RUN_ROOT:$RUN_ROOT" \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  -w "$RUN_ROOT" \
  "$DS_IMAGE" \
  embedding image_embeddings -e "$SPEC"

docker run --rm --gpus "$GPU_COUNT" --shm-size=8g --network=host \
  -v "$RUN_ROOT:$RUN_ROOT" -w "$RUN_ROOT" "$DS_IMAGE" \
  embedding image_embeddings -e "$SPEC" \
  input_parquet=/abs/path/other_input.parquet \
  output_parquet=/abs/path/other_output.parquet

cp skills/data/tao-generate-image-embeddings/assets/default_image_embeddings.yaml "$SPEC"

python3 skills/data/tao-generate-image-embeddings/scripts/verify_image_embeddings_spec.py --spec "$SPEC"

input_parquet: /absolute/path/filepaths.parquet
output_parquet: /absolute/path/results/embeddings.parquet
model: SigLIP
model_path: google/siglip-base-patch16-224
model_config_path: ""          # required only when model_path is a TAO .pth/.ckpt
batch_size: 64

docker info > /dev/null
nvidia-smi -L

DS_IMAGE=nvcr.io/nvidia/tao/tao-toolkit:7.2.0-data-services  # versions-key: images.tao_toolkit.data_services
docker image inspect "$DS_IMAGE" > /dev/null || docker pull "$DS_IMAGE"

python3 skills/data/tao-generate-image-embeddings/scripts/verify_image_embeddings_spec.py \
  --spec "$SPEC"

© NVIDIA, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 8 other files (scripts, references, assets) in skills/tao-generate-image-embeddings of NVIDIA/skills.

• SKILL.md
• BENCHMARK.md
• assets/default_image_embeddings.yaml
• config/skillspector-baseline.yaml
• evals/evals.json
• references/skill_info.yaml
• scripts/verify_image_embeddings_spec.py
• skill-card.md
• skill.oms.sig

Open the folder on GitHubat commit dfdd080