Vss Deploy Detection Tracking 3D

Purpose

Deploy and operate the RTVI-CV-3D microservice as MV3DT (MODE=mv3dt) — per-camera DeepStream perception plus BEV Fusion over multiple calibrated cameras — on the bundled sample dataset, custom videos, or live RTSP, without the full warehouse agent / LLM / VLM stack.

Instructions

Work top-to-bottom: answer the routing questions (Q0–Q3) under Routing, then follow the reference for the chosen path. Detailed step-by-step procedures live in references/ (deploy, calibration chain, camera configuration, verification, teardown, troubleshooting).

Examples

• Enable multi-camera tracking on the sample dataset.
• Deploy RTVI-CV-3D on my videos here: <path/to/videos>.
• Run MV3DT on RTSP streams after calibration.

VSS Deploy Detection & Tracking — 3D (RTVI-CV-3D / MV3DT)

Bring up the RTVI-CV-3D microservice as the MV3DT stack (MODE=mv3dt) from the warehouse blueprint: per-camera DeepStream perception (vss-rtvi-cv-mv3dt) + BEV Fusion (vss-rtvi-cv-bev-fusion) + mosquitto MQTT bus + broker + VST sensor stack — without the agent / LLM / VLM stack that comes with the full warehouse blueprint.

The actual compose machinery lives in deploy/docker/industry-profiles/warehouse-operations/warehouse-mv3dt-app/. This skill drives the env overrides, calibration chain, and verification.

Routing

Ask the user at most four questions, then dispatch.

Q0 — Profile size (overlays or not)

Default to extended unless the user explicitly asks for minimal. Extended deploys ELK + vss-video-analytics-api-mv3dt + vss-kibana-init-mv3dt + vss-import-calibration-output-mv3dt on top of MV3DT core — these are what the VST video wall needs to render bounding-box overlays. Without them, the video wall works but shows raw streams without overlays.

Note on selective ELK: there's no "minimal + ELK only" middle path in the current compose. Every ${MINIMAL_PROFILE:+_extended}-gated service comes up together (ES, Logstash, Kibana, video-analytics-api, kibana-init, import-calibration). bash's :+ parameter expansion produces the _extended suffix when MINIMAL_PROFILE is set; extended switches the gating string back to plain bp_wh_kafka_mv3dt which the active compose profile already matches. Either you accept the full extended bundle or you stay minimal.

Q1 — Data source

Ask this unless the source is explicit in the user's first message. A bare request like "deploy rtvi-cv-3d" routes to this MV3DT skill (MODE=mv3dt), but does not imply sample.

• sample — the bundled 4-camera synthetic dataset (warehouse-4cams-20mx20m-synthetic). Calibration ships in-tree; no AMC run needed.
• videos — the user has local video files (any *.mp4 named after their cameras). Standalone AMC (auto_calib profile) will run if calibration is missing.
• rtsp — the user has live RTSP URLs. Calibration via VIOS-driven AMC; final deploy also needs a Sensor Info File (camera_info.json) with those RTSP URLs.

Q2 — Calibration coverage (skip for sample)

For videos and rtsp, check whether calibration is already on disk at the mount path the perception container expects:

DATASET="${SAMPLE_VIDEO_DATASET:?}"          # the user's dataset slug; see Q3
CAL_DIR="${VSS_APPS_DIR}/industry-profiles/warehouse-operations/warehouse-mv3dt-app/calibration/sample-data/${DATASET}"

# Look for ANY of: calibration.json, plus camInfo/*.yml or *.yaml with either
# 'cam_*' or 'Camera*' naming (the shipped sample uses Camera*.yml, AMC may
# produce cam_*.yaml — broaden accordingly)
test -f "${CAL_DIR}/calibration.json" \
  && ls "${CAL_DIR}/camInfo/"*.{yml,yaml} 2>/dev/null

If the user supplied a calibration path themselves, validate that path instead — don't recompute. See configure-cameras.md for camera-name normalization and authoritative camera-count discovery (parses calibration.json).

Q3 — Detector + dataset slug (only when Q2 triggers AMC)

• resnet (default, fast) or transformer (slower, better under occlusion) — passed to the AMC /v1/calibrate/<id> API at Step B (see vss-generate-video-calibration/SKILL.md:48-62).
• A short kebab-case dataset slug used as SAMPLE_VIDEO_DATASET (e.g. customer-aisle-4cams). This drives the calibration mount path and gets persisted in .env.

Routing table

Every path converges on references/verify-and-view.md once up -d completes. references/troubleshooting.md and references/teardown.md are linked but off the happy path.

Disambiguation rule. In this skill, "RTVI-CV-3D" means the MV3DT microservice deployment and uses MODE=mv3dt. Route to ../vss-deploy-profile/references/warehouse.md only when the user asks for the full warehouse blueprint, Sparse4D, MODE=3d, or warehouse-3d-app. This skill is for MV3DT only without the agent stack / LLM / VLM.

Prerequisites

1. Repo path

Locate video-search-and-summarization/ on disk. All compose commands run from <repo>/deploy/docker/. If unknown, ask the user.

2. NGC CLI + key

$NGC_CLI_API_KEY must be set and must have access to nvidia/vss-core/* images. See vss-deploy-profile/references/ngc.md for setup if missing.

If the user previously ran ngc config set but $NGC_CLI_API_KEY isn't exported in this shell, the key is already on disk:

NGC_CLI_API_KEY=$(awk -F'= ' '/^apikey/{print $2}' ~/.ngc/config 2>/dev/null)
test -n "${NGC_CLI_API_KEY}" && echo "key sourced from ~/.ngc/config"

Make sure the key value also lands in industry-profiles/warehouse-operations/.env:164 (NGC_CLI_API_KEY=...) — compose only reads it from there at up time, not from your shell env.

3. HARDWARE_PROFILE slug

The public MV3DT supported stream counts are listed in the Warehouse Quickstart Guide under "MV3DT Vision AI Profile Supported Deployment Options." Use the matching HARDWARE_PROFILE slug below.

Pick from nvidia-smi --query-gpu=name --format=csv,noheader:

If the user's GPU is not listed here, check industry-profiles/warehouse-operations/.env for available HARDWARE_PROFILE values, then confirm the matching profile exists in blueprint-configurator/blueprint_config.yml before using it. Do not infer a stream count from the slug alone.

The per-GPU MV3DT cap is enforced at deploy time. vss-configurator-mv3dt computes final_stream_count = min(NUM_STREAMS, max_streams_supported) and applies a keep_count file-management op against ${VSS_DATA_DIR}/videos/${SAMPLE_VIDEO_DATASET}/ so only final_stream_count .mp4 files remain (sorted lexicographically, last N kept). If your GPU's MV3DT supported stream count (above table) is below your camera count, perception / mdx-raw / mdx-bev run with the supported stream count. Either pick a GPU with a higher supported stream count or surface the cap explicitly to the user so they're aware which streams will be processed.