Deepstream Run Mv3dt

Skill: Run DeepStream MV3DT

When to Use This Skill

Activate this skill when the user wants to set up, run, verify, or debug the DeepStream Multi-View 3D Tracking reference app. Typical prompts:

• "set up MV3DT DeepStream"
• "run the 4-camera MV3DT sample"
• "run the 12-camera MV3DT sample"
• "run MV3DT on my synchronized MP4s"
• "I have videos but no calibration; calibrate and run MV3DT"
• "show the BEV visualizer or Kafka metadata"
• "stop MV3DT" / "clean up MV3DT" / "tear down MV3DT"

Do not use this skill for single-view 3D tracking, generic DeepStream app development, or live-stream onboarding unless the user explicitly maps that work to this repo's MV3DT pipeline.

Examples

• "Deploy the DeepStream MV3DT 4-camera sample and show the OSD and BEV windows."
• "Run the MV3DT 12-camera sample headlessly with RTDETR and save videos."
• "Run MV3DT on synchronized MP4s under /data/mv3dt-demo using PeopleNetTransformer."
• "My custom MV3DT videos do not have calibration; use AutoMagicCalib, then run MV3DT."

Overview

Operate the Multi-View 3D Tracking reference app in DeepStream/src/apps/reference_apps/deepstream-tracker-3d-multi-view using the DeepStream Container path. The skill supports setup, shipped sample runs, custom synchronized MP4 datasets, calibration handoff to AutoMagicCalib, display/headless execution, OSD/BEV outputs, and Kafka metadata inspection.

Prerequisites

• MV3DT reference app directory on disk under DeepStream/src/apps/reference_apps/deepstream-tracker-3d-multi-view; if it is absent, ask before cloning the public DeepStream repo
• Docker with NVIDIA GPU support
• DeepStream container image access
• MV3DT sample datasets, models, custom parser libraries, Kafka, Mosquitto, and mv3dt_venv prepared by the repo setup script
• Working X11/VNC display for live OSD/BEV windows, or the saved-output headless path for tiled DeepStream MP4 plus Kafka-derived BEV MP4

Instructions

Step 0: Resolve MV3DT App Checkout

The skill can be installed outside the DeepStream repo. Resolve REPO_ROOT to the MV3DT app directory, not necessarily the Git top-level directory.

MV3DT_APP_SUBDIR="src/apps/reference_apps/deepstream-tracker-3d-multi-view"

is_mv3dt_app_dir() {
  test -f "$1/README.md" || return 1
  test -d "$1/config_templates" || return 1
  test -d "$1/scripts" || return 1
  grep -q "Multi-View 3D Tracking" "$1/README.md"
}

GIT_TOP="$(git rev-parse --show-toplevel 2>/dev/null || true)"
CANDIDATES=()
if [ -n "${MV3DT_REPO_ROOT:-}" ]; then CANDIDATES+=("${MV3DT_REPO_ROOT}"); fi
if [ -n "${DEEPSTREAM_REPO_ROOT:-}" ]; then CANDIDATES+=("${DEEPSTREAM_REPO_ROOT}/${MV3DT_APP_SUBDIR}"); fi
CANDIDATES+=("${PWD}")
if [ -n "${GIT_TOP}" ]; then
  CANDIDATES+=("${GIT_TOP}" "${GIT_TOP}/${MV3DT_APP_SUBDIR}")
fi
CANDIDATES+=("${HOME}/DeepStream/${MV3DT_APP_SUBDIR}" "${HOME}/deepstream/${MV3DT_APP_SUBDIR}")

REPO_ROOT=""
for candidate in "${CANDIDATES[@]}"; do
  if [ -n "$candidate" ] && is_mv3dt_app_dir "$candidate"; then
    REPO_ROOT="$(cd "$candidate" && pwd)"
    break
  fi
done

if [ -z "${REPO_ROOT}" ]; then
  cat <<'EOF'
ERROR: MV3DT reference app directory was not found.
Set MV3DT_REPO_ROOT to an existing deepstream-tracker-3d-multi-view app directory, or ask the user to approve cloning the public DeepStream repo and then run:

  DEEPSTREAM_REPO_ROOT="${DEEPSTREAM_REPO_ROOT:-$HOME/DeepStream}"
  git clone https://github.com/NVIDIA/DeepStream.git "$DEEPSTREAM_REPO_ROOT"
  export MV3DT_REPO_ROOT="$DEEPSTREAM_REPO_ROOT/src/apps/reference_apps/deepstream-tracker-3d-multi-view"

Do not clone silently.
EOF
  exit 1
fi

cd "${REPO_ROOT}"
export REPO_ROOT MV3DT_REPO_ROOT="${REPO_ROOT}"

If the app directory cannot be resolved, ask the user for an existing checkout path or for approval to clone https://github.com/NVIDIA/DeepStream. Do not clone silently.

Step 1: Select The Primary Workflow

Load exactly one primary reference for the user's current request:

If setup, datasets, models, Kafka, Mosquitto, Docker GPU runtime, or the Python venv are missing, load references/setup.md before continuing to the user's original workflow.

When sample-run.md or custom-dataset.md needs to regenerate DeepStream configs, load references/generate-configs.md as the canonical shared config-generation reference rather than duplicating the shell logic.

Step 2: Follow The Run Stages

For every run, use this stage order:

Step 3: Apply Defaults Explicitly

• Runtime path: DeepStream Container.
• Display path: if a working X11/VNC display is available, use the repo quick-start path with OSD and BEV windows. The OSD window does not save MP4 by default; when the user explicitly asks to save output in display mode, regenerate configs with both --enable-osd and --enable-file-output plus --enable-msg-broker.
• Headless path: if no working display is available, use saved outputs by default. Generate configs with --enable-file-output and --enable-msg-broker; start offline BEV capture before DeepStream so the run produces both the tiled DeepStream MP4 and the Kafka-derived BEV MP4.
• Sample dataset: support both shipped 4-camera and 12-camera datasets. If the user does not specify a sample, run the 4-camera sample first, then mention in the final report that the 12-camera sample is also available and can be run next.
• Detector: PeopleNetTransformer unless the user asks for RTDETR or PeopleNet2.6.3; carry the selected DETECTOR_MODEL through config generation and AMC camInfo modelInfo normalization. For custom calibration handoff, ask the user to choose the AutoMagicCalib detector instead of silently defaulting.
• Custom data source: synchronized MP4 files. Live-stream handling is outside this first-release skill.
• Calibration handoff: use standalone AutoMagicCalib skills instead of duplicating their setup or API workflow.