Vss Deploy Detection Tracking 3D

Run Stages

Follow these stages for deployment work:

1. Resolve RTCV3D_APP to services/rtvi/rt-cv-3d/rt-cv-mv3dt.
2. Identify the input mode: file for local MP4s or stream for RTSP.
3. If the user asked for the sample dataset or 4-cam example dataset, load references/sample-dataset.md first. Resolve/download app-data, set MODELS_DIR, VIDEO_DIR=<APP_DATA_DIR>/videos/warehouse-4cams-20mx20m-synthetic, CALIBRATION_JSON, BEV_DATASET_PATH, NUM_CAMS=4, and INPUT_MODE=file, then continue with camera validation and the normal display/save decision before setting OSD, SAVE_VIDEO, or BEV_SAVE_VIDEO.
4. Validate or obtain calibration.json. If missing, hand off to vss-generate-video-calibration by name and do not duplicate the AMC workflow inline. Explicitly include the AMC platform preflight failure path: stop and request existing/generated calibration artifacts or a supported x86_64 dGPU/NVENC calibration host. After AMC completes, fetch the AMC MV3DT export ZIP, export calibration.json, validate JSON by filtering sensors where type == "camera" and requiring at least two non-empty safe unique camera IDs, then stage BEV assets before continuing. For saved output or BEV viewing, resolve BEV_DATASET_PATH to a directory containing both map.png and transforms.yml before launch.
5. Set required values in docker/.env: MODELS_DIR, NUM_CAMS, INPUT_MODE, VIDEO_DIR for file input, and optional image/GPU values. For supplied MP4 paths, point VIDEO_DIR at the matching source directory or at a generated symlink directory with one <sensor_id>.mp4 per camera.
6. Initialize broker mode before config generation or staging. For bundled mode, run the bundled resource preflight in references/deploy-rtvi-cv-3d-stack.md now so selected MQTT_PORT, KAFKA_PORT, and KAFKA_BOOTSTRAP are already in docker/.env. For external mode, validate broker endpoints and required topics before launch.
7. Generate generated/camInfo/ and generated/pub_sub_info_config.yml from calibration.json with the standalone scripts/generate-configs.sh, using the selected MQTT endpoint; do not mount warehouse MV3DT calibration directories.
8. Run the concrete display probe from references/configure-cameras.md before staging configs; it must test the current DISPLAY and discovered X socket candidates such as :0/:1, export a working DISPLAY when found, and print RTCV3D_DISPLAY_AVAILABLE. Then choose visualization:
   • If a working display is detected and the user did not ask to save, stage with OSD=1 SAVE_VIDEO=0, set BEV_SAVE_VIDEO=0 BEV_SOURCE=fused, and use live fused BEV visualization when BEV assets are present.
   • If no display is detected, use saved output as the default fallback: set SAVE_VIDEO=1 and save fused BEV after BEV_DATASET_PATH resolves with both required files.
   • If the user asked to save output, set SAVE_VIDEO=1 even when a display exists and also save fused BEV by default after BEV_DATASET_PATH resolves with both required files.
   • If the user asked for both live view and saved output, use OSD=1 SAVE_VIDEO=1 and start saved fused BEV in parallel.
9. Stage DeepStream configs with scripts/stage-configs.sh, then assert generated/configs/ds-main-config-mv3dt.txt contains a Kafka msg-broker-conn-str matching the selected KAFKA_BOOTSTRAP and RAW_TOPIC. For INPUT_MODE=file, also assert the staged config disables live latency dropping: [source-list] low-latency-mode=0, [source-attr-all] drop-on-latency=0, and [source-attr-all] latency=100000.
10. Preflight output/tooling and model-cache writeability. Cold TensorRT engine builds can take 5-10 minutes and must be able to persist engines under the mounted model directories. State this warning and the scoped ACL remediation pattern in the user-visible status/report. If a model cache is not writable by the container UID/GID, stop and request approval before applying an ACL to only the affected model directory, for example sudo setfacl -m u:<uid>:rwx -m d:u:<uid>:rwx <model-dir>; do not use broad chmod 777 or broad recursive chown.
11. For every INPUT_MODE=file run, start the selected brokers and bev-fusion, wait for broker/topic-init/BEV Fusion readiness, then capture Kafka baselines before starting perception. Do this even when saved output or BEV visualization is not requested.
12. If saved BEV is selected/defaulted, or if file input needs any live/saved BEV visualization, use the two-phase launch in references/deploy-rtvi-cv-3d-stack.md: after support readiness and file baselines, start the BEV visualizer/recorder in the same long-lived shell/session that will start perception, wait for EOS, verify outputs, and finalize BEV. Wait for its Kafka consumer group assignment, verify the recorder PID is still alive, then start perception with --no-deps. Saved output uses BEV_SAVE_VIDEO=1 BEV_SOURCE=fused by default; display-only output uses BEV_SAVE_VIDEO=0 BEV_SOURCE=fused so the BEV window is live. For stream mode with no BEV prestart requirement, full-stack Compose launch is acceptable: bundled uses COMPOSE_PROFILES=mosquitto,kafka docker compose up -d; external uses docker compose up -d. Never use full-stack docker compose up -d for file input.
13. For RTSP input, after stream-mode compose is running, register the provided streams with the direct REST registration block in references/configure-cameras.md, which waits on /api/v1/ready for ds-ready=YES using JSON parsing. Use explicit <sensor_id>=<rtsp_url> pairs when provided; otherwise map bare URLs to calibration sensor ids only when the counts and ordering are clear. Preserve the final mapping so it can also be used for the static RTSP source-list fallback if dynamic REST add does not produce active sources.
14. For RTSP, verify REST readiness, exact stream registration, non-zero FPS, mdx-raw/mdx-bev offset growth, and requested visualization artifacts. If registration succeeds but active sources/FPS/Kafka remain zero, restage the same RTSP mapping as a static [source-list], restart only perception as appropriate for stream mode, and rerun the same verification. For file input, do not require ds-ready: YES; treat vss-rtvi-cv-mv3dt Exited (0) with App run successful as EOS success, then require mdx-raw and mdx-bev offsets to be greater than pre-run baselines.
15. For completed file-input runs, after outputs are verified, stop only the remaining standalone support services unless the user asked to keep them running for inspection or reuse. Preserve generated configs, calibration, videos, and outputs.

Success Criteria

• generated/camInfo/ contains one .yml per filtered camera sensor and generated/configs/ exists.
• Runtime images are reported from docker compose config --images; the skill does not infer image tags from its own version.
• docker compose uses services/rtvi/rt-cv-3d/rt-cv-mv3dt/docker/compose.yml.
• vss-rtvi-cv-bev-fusion becomes healthy.
• For RTSP: REST /api/v1/ready reports ds-ready=YES, registered stream count equals NUM_CAMS, registered IDs exactly match generated camInfo IDs with no duplicates/extras, every expected source has recent non-zero FPS, and both mdx-raw and mdx-bev offsets grow while streams are active.
• For file input: vss-rtvi-cv-mv3dt may end as Exited (0) after EOS and is successful only when logs include App run successful and both mdx-raw and mdx-bev offsets exceed pre-run baselines.
• If live OSD was selected, display access was checked before staging with OSD=1 without broad xhost +, and display-mode visualization includes both the DeepStream camera-grid OSD window and the separate live fused BEV window when BEV assets are present.
• If saved output was selected/defaulted, report current-run video-output/grid-view.mkv and saved BEV artifact paths with non-empty size, run-start timestamp checks, ffprobe success, and current BEV log evidence including Video saved with positive frame count. If BEV was skipped because assets were missing, report that explicitly.
• If live BEV visualization was selected, report the tracked visualizer PID/log, Kafka consumer group assignment evidence, and for finite file input either that the user closed the BEV window with q or that the tracked current-run PID was safely stopped after EOS.

Related Skills

• vss-generate-video-calibration owns AMC deployment and calibration from local MP4s or RTSP streams.
• vss-manage-video-io-storage is used only to bring up or verify VIOS when RTSP calibration needs VIOS and it is not already deployed.
• vss-build-vision-ai owns full warehouse blueprint deployments, including explicit warehouse MV3DT requests.