Vss Generate Video Report

Report

Generate a video analysis report by routing to one of two backends — never via POST /generate on the VSS agent.

If the request is ambiguous (e.g. "report on <sensor>" with no time range and no incident wording), default to Mode A. Ask only if the user mentions both a sensor and a time range. See Examples below for the request phrasings that route to each mode.

Instructions

1. Pick the mode — Mode A for a single recorded clip/sensor video, Mode B when the request names a time range or incidents/alerts (match against Examples).
2. Verify the deployment profile for that mode under Deployment prerequisite; hand off to /vss-deploy-profile if its probe fails.
3. Run that mode's numbered steps — Mode A or Mode B below.
4. Rewrite every user-facing clip URL with the $VSS_PUBLIC_HOST:$VSS_PUBLIC_PORT one-liner (Browser-playable clip URL) before embedding it in the report.
5. Return the rendered report markdown to the user.

Output contract for evaluators:

• Mode A top title MUST be exactly # Video Analysis Report.
• Mode B top title MUST be exactly # Incident Range Report (never # Incident Report or sensor-named variants).
• Mode B MUST include ## Basic Information with the exact required rows from the template (Report Identifier, Range, Scope, Total Incidents, Confirmed / Rejected / Unverified).

Examples

• "Generate a report for this video" / "report on <sensor-id>" → Mode A
• "Analyze warehouse_01.mp4" / "create an analysis report on the uploaded video" → Mode A
• "Report on incidents from 12:31Z to 12:32Z" → Mode B
• "Report on alerts today" / "what incidents happened on <sensor> last hour" → Mode B
• "Summarize alerts on <sensor> between <t1> and <t2>" → Mode B

Negative Triggers

Do not use this skill when the request is one of the following:

• Ad-hoc visual Q&A on a clip that do not ask explicitly for a report ("what color is the truck?", "what happens at 00:12?") → use /vss-ask-video.
• Archive/semantic similarity retrieval ("find forklifts", "search all videos for tailgating") → use /vss-search-archive.
• Read-only incident/metrics lookup without report rendering needs → use /vss-query-analytics.
• Deploy/teardown/profile changes ("deploy alerts", "switch profile", "bring up base") → use /vss-deploy-profile.
• Real-time alert/rule management requests → use /vss-manage-alerts.

Never route reports through VSS-agent POST /generate.

Deployment prerequisite

Mode A needs the VSS base profile (VST + VLM NIM). Mode B needs the VSS alerts profile (VA-MCP + Elasticsearch).

Probe:

# Mode A — VST + VLM reachability
curl -sf --max-time 5 "http://${HOST_IP}:30888/vst/api/v1/sensor/version" >/dev/null

# Mode B — VA-MCP
curl -sf --max-time 5 "http://${HOST_IP}:9901/" >/dev/null

If the probe fails, hand off to /vss-deploy-profile with -p base (Mode A) or -p alerts (Mode B). Always confirm the deploy with the user first.

Clip URLs: VLM input vs browser report link

VST returns clip URLs using the agent-internal ${HOST_IP}:30888 host:port. Keep that original URL as VIDEO_URL for local / in-cluster VLM frame pulls. Do not rewrite the VLM input URL just to make it browser-playable.

Only create BROWSER_CLIP_URL for URLs shown in the rendered report. The deploy layer exports the browser-facing host:port as $VSS_PUBLIC_HOST / $VSS_PUBLIC_PORT (and scheme as $VSS_PUBLIC_HTTP_PROTOCOL) in every profile .env — Brev or bare-metal — so the report-link rewrite is:

: "${VSS_PUBLIC_HOST:?Set VSS_PUBLIC_HOST before rewriting clip URLs}"
: "${VSS_PUBLIC_PORT:?Set VSS_PUBLIC_PORT before rewriting clip URLs}"
VSS_PUBLIC_HTTP_PROTOCOL="${VSS_PUBLIC_HTTP_PROTOCOL:-http}"
BROWSER_CLIP_URL=$(echo "$RAW_URL" | sed -E "s|^https?://[^/]+|${VSS_PUBLIC_HTTP_PROTOCOL}://${VSS_PUBLIC_HOST}:${VSS_PUBLIC_PORT}|")

If either required public host value is missing, omit the report-facing clip link and call out that a browser-playable URL could not be produced; do not block the local VLM analysis path. Apply the rewrite to every clip URL surfaced in the rendered report (Mode A Step 4 Clip URL row; Mode B per-incident clip sub-bullet). Leave the VLM video_url content block in Mode A Step 3 on the original internal URL when the VLM is local / in-cluster.

Mode A — Report on a recorded video clip

If the VSS lvs profile is deployed — curl -sf --max-time 5 "http://${HOST_IP}:38111/v1/ready" returns HTTP 200 — run /vss-summarize-video to produce the summary, then paste its output into the report template in Step 4 and skip Steps 1–3 (the VLM-direct path). Run Steps 1–3 only when /v1/ready is non-200.

Step 1 — Resolve the clip URL

Hand off to /vss-manage-video-io-storage to:

1. List sensors and confirm the named <sensor-id> exists (upload first if not).

2. Fetch /storage/<streamId>/timelines for the recorded range when the user did not supply startTime / endTime.

3. Request a clip URL:

   curl -s "http://${HOST_IP}:30888/vst/api/v1/storage/file/<streamId>/url?startTime=<startTime>&endTime=<endTime>&container=mp4&disableAudio=true" | jq -r .videoUrl

   That gives a direct mp4 URL that the local / in-cluster VLM can pull frames from. Bind it to VIDEO_URL (used by the VLM in Step 3) and set RAW_URL="$VIDEO_URL" before applying the report-link rewrite to produce BROWSER_CLIP_URL for Step 4 — the user's browser cannot reach $VIDEO_URL directly. Mode A requires the selected VLM endpoint to be able to fetch VIDEO_URL. Local NIM/RT-VLM deployments normally can; remote endpoints generally cannot fetch localhost, private HOST_IP, or VST-internal URLs. If the live VLM_ENDPOINT is remote, surface that reachability requirement instead of making a chat request that will fail after /v1/models succeeds.

Step 2 — Resolve VLM endpoint and model

The deploy may serve the VLM through either of two stacks. Both expose an OpenAI-compatible chat/completions API — pick whichever is live:

Read the live values off the running agent container — do not guess:

docker exec vss-agent sh -lc '
for k in HOST_IP VLM_MODE VLM_MODEL_TYPE VLM_BASE_URL VLM_NAME RTVI_VLM_BASE_URL RTVI_VLM_MODEL_TO_USE; do
  v="$(printenv "$k")"
  [ -n "$v" ] && printf "%s=%s\n" "$k" "$v"
done
'

Do not require RTVI_VLM_ENDPOINT from vss-agent env; several profiles do not inject it.

Selection rule:

if [ "${VLM_MODEL_TYPE:-}" = "rtvi" ]; then
  VLM_BACKEND="rtvlm"
  VLM_ENDPOINT="${RTVI_VLM_BASE_URL:+${RTVI_VLM_BASE_URL%/}/v1}"
  [ -z "${VLM_ENDPOINT}" ] && VLM_ENDPOINT="http://${HOST_IP}:8018/v1"   # alerts default
  VLM_MODEL="${RTVI_VLM_MODEL_TO_USE}"
elif [ -n "${VLM_BASE_URL}" ] && [ "${VLM_MODE}" != "none" ]; then
  VLM_BACKEND="nim_cosmos"
  VLM_ENDPOINT="${VLM_BASE_URL%/}/v1"
  VLM_MODEL="${VLM_NAME}"
else
  VLM_BACKEND="rtvlm"
  VLM_ENDPOINT="${RTVI_VLM_BASE_URL:+${RTVI_VLM_BASE_URL%/}/v1}"
  [ -z "${VLM_ENDPOINT}" ] && VLM_ENDPOINT="http://${HOST_IP}:30082/v1"  # base default
  VLM_MODEL="${RTVI_VLM_MODEL_TO_USE}"
fi

Probe /v1/models before sending a chat request to confirm the chosen endpoint is alive and the model is loaded:

curl -sf --max-time 5 "${VLM_ENDPOINT}/models" | jq -r '.data[].id'

If the probe fails or the listed ids don't include ${VLM_MODEL}, fall back to the other backend (or surface the error — never silently pick a model that isn't on the server).