Vss Deploy Profile

VSS Deploy

Available Scripts

Profile Routing

Match the user's request to a profile, then load that profile's reference for sizing, services, env recipes, and debugging.

Edge hardware routing (DGX Spark, AGX/IGX Thor): see references/edge.md. DGX Spark uses the Spark Nano 9B standalone local LLM on port 30081; AGX/IGX Thor uses the Edge 4B standalone vLLM fallback.

Each profile's reference owns its sizing table. Don't pick a deployment shape from this file — open the profile reference and check minimum GPU count for the host's hardware against the (mode × platform) matrix there.

Instructions

The deployment flow is always: copy .env to generated.env, apply overrides, dry-run compose into resolved.yml, review, normalize, deploy, then wait for readiness.

# 1. cp dev-profile-<profile>/.env dev-profile-<profile>/generated.env  (clean copy)
# 2. Apply env overrides to generated.env  (source .env stays untouched)
# 3. docker compose --env-file generated.env config > resolved.yml      (dry-run)
# 4. Review resolved.yml
# 5. docker compose --env-file generated.env -f resolved.yml up -d

.env is read-only checked-in defaults; generated.env is the per-deploy working copy. Step 1c covers this in full.

Prerequisites

1. Repo path — auto-detect video-search-and-summarization/ before asking the user. Use the detected path as $REPO for all subsequent commands.
2. Credential gates — see references/credentials.md: NGC_CLI_API_KEY for local/local_shared NIM pulls, NVIDIA_API_KEY for remote NIM endpoints, and HF_TOKEN for edge recipes that use gated HF models.
3. System prerequisites (GPU driver, Docker, NVIDIA Container Toolkit, kernel sysctls, and — if ufw is active — the Docker-bridge→host firewall allow so bridge NIMs can fetch clips from host-mode VST) — full checks in references/prerequisites.md. Canonical hardware/driver matrix is the VSS prerequisites page.

The auto-detect snippet (git-root, then a common-path probe gated on deploy/docker/compose.yml + dev-profile.sh + skills/vss-deploy-profile) lives in references/prerequisites.md. Export the resolved $REPO; if detection fails, ask the user for the checkout path.

Pre-flight check

Run before every deploy. The full system checklist and remediation steps live in references/prerequisites.md. For DGX Spark / IGX Thor / AGX Thor, also run the cache-cleaner check in references/edge.md.

Detect sudo mode first. Several pre-flight remediations and the edge cache-cleaner installer call sudo. If the host requires a sudo password, those steps will silently no-op under sudo -n and leave the deploy in a half-prepared state.

if sudo -n true 2>/dev/null; then
  echo "passwordless sudo — pre-flight will auto-install missing pieces"
else
  echo "sudo requires password — pre-flight will NOT auto-install; hand commands to the user"
fi

When sudo needs a password, the skill must not run privileged installers itself. Surface the copy-pasteable command block from references/prerequisites.md to the user with a "run this once and confirm" handoff, then resume after the user replies.

Minimum smoke test (must succeed):

nvidia-smi --query-gpu=index,name --format=csv,noheader
docker info 2>/dev/null | grep -qi runtimes \
  && docker run --rm --gpus all ubuntu:22.04 nvidia-smi >/dev/null 2>&1 \
  && echo "nvidia runtime OK"

If the smoke test fails, do not proceed; open references/prerequisites.md for the remediation tree.

Model Selection

• $LLM_REMOTE_URL / $VLM_REMOTE_URL if the user asks for remote
• $NGC_CLI_API_KEY (local NIMs) or $NVIDIA_API_KEY (remote)

Endpoint intent gate. Don't infer remote placement from stray env vars (LLM_ENDPOINT_URL, VLM_ENDPOINT_URL, LLM_BASE_URL, VLM_BASE_URL may be leftovers). Use remote LLM/VLM only when (1) the user asked for / supplied a remote endpoint, (2) local sizing can't fit the selected models and the user agrees, or (3) an edge recipe needs a standalone local service VSS treats as remote (e.g. DGX Spark Nano 9B on localhost:30081). If an endpoint var is set but the user didn't ask for remote, surface it in Step 1 and ask — never silently deploy remote because a var happened to exist.

If no combination on this host satisfies the profile's sizing requirements, stop and report the blocker — don't silently pick another shape.

Edge shared mode is platform-specific. Full recipes are in references/edge.md.

Deployment Flow

Always follow this sequence. Never skip the dry-run.

Step 0 — Tear down any existing deployment + clear data volumes

If a deployment already exists, tear it down AND clear stale data volumes before redeploying.

Full procedure lives in references/teardown.md.

Step 0a — Credentials gate (run before any env mutation)

Validate every credential and selected remote endpoint the chosen profile needs before Step 1c copies .env to generated.env. A 401 here is a 30-second failure; the same 401 inside a NIM cold-start is a 10–20 min failure. Run the discovery and probe flow in references/credentials.md, including scripts/probe_remote_models.sh for any LLM/VLM endpoint you plan to write into generated.env. Map the result against the chosen mode: missing or invalid required credentials/endpoints are blockers, optional credentials are not.

Step 1 — Gather context

Before building env overrides, confirm:

Before docker compose up, verify EXTERNAL_IP, HAPROXY_PORT, VSS_PUBLIC_HOST, and VSS_PUBLIC_PORT are populated with browser-reachable values. Otherwise the stack may appear healthy while UI/API/VST links 404 or loop through Cloudflare Access.

Step 1b — Prepare the data directory

Layout (asset paths, ownership, mount points, profile-specific subdirs) is documented in references/data-directory.md. Read that file before deploying for the first time on a host or when changing profiles.

Step 1c — Initialize generated.env

The skill's per-deploy working copy. Always start from a fresh copy of the source .env , never mutate the source.

PROFILE=base
ENV_SRC=$REPO/deploy/docker/developer-profiles/dev-profile-$PROFILE/.env
ENV_GEN=$REPO/deploy/docker/developer-profiles/dev-profile-$PROFILE/generated.env

cp "$ENV_SRC" "$ENV_GEN"

All subsequent writes (Brev EXTERNAL_IP, the env_overrides dict from Step 2) go to $ENV_GEN. $ENV_SRC is read-only from here on.

Step 1d — Brev only: detect first, then set EXTERNAL_IP to the secure-link domain

Detect Brev before anything else — a Brev-provisioned instance sets BREV_ENV_ID in /etc/environment; nothing else does:

grep -qE '^BREV_ENV_ID=' /etc/environment && echo "on Brev" || echo "not Brev"

• not Brev → skip the rest of this step and do not read references/brev.md; keep the normal ${HOST_IP}-based EXTERNAL_IP.
• on Brev → apply the Brev secure-link overrides from references/brev.md § Setup flow to generated.env (NOT .env). Those set EXTERNAL_IP / VSS_PUBLIC_HOST to the secure-link domain and VSS_PUBLIC_HTTP_PROTOCOL=https / VSS_PUBLIC_WS_PROTOCOL=wss / VSS_PUBLIC_PORT=443 — setting EXTERNAL_IP alone leaves http://…:7777 UI/API/WS links that the browser blocks as mixed content.

Step 2 — Build env_overrides

Produce an env_overrides dict from the user request and the gathered context: explicitly choose remote/local LLM/VLM, set credentials, point at endpoints, set platform-specific flags. Do not let existing shell env vars silently pick placement; write the selected LLM_MODE / VLM_MODE and matching endpoint/model fields into generated.env. The full mapping (every override key, when it applies, defaults, profile-specific differences) lives in references/env-overrides.md. Each profile reference has worked examples for that profile's common scenarios.

Step 3 — Apply overrides + dry-run

Working env file: <repo>/deploy/docker/developer-profiles/dev-profile-<profile>/generated.env (created in Step 1c).

Reminder (see Step 1c): apply all overrides (Step 2 dict + Brev EXTERNAL_IP) to generated.env; --env-file always points at it, and post-deploy verifiers read it for the actually-deployed values.

# (Step 1c already ran: cp $ENV_SRC $ENV_GEN)

# Apply the env_overrides dict from Step 2 to generated.env
# (read lines, update matching keys, append new keys, write)
# Example:
#   sed -i "s|^LLM_MODE=.*|LLM_MODE=remote|" "$ENV_GEN"
#   sed -i "s|^LLM_BASE_URL=.*|LLM_BASE_URL=http://localhost:30081|" "$ENV_GEN"

# Resolve compose
cd $REPO/deploy/docker
docker compose --env-file $ENV_GEN config > resolved.yml

The resolved YAML is saved to <repo>/deploy/docker/resolved.yml.

Step 3b — Verify resolved.yml has no unexpanded ${...} tokens

Unexpanded ${VAR} tokens in resolved.yml mean compose did not see those env values. Diagnostic procedure and common culprits live in references/troubleshooting.md.