Amc Run Video Calibration

Skill: Calibrate from Video Files

When to Use This Skill

Activate this skill when the user has pre-recorded MP4 files and wants to calibrate them via the AMC REST API. Typical prompts:

• "calibrate my videos" / "run AMC on these videos"
• "calibrate from video files"

Drives calibration through the REST API on user-supplied pre-recorded MP4 files — no CLI scripts or Docker bind-mounts required, just a running microservice and your files.

Do not use this skill for live RTSP streams or rtsp://... URLs; route those requests to skills/amc-run-rtsp-calibration/SKILL.md.

Purpose

Guide the agent through project creation, sorted MP4 upload, local asset resolution, UI fallback only when necessary, project verification, calibration, polling, evaluation, and optional VGGT refinement for a user-provided multi-camera dataset.

Prerequisites

• AMC microservice and UI running (follow skills/amc-setup-calibration-stack/SKILL.md)
• You know the microservice URL (use https://<HOST_IP>:<MS_PORT> for remote AMC, or http://localhost:<MS_PORT> on loopback) and UI URL
• Video files locally as contiguous cam_00.mp4, cam_01.mp4, … time-synchronized, ~1920×1080
• Python 3 with requests
• If AMC stores project outputs outside the default projects/ directory, you know the host PROJECTS_DIR

Interaction Model

• The "host's question mechanism" means the runtime's built-in prompt API for short user decisions, such as terminal stdin, an IDE ask tool, or an equivalent interactive dialog.
• If that mechanism is unavailable, ask in chat and wait before any guarded step that requires user confirmation or a missing-file decision.
• For unattended runs, the bundled script requires all non-UI inputs up front and exits before /calibrate unless CONFIRM_CALIBRATION=true is set. RUN_VGGT=true remains a separate opt-in for the optional VGGT step.

Data Privacy

Video files uploaded via this skill are transmitted to the AutoMagicCalib backend (REST endpoint). Only use this skill when the backend is deployed on a trusted platform / network.

Inputs

• Required inputs: VIDEO_DIR, BASE_URL, and PROJECT_NAME.
• Optional local inputs: CONFIG_FILE, ALIGNMENT_JSON, LAYOUT_PNG, GT_ZIP, FOCAL_LENGTHS, and DETECTOR_TYPE.
• Optional control inputs: CONFIRM_CALIBRATION, RUN_VGGT, PROJECTS_DIR, CALIBRATION_TIMEOUT_SECONDS, and VGGT_TIMEOUT_SECONDS.
• BASE_URL should use HTTPS for non-loopback hosts. Set ALLOW_INSECURE_HTTP=true only for trusted development setups that intentionally use remote plain HTTP.
• Resolution precedence for settings, alignment, and layout: explicit path, single local auto-detected match, then UI fallback.

What to Ask the User

Required

(Video-file naming and the microservice URL are specified under Prerequisites above — collect the inputs below.)

1. Videos directory — the folder the skill globs for cam_*.mp4, uploaded sorted alphabetically.
2. Microservice URL
3. Project name — short descriptive string

Auto-Detected (ask only if not found)

The script searches the videos dir, its first-level subdirectories, and its parent. If exactly one match is found, it is used; otherwise the script prints the searched locations and continues to explicit path or UI fallback:

Posting the settings file replaces UI Step 3 and may pin the detector (resnet/transformer), which is passed to /calibrate separately — see Step 4.

Optional

4. Ground truth zip — GT.zip with _World_Cameras_Camera_XX/ folders (enables evaluation metrics)
5. Focal lengths — one per camera, e.g. 1269.0, 1099.5, 1099.5
6. Detector type — resnet (default, fast) or transformer (slower, better under occlusion)
7. Run VGGT refinement? — if VGGT is ready after AMC completes, ask the user whether to run refinement (see setup skill)

See root README.md "Custom Dataset" section for input-video guidelines and ground-truth format.

Available Scripts

Instructions

All endpoints below are implemented end-to-end in the Complete Python Script — the prose is the workflow plus the decisions the agent must make; the script is the authoritative runnable.

Step 1 — Create Project

POST /v1/create_project (form field project_name) → save the returned project_id.

Step 2 — Upload Videos (required)

POST /v1/upload_video_files/<project_id> (multipart files). Upload sorted alphabetically — the server assigns camera indices by upload order. The bundled script rejects non-contiguous or non-zero-based camera sequences up front; the directory must contain cam_00.mp4, cam_01.mp4, ... with no gaps.

Step 3 — Resolve Local Files (Auto-Scan, Ask, or UI)

For each of calibration-settings, alignment, and layout, run this resolution:

Auto-use rule: if exactly one match is found, the script uses it automatically and prints the resolved path. No extra prompt occurs for that file.

1. Auto-scan VIDEO_DIR, one level of subdirectories under VIDEO_DIR, and VIDEO_DIR.parent for the candidate filenames (table above).
2. If exactly one match, use it and print what was found.
3. If zero or multiple matches, print the searched locations, then ask the user for an explicit path using the host's question mechanism; if none is available, ask in chat and wait. If they don't have the file, mark it for UI fallback.
4. UI fallback: tell the user to complete the corresponding UI step; wait for confirmation; then continue to Step 6 and treat verify_project as the source of truth for whether the UI-supplied alignment/layout data is complete.

Step 4 — Upload Resolved Files

Upload each file resolved locally:

Upload all resolved local files first, in any order. After the local uploads are complete, continue to Step 5 only for unresolved files, then run Step 6 exactly once to verify the assembled project.

After a successful settings POST, parse the file for "detector" / "detector_type" — if it's "resnet" or "transformer", use that value for the /calibrate call in Step 7 (detector is a separate API parameter, not consumed by /config).

Step 5 — UI Fallback (only for files the user doesn't have locally)

If any of settings / alignment / layout was not resolved in Step 3, direct the user to the appropriate UI step:

• Settings missing → "Open UI project <project_id>, go to Step 3: Parameters, tune via the settings dialog (or accept defaults), click Save." Also: before the /calibrate call, ask the user which detector to use (resnet or transformer) using the host's question mechanism; if none is available, ask in chat and wait. UI Step 3 does not cover detector choice.
• Alignment or layout missing → "Open UI project <project_id>, go to Step 4: Alignment, upload layout, mark correspondence points, click Save."

Wait for user confirmation. For non-interactive script runs, provide the needed files up front; the script exits with a clear message rather than waiting on input. Do not require local access to AMC's projects/ storage for the UI fallback; Step 6 is the canonical server-side verification step.

Step 6 — Verify Project

POST /v1/verify_project/<project_id> → must return {"project_state": "READY"} before calibrating.

Step 7 — Start Calibration

Confirm the plan before calibrating. Whether the settings file and detector were auto-detected or asked, present a short summary and get explicit user confirmation before POST /calibrate using the host's question mechanism; if none is available, ask in chat and wait. The resolved values are the defaults, so confirming is one click, but the user can switch the detector or skip an auto-detected settings file. The standalone Python script prompts when stdin is interactive; in non-interactive runs it exits before /calibrate unless CONFIRM_CALIBRATION=true is set. Summarize:

• Detector — resnet or transformer (the value to be sent).
• Calibration settings — the file being applied (path), or "defaults" if none.
• Optional overrides — ground-truth zip and focal lengths, if any.

POST /v1/calibrate/<project_id>
Content-Type: application/json

{"detector_type": "resnet"}