Video Analysis

What it does

When your agent uses it

• The user has a video that needs sound but nobody knows yet what it should sound like
• A first generation missed and you need a better prompt rather than another reroll

Example prompts

• “/video-analysis”

Requirements

• Python 3
• Node.js
• A credential in SONILO_API_KEY
• Compatibility (from SKILL.md): Requires Sonilo through either transport — the MCP server connected, or the `sonilo` CLI installed and signed in — plus credentials: a `sonilo login` sign-in, the hosted OAuth plugin, or SONILO_API_KEY. See the setup-api-key skill.
• Pre-approved tools (allowed-tools): Bash, Read, Write, mcp__sonilo__*

Workflow steps

3 steps, taken from the first numbered list in SKILL.md.

1. 1Sonilo MCP tools visible in this session (analyze_video and friends) — use them. This is the preferred path: it needs no shell, and it is…
2. 2No usable Sonilo MCP tools, but sonilo account exits 0 — use the CLI commands below. Same API, same account, same credential file. Probe…
3. 3Neither — stop and run the setup-api-key skill. Do not call api.sonilo.com with curl to work around it; both transports handle uploads…

What it can do on your machine

Read from SKILL.md and the folder at commit 1ce1bd8. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves these tools, so the agent can use them without asking each time:

  • Bash
  • Read
  • Write
  • mcp__sonilo__*

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Shell commands in SKILL.md call:

  • curl
  • pip
  • npm
  • jq

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  Hosts in commands or code, which the agent is likely to contact:

  • api.sonilo.com

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names these keys or tokens, usually read from environment variables:

  • SONILO_API_KEY

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

• Compatibility

  Requires Sonilo through either transport — the MCP server connected, or the `sonilo` CLI installed and signed in — plus credentials: a `sonilo login` sign-in, the hosted OAuth plugin, or SONILO_API_KEY. See the setup-api-key skill.

  From compatibility in the SKILL.md frontmatter.

Context cost

Video Analysis loads about 3.5k tokens when it runs. Until then it costs about 162 tokens; SKILL.md has 1,467 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~162

When it runs · the whole SKILL.md, loaded when a task matches

~3.5k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

• NotePre-approves every shell command (allowed-tools: Bash)SKILL.md

  allowed-tools: Bash, Read, Write, mcp__sonilo__*

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from sonilo-ai/skills at commit 1ce1bd8, republished under its MIT licence (© sonilo-ai). 1,467 words, ~3,481 tokens.

name

video-analysis

description

Analyze a video with Sonilo and get back a creative brief for its sound, derived from the footage itself — by default both a music-direction brief (a time-aligned section plan plus one or more ready-to-use generation prompts) and a sound-design brief (shot-sized SFX segments plus one whole-clip SFX prompt); `mode` picks just one. Use when the user has a video that needs sound but nobody knows yet what it should sound like, or when a first generation missed and you need a better prompt rather than another reroll. Generates no audio and no video; the output is text you feed into video-to-music, video-to-sfx, or video-to-sound.

allowed-tools

Bash, Read, Write, mcp__sonilo__*

compatibility

Requires Sonilo through either transport — the MCP server connected, or the `sonilo` CLI installed and signed in — plus credentials: a `sonilo login` sign-in, the hosted OAuth plugin, or SONILO_API_KEY. See the setup-api-key skill.

license

MIT

analyze_video(
    video_path="~/Desktop/trailer.mp4",
    prompt="focus on the chase",
    variants_num=2
)

analyze_video(video_path="~/Desktop/trailer.mp4", mode="sfx")

from sonilo import Sonilo

client = Sonilo()  # reads SONILO_API_KEY

brief = client.video_analysis.analyze(
    video="trailer.mp4",
    prompt="focus on the chase",
    variants_num=2,
)

for segment in brief.segments:
    print(f"{segment.start}-{segment.end}s [{segment.label}] {segment.prompt}")
print(brief.sfx_prompt)  # the whole-clip sound-design prompt (present in mode "both")

# Feed a variation's prompt straight into a generation call.
score = client.video_to_music.generate(
    video="trailer.mp4", prompt=brief.variations[0].prompt
)
score.save("score.m4a")

# Only the sound-design brief:
sfx_brief = client.video_analysis.analyze(video="trailer.mp4", mode="sfx")

import { SoniloClient } from "sonilo";

const client = new SoniloClient(); // reads SONILO_API_KEY

const brief = await client.videoAnalysis.analyze({
  video: "./trailer.mp4",
  prompt: "focus on the chase",
  variantsNum: 2,
});

const score = await client.videoToMusic.generate({
  video: "./trailer.mp4",
  prompt: brief.variations![0]!.prompt,
});

const sfx = await client.videoToSfx.generate({
  video: "./trailer.mp4",
  prompt: brief.sfx_prompt!, // the whole-clip sound-design prompt
});

// Only the sound-design brief:
const sfxBrief = await client.videoAnalysis.analyze({ video: "./trailer.mp4", mode: "sfx" });

sonilo video-analysis --video trailer.mp4 --prompt "focus on the chase" --variants 2
sonilo video-analysis --video trailer.mp4 --mode sfx   # only the sound-design brief

sonilo video-analysis --video trailer.mp4 --output brief.json
sonilo video-to-music --video trailer.mp4 --prompt "$(jq -r '.variations[0].prompt' brief.json)"

curl -X POST "https://api.sonilo.com/v1/video-analysis" \
  -H "Authorization: Bearer $SONILO_API_KEY" \
  -F "video=@trailer.mp4" \
  -F "variants_num=2"
# -> 202 {"task_id": "...", "status": "processing"}
# Add -F "mode=sfx" (or "mode=music") for a single brief; omitted = both.

curl "https://api.sonilo.com/v1/tasks/<task_id>" -H "Authorization: Bearer $SONILO_API_KEY"

{
  "task_id": "…",
  "status": "succeeded",
  "segments": [
    {"start": 0, "end": 12, "label": "intro", "prompt": "sparse piano, rising"},
    {"start": 12, "end": 30, "label": "none", "prompt": "full strings, driving"}
  ],
  "variations": [
    {"prompt": "cinematic strings, 90bpm, building to a brass hit"},
    {"prompt": "lo-fi hip hop, warm keys, steady throughout"}
  ],
  "mode": "both",
  "sfx_segments": [
    {"start": 0, "end": 4, "label": "none", "prompt": "wind across an empty lot, distant traffic hum"},
    {"start": 4, "end": 12, "label": "none", "prompt": "car door slam, engine turning over, tires on gravel"}
  ],
  "sfx_prompt": "urban chase: engine roar, tires skidding on wet asphalt, passing sirens, metal scrape on impact"
}

© sonilo-ai, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in video-analysis of sonilo-ai/skills.

Open the folder on GitHubat commit 1ce1bd8