Capy Video Gen Skill

What it does

When your agent uses it

• The user asks to create a video from a script
• Wants multi-shot video with consistent characters

Example prompts

• “/capy-video-gen-skill”

Requirements

• Python 3
• A credential in AI_GATEWAY_API_KEY
• Pre-approved tools (allowed-tools): Bash, Read, Write, Edit

Workflow steps

8 steps, taken from the first numbered list in SKILL.md.

1. 1Extract characters from script with detailed physical features
2. 2Generate front/side/back character portraits
3. 3Design shot-by-shot storyboard
4. 4Decompose each shot into first_frame, last_frame, and motion descriptions
5. 5Build camera tree for shot relationships
6. 6Generate frames with reference image selection (face identity as top priority)
7. 7Generate video clips from frames
8. 8Concatenate into final video

What it can do on your machine

Read from SKILL.md and the folder at commit 9ff72fe. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves these tools, so the agent can use them without asking each time:

  • Bash
  • Read
  • Write
  • Edit

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Ships script files (Python, from the files we listed), which the agent can run.

  Shell commands in SKILL.md call:

  • python

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  Hosts in commands or code, which the agent is likely to contact:

  • ai-gateway.happycapy.ai

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names these keys or tokens, usually read from environment variables:

  • AI_GATEWAY_API_KEY

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Capy Video Gen Skill loads about 2.4k tokens when it runs. Until then it costs about 100 tokens; SKILL.md has 673 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~100

When it runs · the whole SKILL.md, loaded when a task matches

~2.4k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

• NotePre-approves every shell command (allowed-tools: Bash)SKILL.md

  allowed-tools: Bash, Read, Write, Edit

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from happycapy-ai/Happycapy-skills at commit 9ff72fe, republished under its MIT licence (© happycapy-ai). 673 words, ~2,426 tokens.

name

capy-video-gen-skill

description

Multi-shot AI video generation pipeline with face identity consistency. Converts scripts or ideas into complete videos using character extraction, storyboarding, frame generation, and video assembly. 300 experiments validated, 70% face distance improvement. Use when the user asks to create a video from a script, story, idea, or wants multi-shot video with consistent characters.

allowed-tools

Bash, Read, Write, Edit

cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax

cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax
.venv/bin/python main_happycapy_script2video.py

cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax
.venv/bin/python main_happycapy_idea2video.py

import asyncio
from langchain.chat_models import init_chat_model
from tools.render_backend import RenderBackend
from utils.config_loader import load_config
from pipelines.script2video_pipeline import Script2VideoPipeline

config = load_config("configs/happycapy_script2video.yaml")
chat_model = init_chat_model(**config["chat_model"]["init_args"])
backend = RenderBackend.from_config(config)

pipeline = Script2VideoPipeline(
    chat_model=chat_model,
    image_generator=backend.image_generator,
    video_generator=backend.video_generator,
    working_dir=config["working_dir"],
)

# Run the pipeline
asyncio.run(pipeline(
    script="Your script here...",
    user_requirement="No more than 8 shots total.",
    style="Cinematic, warm lighting"
))

chat_model:
  init_args:
    model: gpt-4.1
    model_provider: openai
    api_key: ${AI_GATEWAY_API_KEY}
    base_url: https://ai-gateway.happycapy.ai/api/v1/openai/v1

image_generator:
  class_path: tools.ImageGeneratorHappyCapyAPI
  init_args:
    api_key: ${AI_GATEWAY_API_KEY}
    model: google/gemini-3.1-flash-image-preview

video_generator:
  class_path: tools.VideoGeneratorHappyCapyAPI
  init_args:
    api_key: ${AI_GATEWAY_API_KEY}
    model: google/veo-3.1-generate-preview

working_dir: .working_dir/script2video

.working_dir/script2video/
  characters.json                      # Extracted characters
  character_portraits_registry.json    # Portrait paths registry
  character_portraits/                 # Generated portraits
    0_CharacterName/
      front.png
      side.png
      back.png
  storyboard.json                     # Shot descriptions
  camera_tree.json                    # Camera relationships
  shots/
    0/
      shot_description.json
      first_frame.png
      last_frame.png (if medium/large variation)
      video.mp4
    1/
      ...
  final_video.mp4                     # Final concatenated output

# Build a portrait registry pointing to your photos
character_portraits_registry = {
    "Alice": {
        "front": {"path": "/path/to/alice_front.png", "description": "Front view of Alice"},
        "side": {"path": "/path/to/alice_side.png", "description": "Side view of Alice"},
        "back": {"path": "/path/to/alice_back.png", "description": "Back view of Alice"},
    }
}

# Pass to pipeline (skips portrait generation)
await pipeline(
    script=script,
    user_requirement=user_requirement,
    style=style,
    character_portraits_registry=character_portraits_registry,
)

cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax
.venv/bin/python main_happycapy_script2video.py

© happycapy-ai, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 74 other files (assets) in skills/capy-video-gen-skill of happycapy-ai/Happycapy-skills.

• SKILL.md
• .gitignore
• Communication.md
• FACE_IDENTITY_GUIDE.md
• LICENSE
• README.md
• README_ZH.md
• TODO
• agents/__init__.py
• agents/best_image_selector.py
• agents/camera_image_generator.py
• agents/character_extractor.py
• agents/character_portraits_generator.py
• agents/event_extractor.py
• agents/global_information_planner.py
• agents/novel_compressor.py
• agents/reference_image_selector.py
• agents/scene_extractor.py
• agents/screenwriter.py
• agents/script_enhancer.py
• … and 55 more

Open the folder on GitHubat commit 9ff72fe