Media Generation

Media Generation

Handle image generation, image editing, and short video generation through one workflow: choose the right modality, pass caller intent through to the provider, save outputs under tmp/images/ or tmp/videos/, and prefer the bundled helpers over ad-hoc one-off API calls.

Workflow decision

• If the user wants a brand-new still image, use an image-generation model.
• If the user supplies an image or wants a specific existing image changed, use an image-edit workflow.
• If the user wants motion / a clip / a short video, use a video-generation model.
• If the request includes one or more reference images, use the helper that supports reference-image transport.

Standard workflow

1. Determine whether the task is image generation, image editing, or video generation.
2. Clarify only when required to execute the request correctly.
3. Prefer scripts/generate_image.py for still-image generation.
4. Prefer scripts/edit_image.py for direct image edits.
5. Prefer scripts/mask_inpaint.py for localized edits with masks or generated regions.
6. Prefer scripts/outpaint_image.py for canvas expansion / outpainting.
7. Prefer scripts/generate_consistent_media.py when reference images need to be passed through.
8. Prefer scripts/generate_video.py for video generation, especially when the provider may return async job payloads.
9. Prefer scripts/generate_batch_media.py for repeatable batch jobs, templated variations, or auditable manifests.
10. Prefer scripts/object_select_edit.py for simple object-vs-background edits on transparent assets or clean backdrops.
11. If the provider returns a URL, path, HTML snippet, markdown snippet, data: URL, or b64_json, use scripts/fetch_generated_media.py.
12. Save outputs under:
    • images → tmp/images/
    • videos → tmp/videos/
13. If the user wants files sent in chat, prefer sending the local downloaded file.
14. Keep the original remote reference as fallback when local retrieval fails.

Prompt handling

Default to prompt pass-through.

• Pass the caller's prompt through unchanged.
• Use optional request fields only when the caller provides them.
• Keep prompt semantics under caller control.

Use the scripts mainly as functional helpers:

• normalize arguments
• map fields to provider-specific JSON
• upload files
• poll async jobs
• download returned media
• save outputs under tmp/images/ or tmp/videos/

Delivery rules

• Save generated or edited images in tmp/images/.
• Save generated videos in tmp/videos/.
• Never scatter generated files in the workspace root.
• If message delivery blocks remote URLs, download locally first and then send the local file.
• If a remote file cannot be fetched locally but the raw link may still help, provide the original link clearly.

Image generation helper

Use scripts/generate_image.py for direct still-image generation.

Example:

python3 skills/media-generation/scripts/generate_image.py \
  --prompt 'person' \
  --size '1024x1024' \
  --out-dir 'tmp/images' \
  --prefix 'generated'

The helper:

• reads provider credentials from OpenClaw config (~/.openclaw/openclaw.json by default, or --config / $OPENCLAW_CONFIG)
• calls /images/generations by default
• supports size, quality, style, background, n, seed, extra-json, and extra-json-file
• downloads the returned image into tmp/images/ by default
• handles providers that reply with URL/path, data: URL, or b64_json

Image edit helper

Use scripts/edit_image.py for direct image-edit calls.

Example:

python3 skills/media-generation/scripts/edit_image.py \
  --image 'tmp/images/source.jpg' \
  --prompt 'replace the background' \
  --out-dir 'tmp/images' \
  --prefix 'edited'

The helper:

• reads provider credentials from OpenClaw config
• calls /images/edits by default
• supports optional --mask input for localized edits
• downloads the returned image into tmp/images/ by default
• handles URL/path, data: URL, or b64_json

Mask inpaint helper

Use scripts/mask_inpaint.py for localized repainting tasks.

Example:

python3 skills/media-generation/scripts/mask_inpaint.py \
  --image 'tmp/images/source.jpg' \
  --x 120 --y 80 --width 220 --height 180 \
  --prompt 'replace the masked area' \
  --out-dir 'tmp/images' \
  --prefix 'mask-result'

The helper:

• accepts either an existing --mask image or generated regions
• supports rectangle / ellipse regions and repeatable --region specs
• supports percentage-based regions like rect-pct / ellipse-pct
• supports --expand / --shrink before feathering
• supports --mask-only for local preparation / testing without a live API call
• forwards --config, --provider, --model, and --endpoint to scripts/edit_image.py
• reuses scripts/edit_image.py for the final edit call

Outpaint helper

Use scripts/outpaint_image.py for extension / canvas expansion tasks.

Example:

python3 skills/media-generation/scripts/outpaint_image.py \
  --image 'tmp/images/source.jpg' \
  --left 512 --right 512 --top 128 --bottom 128 \
  --mode blur \
  --prompt 'extend outward' \
  --out-dir 'tmp/images' \
  --prefix 'outpaint-result'

The helper:

• expands the canvas locally before calling the model
• supports directional expansion on each side
• supports transparent, blur, and solid initialization modes
• forwards --config, --provider, --model, and --endpoint to scripts/edit_image.py
• reuses scripts/edit_image.py for the final edit call

Reference-image helper

Use scripts/generate_consistent_media.py when one or more reference images need to be passed through to the provider.

Note: the script name is historical; its current role is reference-image transport and delegation.

Example:

python3 skills/media-generation/scripts/generate_consistent_media.py \
  --mode image \
  --reference-image 'tmp/images/reference.png' \
  --prompt 'character' \
  --size '1024x1024' \
  --out-dir 'tmp/images' \
  --prefix 'reference-output'

The helper:

• can pass encoded reference images in provider JSON (default key: reference_images)
• can retry without provider-json references when transport is auto
• delegates to scripts/generate_image.py or scripts/generate_video.py

Batch generation helper

Use scripts/generate_batch_media.py when the user wants several related outputs, repeatable batch rendering, or a manifest-driven workflow.

Example:

python3 skills/media-generation/scripts/generate_batch_media.py \
  --manifest 'tmp/images/media-batch.jsonl' \
  --vars-json '{"subject":"item"}' \
  --summary-out 'tmp/images/media-batch-summary.json' \
  --continue-on-error \
  --print-json

The helper supports:

• JSON array or JSONL manifests
• image generation, video generation, and reference-image generation
• shared templating vars via --vars-json or --vars-file
• item-local vars objects for per-item string rendering such as {index}
• --summary-out to persist the resolved batch result JSON
• --dry-run to validate a manifest before spending live generation calls