Media Gen

media-gen

Generate and edit images on demand by calling the local Clacky HTTP server, which dispatches to whichever image-generation model the user configured (type=image in their model settings). Editing (image-in → image-out) works with any image model that accepts image input — most current ones do.

Endpoint

POST http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/image
GET  http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/types

Step 1 — Verify a backend is configured

Before generating anything, confirm the user has a type=image model set up:

curl -s http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/types

If the response shows image.configured = false, stop and tell the user:

还没有配置生图模型。请打开设置页 → 添加模型 → 类型选 image（走 openclacky 官方网关时推荐 or-gemini-3-pro-image 或 or-gpt-image-2）。配好后再让我生图。

Do NOT try to fall back to terminal + a hand-written curl https://api.openai.com/... — that bypasses the user's configured backend and won't be billed correctly.

You do NOT configure models — the user does, in the settings page. Never edit the user's config.yml to add or change a model, and never invent a model name from memory (e.g. or-gpt-5.4-image-2 does not exist). The real, current model is whatever /api/media/types reports under image.model. If you think a different model is needed, tell the user which one to set in the settings page — don't touch the config file yourself.

Step 2 — Generate the image

The model does NOT honor exact pixel sizes

There is no size / width / height field — the only shape control is aspect_ratio (landscape / square / portrait), and even that is just a rough hint (ask for 576x96 and you may get 1408x768). When the user needs an exact pixel size, a grid, an icon at NxN, or a spritesheet, generate first at whatever size the model gives, then resize / crop / tile to the exact pixels with ImageMagick (magick). Verify with magick identify before reporting done.

Important: generation speed & concurrency

• Image generation can be slow — up to 2 minutes per image depending on the model. Before calling the API, warn the user that it may take a minute or two. The curl request blocks until the image is ready; do NOT run it in the background.
• One at a time only. Never generate multiple images concurrently (e.g. by running several curl commands simultaneously or in a script loop). Each call consumes significant server-side resources, and parallel requests will almost certainly cause timeouts. If the user wants several images, generate them sequentially, one after another.

curl -s -X POST http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/image \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "A clean, modern hero illustration for a tech startup landing page. Soft gradient background, abstract geometric shapes in blue and purple, minimal style, 4K quality.",
    "aspect_ratio": "landscape",
    "output_dir": "'"$(pwd)"'/assets/generated",
    "session_id": "<%= session_id %>"
  }'

• The terminal blocks multi-line commands — write the request into a .sh file and run it, don't paste a multi-line curl.
• If a call fails with 400 / INVALID_ARGUMENT, drop the aspect_ratio field and retry once before reporting the error.
• If a call fails with unknown image model (400), the configured model name isn't recognized by its backend — tell the user to fix the model name in the settings page; do NOT guess another name and retry.

Content-safety refusal (content_filter)

If a call fails with no image and mentions content_filter / safety / blocked (or HTTP 422), the prompt was blocked. Don't resend as-is — rephrase the prompt (soften the sensitive part, keep the intent) and retry, up to 3 times, checking each returned image. Still blocked? Stop and ask the user to adjust.

Request fields

Input / reference image format

Images passed as image / images must be PNG, JPEG, or WebP. Other formats (SVG, GIF, BMP, TIFF, …) are rejected upstream and fail the call — convert them to one of the supported formats first.

Editing an existing image

To edit instead of generate from scratch, pass the existing image as image (a local file path is easiest — the skill reads and encodes it for you) plus a prompt describing the change. The configured image model receives the image alongside the prompt and returns an edited result.

curl -s -X POST http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/image \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "change the background to a starry night sky, keep the cat unchanged",
    "image": "/abs/path/to/input.png",
    "session_id": "<%= session_id %>"
  }'

• The result is a new edited image saved next to the inputs in output_dir — the original file is never modified in place.
• For combining several inputs (e.g. "put the product from image 1 onto the background from image 2"), pass them as images: ["/path/a.png", "/path/b.png"] and describe the composition in the prompt.
• Same speed/concurrency rules apply: editing is as slow as generation, one at a time.

Response shape (success)

{
  "success": true,
  "image": "/abs/path/to/working_dir/assets/generated/img_20260525_011820_a1b2c3d4.png",
  "model": "<the configured image model>",
  "provider": "openclacky",
  "prompt": "A clean, modern hero illustration ...",
  "aspect_ratio": "landscape",
  "size": "1536x1024",
  "usage": {
    "prompt_tokens": 50,
    "completion_tokens": 4500,
    "cache_read_tokens": 0,
    "cache_write_tokens": 0,
    "total_tokens": 4550
  }
}

The image field is an absolute path on disk. To embed it in markdown, slides, or HTML, convert it to a path relative to the document you're writing.

usage may be absent when the configured backend doesn't return token counts. Treat it as optional.

Response shape (failure)

{
  "success": false,
  "image": null,
  "error": "Upstream 401: Invalid API key",
  "error_type": "api_error",
  "model": "...",
  "provider": "..."
}

Common error_type values: not_configured, auth_required, network_error, api_error, empty_response. Tell the user the error plainly; if it's auth_required or api_error 401/403, point them at settings to fix the api_key.

Step 3 — Show the image

Read does NOT show the image to the user — it only feeds it into your own context. To make the user actually see it, write a markdown tag in your reply:

![](file:///abs/path/from/response.png)

Take the image field from the response and prefix file:// (three slashes, since the path is absolute).

If you're also embedding it in a document (README, PPT, etc.), convert the returned path to one relative to that document.

Prompt writing tips

A good image prompt has 4 layers, in this order:

1. Subject — what is in the image, concretely. ("a golden retriever puppy", "a stylized icon of a rocket")
2. Style / medium — photo / illustration / 3D render / watercolor / flat vector / line art
3. Composition / lighting — close-up / wide shot / overhead / soft natural light / dramatic backlight
4. Mood / palette — minimal / playful / corporate / pastel / high-contrast monochrome

For PPT / slide decks specifically:

• Hero / cover slides: aspect_ratio: landscape, prompt should emphasise "clean", "minimal", "negative space" so text overlays well
• Section dividers: aspect_ratio: landscape, abstract or pattern-style works better than literal subjects
• Inline figures: aspect_ratio: square or portrait, more literal subject is fine

When the user gives a vague request like "给我配张图", ask one clarifying question (subject? style?) before calling the API — costs real money per image.

When NOT to use this skill

• The user wants a diagram / chart with specific data — use a charting library (matplotlib, mermaid, etc.) instead; image gen is for illustrations, not data viz
• The user asks for screenshots of real software — use the browser tool

Generating video (Veo)

The same /api/media/ namespace serves video generation. The user must configure a type=video model in settings (recommended: or-veo-3-1).

Endpoint

POST http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/video

Check GET /api/media/types first — if video.configured = false, tell the user to add a type=video model in settings before generating.

Video is slow and expensive

• A single clip can take 1–3 minutes (sometimes longer). Warn the user before calling, and run the curl in the foreground — it blocks until the MP4 is ready. Do NOT background it.
• One at a time. Never run multiple video generations concurrently.
• Each clip costs real money (billed per output-second). Confirm the prompt with the user before generating.

Request

curl -s -X POST http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/video \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "A cinematic drone shot flying over a misty mountain range at sunrise, golden light, 4K.",
    "aspect_ratio": "landscape",
    "duration_seconds": 8,
    "output_dir": "'"$(pwd)"'/assets/generated",
    "session_id": "<%= session_id %>"
  }'

Response (success)

{
  "success": true,
  "video": "/abs/path/to/working_dir/assets/generated/vid_20260615_011820_a1b2c3d4.mp4",
  "model": "or-veo-3-1",
  "provider": "openclacky",
  "prompt": "A cinematic drone shot ...",
  "aspect_ratio": "landscape",
  "duration_seconds": 8,
  "cost_usd": 2.688
}

The video field is an absolute path on disk. Show it to the user with a markdown link or an HTML5 <video> tag pointing at the file:// path; embed it in documents with a path relative to the document.

Response (failure)

Same shape and error_type values as image generation, but with "video": null. not_configured means no type=video model is set up.

Continuous / long video (last-frame chaining)

A single Veo call maxes out at 8 seconds, and separate calls are visually unrelated (the character, lighting and framing jump between clips). To make several clips flow as one continuous shot, chain them: take the last frame of clip N and feed it as the image (first frame) of clip N+1. Veo's image-to-video then continues from exactly where the previous clip ended, so the seam is smooth.

Use the helper script (it only does the ffmpeg mechanics — you drive the generation with the same /api/media/video curl as above). The script's absolute path is given in the Supporting Files block; assign it once:

SEQ="SKILL_DIR/scripts/video_seq.sh"   # SKILL_DIR is provided in Supporting Files
# subcommands: lastframe | tob64 | payload | concat | probe

Workflow for an N-segment continuous video:

1. Plan the shots. Split the story into 4–8s beats. Write one prompt per beat; each prompt should describe the continuation, e.g. "The same girl keeps walking forward, the camera pushes in…". Keep subject, style and lighting wording consistent across prompts.
2. Segment 1 — normal text-to-video call. Save the returned mp4 path.
3. Extract its last frame (as JPEG — keep the .jpg extension):

   "$SEQ" lastframe seg1.mp4 /tmp/seg1_last.jpg

4. Segment 2 — build the request body with payload, then post it with curl --data @file. Do NOT inline the base64 into -d "{…}" — a frame's base64 is ~150KB+ and overflows the shell's argument limit ("Argument list too long"). The payload subcommand reads the frame, base64-encodes it, and writes a ready-to-send JSON file:

   "$SEQ" payload /tmp/seg2.json /tmp/seg1_last.jpg 8 landscape "$OUT_DIR" \
     "Continuing the same scene, the camera keeps pushing forward…" "<%= session_id %>"
   curl -s -X POST .../api/media/video -H "Content-Type: application/json" \
     --data @/tmp/seg2.json

   (payload <out.json> <frame> <duration_seconds> <aspect_ratio> <output_dir> <prompt> [session_id], where OUT_DIR is the same directory you'd pass as output_dir above)
5. Repeat steps 3–4 for each subsequent segment, always chaining off the previous segment's last frame.
6. Stitch all clips in order into one file:

   "$SEQ" concat final.mp4 seg1.mp4 seg2.mp4 seg3.mp4

Rules & caveats:

• Strictly sequential. Generate one segment, wait for it, extract its frame, then start the next. Never run two video generations at once.
• Keep prompts consistent. The image carries visual continuity, but the prompt must not contradict it (don't switch the subject or scene mid-chain unless you intend a cut).
• Aspect ratio must match across all segments, or concat falls back to a slower re-encode (and may letterbox). Use the same aspect_ratio everywhere.
• Cost adds up linearly — N segments ≈ N × single-clip price. Confirm the number of segments and total length with the user before starting.
• For >30s or a true single-take >8s with no seam at all, this client-side chaining is the practical option today; Veo's native server-side extend (148s) is not wired into this endpoint yet.