Comfyui Agent Skill Mie

comfyui-agent-skill-mie

Purpose

Run registered ComfyUI workflows through a stable Agent-facing CLI, with prompt enhancement, fail-fast errors, and structured JSON results.

Use this skill when the user asks to:

• Generate an image from text.
• Generate a new image inspired by a reference image.
• Edit an input image while preserving some structure or subject details.
• Generate text-to-video or image-to-video MP4 output.
• Generate music / instrumental / song-style MP3 output.
• Synthesize spoken voice audio with Qwen3-TTS.
• Cut out or mask an object in a picture by describing it in words.
• Transfer a face, expression, or pose from a driving video onto a reference face photo.
• Clone a specific voice from a reference audio sample and speak new text in that voice.
• Check whether a ComfyUI server is available.
• Import a workflow the user already validated in ComfyUI into their private registry, and activate it only after the generated config is reviewed.
• Preflight a registered workflow's nodes and models before a long run.
• Save a persistent ComfyUI server URL when the user asks.

Do not use this skill when the user only wants prompt writing, brainstorming, or discussion without actual generation.

A ComfyUI server being unavailable is not a reason to skip this skill. Return SERVER_UNAVAILABLE, ask whether ComfyUI is running on the local machine or on another machine, and point the user at check (single health check) and doctor (server health plus per-workflow preflight) to diagnose it.

Hard Rules

• Source mode: run CLI commands from the skill root (the directory containing SKILL.md and scripts/).
• Tool-install mode: comfyui-agent-skill-mie / comfyui-skill can be run from any directory.
• Source mode: use uv run --no-sync python -m comfyui (or uv run --no-sync comfyui-skill) for runtime calls.
• Tool-install mode: use comfyui-skill (or comfyui-agent-skill-mie) directly; do not wrap with uv run.
• Use registered workflows only. Do not run arbitrary unreviewed ComfyUI workflow JSON.
• If server health fails, stop generation and return/handle SERVER_UNAVAILABLE; do not search disk for ComfyUI installs or guess ports.
• Do not create or edit config.local.json unless the user explicitly wants a persistent server URL. For one-off runs, use --server or COMFYUI_URL.
• For reference_to_image, inspect the reference image with Agent vision and create a prompt. Do not upload that reference image to ComfyUI.
• Upload local media with the role's flag from the quick command table. --image is the image role. liveportrait also needs --video. qwen3_tts_clone needs -p, --text-input "ref_text=...", and --audio. Do not choose those flags from the capability name.
• Pass --width/--height only when the workflow config maps both dimensions and does not use size_strategy: "workflow_managed"; always pass both together. Otherwise the CLI returns INVALID_PARAM on both the sync and --submit paths instead of ignoring them.
• Analyzer-generated workflow configs require human review before activation.

Workflow Selection Policy

• Built-in defaults are fallback choices, not hard requirements.
• If another registered workflow is a stronger semantic match for the request, prefer the stronger match.
• Use workflow capability metadata and workflow selection guidance to choose among registered workflows.
• Any automatic workflow selection in the CLI is only a low-risk fallback when no explicit workflow has been chosen.
• If multiple workflows appear suitable and the user’s preference is ambiguous, ask a brief clarifying question.

User-Added Workflows

• Users may add custom workflows that were first validated in ComfyUI, then imported into the skill as reviewed registered capabilities.
• Do not execute arbitrary raw workflow JSON directly.
• Import the workflow, generate/review the config template, and activate it only after a reviewed workflow.config.json exists.
• User-added workflows must be stored in the per-user workflow registry so skill upgrades do not overwrite them.
• Once activated, user-added workflows may be selected like other registered workflows when they are a stronger semantic match and required inputs are available.
• import-workflow only accepts API format (ComfyUI's Save (API) button, requires Settings → Enable Dev mode options). A plain Save product (UI/Save format, {nodes, links}) is refused with WORKFLOW_NOT_API_FORMAT. To convert existing UI/Save workflows in place, run convert-ui (needs a running ComfyUI + Playwright).

Setup

Recommended install (tool-install mode):

pipx install comfyui-agent-skill-mie

• Install package: comfyui-agent-skill-mie
• Main command: comfyui-agent-skill-mie
• Short alias: comfyui-skill

Prerequisites:

• ComfyUI server with GET /system_stats available.
• Python 3.10+.
• Source mode only: uv.
• Required ComfyUI models/custom nodes for the selected workflow.

Networking note:

• Default local examples use http://127.0.0.1:8188 for same-environment setups.
• If the agent runs inside WSL/container/sandbox while ComfyUI runs on the host OS, 127.0.0.1 may refer to the runtime itself. Try --server http://localhost:8188 or the host machine IP (and optionally persist it via save-server).

Initial setup from the skill root:

uv sync
uv run --no-sync python -m comfyui --help

Tool-install mode:

comfyui-agent-skill-mie --help
comfyui-agent-skill-mie check
comfyui-skill --help
comfyui-skill check

Quick Workflow Choice

Minimal decision tree:

• User gives text only → start from general T2I (z_image_turbo fallback), but prefer a stronger registered match when workflow guidance indicates one (e.g. posters / anime / artistic styles).
• Poster / text-in-image requests → prefer qwen_image_2512_4step
• Anime / manga / illustration style → prefer anima_turbo
• Artistic / painterly realism / concept art / product → prefer krea2_turbo
• User gives a reference image and wants a new similar image → vision → reference_to_image prompt → run z_image_turbo
• User gives an input image and wants edits → generate --workflow klein_edit --image input_image=... -p "..."
• User wants to mask / cut out an object by describing it → sam3_mat_image
• User wants face/expression transfer (face photo + driving video) → liveportrait
• User wants TTS / voice audio → generate --workflow qwen3_tts --speech-text "..." --instruct "..."
• User wants to clone a specific voice from a sample → qwen3_tts_clone
• User wants video → ltx_23_t2v_distill (text→video) or ltx_23_i2v_distilled (image→video)

For workflow-specific size rules, capability boundaries, and examples, read references/workflows.md.