At Vision

Visual Reasoning Policy

If the prompt already contains actual image content, or a host image viewer returned that content, inspect it directly and do not call inspect_image. A file path or URL alone is not image content.

When only a file path or URL is available, direct inspection fails, or the user explicitly requests the provider, the inspect_image MCP tool (server agent-tools-vision) sends one image plus narrow factual questions to a configured vision model. You stay in charge of reasoning and the final answer; the vision model only reports observations.

inspect_image is a callable MCP tool, not an MCP resource. Call the tool directly. Never call list_mcp_resources or read_mcp_resource for images, and never use inspect_image as a resource URI.

When fallback inspection is needed, prefer inspect_image. If it is not exposed as a callable tool, or the host/model gateway cannot invoke MCP namespace tools, use the host's shell/command execution tool to run the installed fallback.

First use a structured file-write capability to create a temporary JSON request; do not construct it with shell interpolation. Use the same shape as the MCP input:

{
  "image_source": { "type": "file", "value": "<path>" },
  "questions": [{ "id": "q1", "text": "<question>" }]
}

Choose a temporary request path containing no shell metacharacters, then run:

node "{{VISION_CLI_PATH}}" --request-file "<safe-temp-request.json>" --json

Delete the temporary request file afterward. Quote the command for the active shell: in PowerShell, use single-quoted literal arguments and double any embedded '; in POSIX shells, use single quotes and encode an embedded ' as '"'"'. The installed CLI path and agent-chosen temporary path are the only dynamic command arguments; image paths, URLs, and questions belong only in the JSON file.

Use only this installed CLI: never run npx, install a package, or use MCP resource APIs as a fallback.

When to call — and when not to

• Call inspect_image when your answer depends on visible content and the prompt contains only a local path or image URL, direct inspection failed, or the user explicitly requested the provider. Never infer image content from a file name or URL.
• A bare path or URL without a task that depends on visible content is not a reason to call it.
• Do not call it when the prompt already contains actual image content or a host image viewer returned that content, unless the user explicitly requested the provider.
• Never call it when the user says not to send the image to the provider.
• Do NOT call it when the task merely involves an image file without needing its content: renaming, moving, deleting, uploading, listing, or referencing a file path.
• Before calling, decide the minimum visual facts you are missing and ask exactly those. Never request a general description of the whole image.

How to ask

• Pass the image as { "type": "file", "value": "<path>" } or { "type": "url", "value": "<http(s) url>" }. One concrete image per call; no directories or globs.
• Give each question a short id (q1, q2, …) and a narrow, factual text: "What error code is shown in the dialog?", "What are the card's background color, border radius, and padding?" — not "Describe this screenshot".
• For design mockups and UI screenshots, ask for quantitative values explicitly: hex colors, pixel sizes/spacing, font weight. Treat returned colors/dimensions as visual estimates — close enough to implement from, not pixel-exact; verify against design tokens or a color picker when exactness matters.
• Batch related questions about the same image into one call instead of calling repeatedly.