Venice Chat

Prompt caching (cache_control)

Any content part can carry {"cache_control": {"type": "ephemeral"}} or {"type": "ephemeral", "ttl": "1h"}. Explicit markers matter for models that require them (Claude); other models cache automatically on prefix matches. Those models allow at most 4 breakpoints per request; Venice adds its own to the system prompt and conversation history only while your markers leave slots free. Pair with a stable prompt_cache_key for consistent routing. Cache read / write prices are per model (model_spec.pricing.cache_input / cache_write); hits are reported in usage.prompt_tokens_details.

Tools & function calling

{
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get current weather for a city",
      "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
      "strict": true
    }
  }],
  "tool_choice": "auto"
}

• tool_choice: "auto", "required", "none", or {"type":"function","function":{"name":"get_weather"}}. {"type":"auto" | "none" | "required"} is normalized to the string form.
• parallel_tool_calls defaults to true — be ready to run several calls before replying.
• Reply with one {"role":"tool","tool_call_id":"...","content":"..."} message per call, then call again.
• Flat ({type, name, parameters}) and Anthropic ({name, input_schema}) tool definitions are converted to the nested format.
• When the message carries tool calls, Venice reports finish_reason: "tool_calls" rather than stop (a length cut-off stays length).
• The schema also accepts {"type":"web_search"} / {"type":"x_search"} tool entries, but they do not turn on Venice search or xAI X search. Use venice_parameters.enable_web_search / enable_x_search instead.

Reasoning models

{
  "model": "zai-org-glm-5-1",
  "reasoning": {"effort": "medium"},
  "messages": [{"role": "user", "content": "Prove there are infinitely many primes."}]
}

• Reasoning arrives in message.reasoning_content (delta.reasoning_content when streaming); <think> tags are removed from content. Some providers return encrypted or summarized reasoning.
• Supported effort values are per model: check model_spec.capabilities.supportsReasoningEffort, reasoningEffortOptions, and defaultReasoningEffort. On most Claude models and on OpenAI GPT models, a value outside reasoningEffortOptions → 400 (none is always accepted as a Venice-level off switch; on OpenAI models minimal is also accepted and maps to the lowest supported level). Other models are not pre-validated, so stick to reasoningEffortOptions.
• To turn thinking off, prefer reasoning: {"enabled": false} or venice_parameters.disable_thinking: true — both degrade gracefully on mandatory-reasoning models. reasoning_effort: "none" also disables thinking where the model allows it, but some mandatory-reasoning models reject it; prefer the two switches above.
• Some models return reasoning_details[] (and Gemini via native transport returns thought_signature) on the assistant message. Pass them back verbatim on the next turn, especially in tool loops, to preserve thought signatures.
• Reasoning tokens count toward max_completion_tokens and are billed as output.

Structured output (response_format)

{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "person",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {"name": {"type": "string"}, "age": {"type": "number"}},
        "required": ["name", "age"],
        "additionalProperties": false
      }
    }
  }
}

Put the JSON Schema under json_schema.schema (OpenAI shape) — Claude, Gemini, and Grok models only read it from there. json_object is deprecated and ignored by Claude models. Requires supportsResponseSchema.

E2EE (end-to-end encryption)

For models with supportsE2EE: true (the e2ee-* IDs), following the TEE & E2EE guide:

1. GET /api/v1/tee/attestation?model=<id>&nonce=<64 hex chars> (no auth needed, 10 req/min/IP). Verify it and take the model's public key.
2. Generate a per-session secp256k1 key pair. Encrypt every user and system message with ECDH → HKDF-SHA256 → AES-256-GCM.
3. Send the request with X-Venice-TEE-Client-Pub-Key and X-Venice-TEE-Model-Pub-Key (secp256k1 hex keys), X-Venice-TEE-Signing-Algo: ecdsa, and stream: true (the guide requires streaming). Malformed headers → 400 with {"error":{"message":"Invalid E2EE headers: …","type":"invalid_request_error"}}.
4. Decrypt the streamed content with your private key.

On E2EE requests Venice injects nothing: no Venice system prompt, character, web search, or scraping. file parts → 400. The guide also lists function calling as unsupported. Without E2EE headers (or with enable_e2ee: false) the same model runs in TEE-only mode. TEE responses carry X-Venice-TEE: true and X-Venice-TEE-Provider. E2EE is not available on /responses.

Streaming

{"stream": true}

• text/event-stream, one data: {chat.completion.chunk} per event, terminated by data: [DONE].
• Venice requests usage from the model and emits it in a choices: [] chunk with usage (and cost) before [DONE], whatever stream_options.include_usage says. Usage is not repeated on content chunks.
• If the upstream fails after headers are sent, you get an in-band data: {"error": {...}} chunk (e.g. code: "model_overloaded" with retry_after, upstream_error, or content_blocked_by_provider) followed by [DONE].
• Web-search citations are not in the stream unless include_search_results_in_stream: true, in which case a choices: [] chunk with venice_parameters.web_search_citations arrives at the end of the stream, before [DONE].
• With return_search_results_as_documents: true, the synthetic web_search_call tool call (see Web search) is streamed as a delta.tool_calls chunk before the content.

Web search

• Non-streaming responses include venice_parameters.web_search_citations[] with url, title, content (snippet), and date. Add enable_web_citations: true to get ^n^ markers in the text.
• Search is skipped on E2EE requests, when the last user message contains an image, and when scraping found URLs in that message.
• return_search_results_as_documents: true adds a synthetic tool call {id:"web_search_call", type:"function", function:{name:"web_search", arguments:"{\"documents\":[{id,title,url,snippet,published_at}]}"}} and sets finish_reason: "tool_calls". Don't try to execute it.
• Billing: $0.01 per search-augmented request, $0.01 per scraped URL, $0.01 per xAI X search.

Error handling specifics

See venice-errors for shapes and retry strategy.

Common gotchas

• max_tokens is deprecated — use max_completion_tokens, and stay under model_spec.maxCompletionTokens.
• Unknown top-level fields are rejected (400); unknown venice_parameters keys are silently dropped.
• Image and video URLs must be publicly reachable without redirects. Signed S3 URLs that redirect or localhost URLs fail.
• Audio cannot be a URL — always base64.
• Single-image vision models drop older images each turn; put images in the last user message.
• Round-trip reasoning_details / thought_signature unchanged in multi-turn tool loops.
• character_slug adds the character's system prompt ahead of yours. Venice's own prompt is still included unless the character uses a custom system prompt or you set include_venice_system_prompt: false.
• web_search / x_search tool entries are not Venice search — use venice_parameters.
• OpenAI reasoning models reject seed, stop, n > 1, and non-zero penalties.