Caveman Gateway Setup

You are wiring this repository through the Caveman gateway. Caveman is a byte-preserving LLM proxy: in record mode it measures what your app sends and what it costs, and changes nothing else. Your job is a minimal, verified integration — not a refactor.

The prompt that sent you here provides four values. Refer to them as:

• GATEWAY — the gateway base URL (e.g. https://gateway.caveman.so or http://127.0.0.1:8787)
• CAVE_API_KEY — the gateway auth secret (treat like any API key: env var only, never committed, never printed in full)
• PROVIDER_KEYS — stored (provider keys live encrypted in Caveman Cloud) or byok (this app sends its own provider key per request)
• DASHBOARD — the dashboard base URL (e.g. https://app.caveman.so)

If any value is missing, stop and ask for it. Do not guess a URL or mint a key.

Rules (non-negotiable)

1. Coherent integration. Wire every live LLM callsite through existing configuration and responsible seams. Touch each layer correctness requires. No drive-by refactors or formatting sweeps; add an abstraction only when it clarifies ownership or lowers lifecycle cost.
2. Secrets stay in env vars. CAVE_API_KEY goes into the env file the repo already uses (.env, .env.local, …). If that file isn't gitignored, add it to .gitignore and say so. Never hardcode the key in source.
3. Report only what you observed. The final report states the HTTP status and usage numbers from the real verification response — never assumed success. If verification fails, report the failure template instead.
4. Record mode only. You are adding measurement. You do not enable any optimization, and you do not claim any savings — verified savings are $0 until an optimizer is explicitly turned on and passes its eval gate.
5. Provider keys are not your business. With PROVIDER_KEYS: stored you never see one. With byok, the app's existing provider key stays exactly where it already is.

Step 1 — Find every live LLM callsite

Read dependency files (package.json, requirements.txt, pyproject.toml, go.mod, lockfiles) and search the source for LLM clients:

• SDK imports: openai, @anthropic-ai/sdk, anthropic, ai + @ai-sdk/* (Vercel), langchain*, litellm, google-genai / @google/genai, crewai, pydantic_ai, openai-agents / agents
• Raw HTTP to api.openai.com, api.anthropic.com, generativelanguage.googleapis.com
• Existing base-URL env vars: OPENAI_BASE_URL, OPENAI_API_BASE, ANTHROPIC_BASE_URL, GEMINI_BASE_URL, GOOGLE_GEMINI_BASE_URL

List what you found (file:line per callsite) before changing anything. If you find no LLM callsites, stop and report the "nothing to wire" template at the end of this file — do not invent an integration.

Step 2 — Pick the app slug

One slug names this app in the gateway path: GATEWAY/w/<app>. Derive it from the package/module name (e.g. support-bot, acme-api). Grammar: lowercase [a-z0-9] first, then [a-z0-9._-], max 64 chars. Spend for this whole app groups under that slug on the dashboard.

Step 3 — Wire each callsite

The pattern is always the same: base URL → the gateway with /w/<app>, plus one auth header. Gateway auth is x-cave-api-key: CAVE_API_KEY (Authorization: Bearer CAVE_API_KEY also works where a header is awkward). With PROVIDER_KEYS: byok, also send x-cave-upstream-key: <the provider key the app already uses>.

Two facts that make the wiring safe (both are gateway-enforced, not hopes): the gateway rebuilds upstream auth headers from scratch, so a client's Authorization/x-api-key value is never forwarded to the provider; and with stored, upstream auth comes from the encrypted connection server-side. So in stored mode, where an SDK insists on an api-key parameter, set it to the Cave key — it authenticates the gateway and goes no further.

Exact shapes (use the one matching each callsite — these are the product's published recipes, not suggestions):

OpenAI SDK (TS) — Chat Completions and Responses both route through:

const client = new OpenAI({
  baseURL: `${process.env.CAVE_GATEWAY_URL}/w/<app>/openai/v1`,
  apiKey: process.env.OPENAI_API_KEY,           // byok: unchanged · stored: use CAVE_API_KEY
  defaultHeaders: {
    "x-cave-api-key": process.env.CAVE_API_KEY!,
    // byok only:
    "x-cave-upstream-key": process.env.OPENAI_API_KEY!,
  },
});

OpenAI SDK (Python) — same shape: base_url=f"{gw}/w/<app>/openai/v1", default_headers={"x-cave-api-key": ..., "x-cave-upstream-key": ...}.

Anthropic SDK (TS/Python) — the SDK appends /v1/messages itself. The x-cave-api-key header is required here in both modes (this SDK's own key param rides x-api-key, which is not a gateway-auth header):

client = anthropic.Anthropic(
    base_url=f"{os.environ['CAVE_GATEWAY_URL']}/w/<app>",
    api_key=os.environ["ANTHROPIC_API_KEY"],      # byok: unchanged · stored: use CAVE_API_KEY
    default_headers={
        "x-cave-api-key": os.environ["CAVE_API_KEY"],
        # byok only:
        "x-cave-upstream-key": os.environ["ANTHROPIC_API_KEY"],
    },
)

Vercel AI SDK — createOpenAICompatible({ baseURL: ${gw}/w/<app>/openai/v1, headers: { "x-cave-api-key": ... } }); Anthropic models via createAnthropic({ baseURL: ${gw}/w/<app>/v1, headers: { ... } }).

LangChain / LangGraph — ChatOpenAI(base_url=f"{gw}/w/<app>/openai/v1", default_headers={...}); ChatAnthropic(base_url=f"{gw}/w/<app>", default_headers={...}). LangGraph inherits whatever model you pass it.

LiteLLM — per call api_base=f"{gw}/w/<app>/openai/v1" + extra_headers={...}, or fleet-wide in the LiteLLM proxy config.yaml.

Raw HTTP / anything else — swap the host, keep the provider's native path: GATEWAY/w/<app>/v1/chat/completions (OpenAI protocol) or GATEWAY/w/<app>/v1/messages (Anthropic protocol), add the header(s).

Concretely, with slug support-bot and the hosted gateway, an OpenAI-SDK base URL reads https://gateway.caveman.so/w/support-bot/openai/v1. And in stored mode, drop every x-cave-upstream-key line entirely — it is byok-only.

For frameworks not listed (google-genai, crewai, pydantic-ai, openai-agents), fetch the matching page under <docs origin>/docs/integrations/ — same origin this skill came from — and follow it.

Add to the repo's env file (and reference from code — no literals):

CAVE_GATEWAY_URL=<GATEWAY>
CAVE_API_KEY=<CAVE_API_KEY>