Add Hosted Key

Adding Hosted Key Support to a Tool

When a tool has hosted key support, Sim provides its own API key if the user hasn't configured one (via BYOK or env var). Usage is metered and billed to the workspace.

Overview

Step 1: Register the BYOK Provider ID

Add the new provider to the BYOKProviderId union in tools/types.ts:

export type BYOKProviderId =
  | 'openai'
  | 'anthropic'
  // ...existing providers
  | 'your_service'

Then add the same provider id to the byokProviderIdSchema enum in lib/api/contracts/byok-keys.ts (this is what the byok-keys route validates against):

export const byokProviderIdSchema = z.enum([
  'openai',
  'anthropic',
  // ...existing providers
  'your_service',
])

Step 2: Research the API's Pricing Model and Rate Limits

Before writing any getCost or rateLimit code, look up the service's official documentation for both pricing and rate limits. You need to understand:

Pricing

1. How the API charges — per request, per credit, per token, per step, per minute, etc.
2. Whether the API reports cost in its response — look for fields like creditsUsed, costDollars, tokensUsed, or similar in the response body or headers
3. Whether cost varies by endpoint/options — some APIs charge more for certain features (e.g., Firecrawl charges 1 credit/page base but +4 for JSON format, +4 for enhanced mode)
4. The dollar-per-unit rate — what each credit/token/unit costs in dollars on our plan

Rate Limits

1. What rate limits the API enforces — requests per minute/second, tokens per minute, concurrent requests, etc.
2. Whether limits vary by plan tier — free vs paid vs enterprise often have different ceilings
3. Whether limits are per-key or per-account — determines whether adding more hosted keys actually increases total throughput
4. What the API returns when rate limited — HTTP 429, Retry-After header, error body format, etc.
5. Whether there are multiple dimensions — some APIs limit both requests/min AND tokens/min independently

Search the API's docs/pricing page (use WebSearch/WebFetch). Capture the pricing model as a comment in getCost so future maintainers know the source of truth.

Setting Our Rate Limits

Our rate limiter (lib/core/rate-limiter/hosted-key/) uses a token-bucket algorithm applied per billing actor (workspace). It supports two modes:

• per_request — simple; just requestsPerMinute. Good when the API charges flat per-request or cost doesn't vary much.
• custom — requestsPerMinute plus additional dimensions (e.g., tokens, search_units). Each dimension has its own limitPerMinute and an extractUsage function that reads actual usage from the response. Use when the API charges on a variable metric (tokens, credits) and you want to cap that metric too.

When choosing values for requestsPerMinute and any dimension limits:

• Stay well below the API's per-key limit — our keys are shared across all workspaces. If the API allows 60 RPM per key and we have 3 keys, the global ceiling is ~180 RPM. Set the per-workspace limit low enough (e.g., 20-60 RPM) that many workspaces can coexist without collectively hitting the API's ceiling.
• Account for key pooling — our round-robin distributes requests across N hosted keys, so the effective API-side rate per key is (total requests) / N. But per-workspace limits are enforced before key selection, so they apply regardless of key count.
• Prefer conservative defaults — it's easy to raise limits later but hard to claw back after users depend on high throughput.

Step 3: Add hosting Config to the Tool

Add a hosting object to the tool's ToolConfig. This tells the execution layer how to acquire hosted keys, calculate cost, and rate-limit.

hosting: {
  envKeyPrefix: 'YOUR_SERVICE_API_KEY',
  apiKeyParam: 'apiKey',
  byokProviderId: 'your_service',
  pricing: {
    type: 'custom',
    getCost: (_params, output) => {
      if (output.creditsUsed == null) {
        throw new Error('Response missing creditsUsed field')
      }
      const creditsUsed = output.creditsUsed as number
      const cost = creditsUsed * 0.001 // dollars per credit
      return { cost, metadata: { creditsUsed } }
    },
  },
  rateLimit: {
    mode: 'per_request',
    requestsPerMinute: 100,
  },
},

Hosted Key Env Var Convention

Keys use a numbered naming pattern driven by a count env var:

YOUR_SERVICE_API_KEY_COUNT=3
YOUR_SERVICE_API_KEY_1=sk-...
YOUR_SERVICE_API_KEY_2=sk-...
YOUR_SERVICE_API_KEY_3=sk-...

The envKeyPrefix value (YOUR_SERVICE_API_KEY) determines which env vars are read at runtime. Adding more keys only requires bumping the count and adding the new env var.

Pricing: Prefer API-Reported Cost

Always prefer using cost data returned by the API (e.g., creditsUsed, costDollars). This is the most accurate because it accounts for variable pricing tiers, feature modifiers, and plan-level discounts.

When the API reports cost — use it directly and throw if missing:

pricing: {
  type: 'custom',
  getCost: (params, output) => {
    if (output.creditsUsed == null) {
      throw new Error('Response missing creditsUsed field')
    }
    // $0.001 per credit — from https://example.com/pricing
    const cost = (output.creditsUsed as number) * 0.001
    return { cost, metadata: { creditsUsed: output.creditsUsed } }
  },
},

When the API does NOT report cost — compute it from params/output based on the pricing docs, but still validate the data you depend on:

pricing: {
  type: 'custom',
  getCost: (params, output) => {
    if (!Array.isArray(output.searchResults)) {
      throw new Error('Response missing searchResults, cannot determine cost')
    }
    // Serper: 1 credit for <=10 results, 2 credits for >10 — from https://serper.dev/pricing
    const credits = Number(params.num) > 10 ? 2 : 1
    return { cost: credits * 0.001, metadata: { credits } }
  },
},

getCost must always throw if it cannot determine cost. Never silently fall back to a default — this would hide billing inaccuracies.

Capturing Cost Data from the API

If the API returns cost info, capture it in transformResponse so getCost can read it from the output:

transformResponse: async (response: Response) => {
  const data = await response.json()
  return {
    success: true,
    output: {
      results: data.results,
      creditsUsed: data.creditsUsed,  // pass through for getCost
    },
  }
},

For async/polling tools, capture it in postProcess when the job completes:

if (jobData.status === 'completed') {
  result.output = {
    data: jobData.data,
    creditsUsed: jobData.creditsUsed,
  }
}

Step 4: Hide the API Key Field When Hosted

In the block config (blocks/blocks/{service}.ts), add hideWhenHosted: true to the API key subblock. This hides the field on hosted Sim since the platform provides the key:

{
  id: 'apiKey',
  title: 'API Key',
  type: 'short-input',
  placeholder: 'Enter your API key',
  password: true,
  required: true,
  hideWhenHosted: true,
},

The visibility is controlled by isSubBlockHidden() in lib/workflows/subblocks/visibility.ts, which checks both getDeploymentShape().hosted (hideWhenHosted) and optional env var conditions (hideWhenEnvSet).