Validate Integration

Validate Integration Skill

You are an expert auditor for Sim integrations. Your job is to thoroughly validate that an existing integration is correct, complete, and follows all conventions.

Your Task

When the user asks you to validate an integration:

1. Read the service's API documentation (via WebFetch or Context7)
2. Read every tool, the block, and registry entries
3. Cross-reference everything against the API docs and Sim conventions
4. Report all issues found, grouped by severity (critical, warning, suggestion)
5. Fix all issues after reporting them

Step 1: Gather All Files

Read every file for the integration — do not skip any:

apps/sim/tools/{service}/          # All tool files, types.ts, index.ts
apps/sim/blocks/blocks/{service}.ts # Block definition
apps/sim/tools/registry.ts          # Tool registry entries for this service
apps/sim/blocks/registry-maps.ts    # Block + meta registry entry (BLOCK_REGISTRY / BLOCK_META_REGISTRY)
apps/sim/components/icons.tsx        # Icon definition
apps/sim/lib/auth/connectors/providers.ts # Better Auth connector providers (buildConnectorProviders) — should use getCanonicalScopesForProvider()
apps/sim/lib/oauth/oauth.ts         # OAuth provider config — single source of truth for scopes
apps/sim/lib/oauth/utils.ts         # Scope utilities, SCOPE_DESCRIPTIONS for modal UI
packages/deployment-config/src/env-capabilities.ts # OAuth client runtime capability source of truth
apps/sim/lib/core/config/env.ts     # Runtime env schema for capability fields
packages/sim-setup/src/capability-config.ts # Exhaustive CLI input-mode mapping for OAuth fields
packages/deployment-config/src/integrations.json # Generated client-safe integration catalog
packages/deployment-config/src/service-account-providers.generated.ts # Generated provider-ID facts
packages/deployment-config/src/service-account-metadata.ts # Handwritten deployment policy

If the block, its triggers, or connector fields use a selectorKey, also apply the validate-selector skill and read the key's entry in apps/sim/lib/selectors/manifest.ts, its server attachment and provider listing primitive, and the shared context builder. Selector provider logic runs only server-side through selectors.execute.

Step 2: Pull API Documentation

Fetch the official API docs for the service. This is the source of truth for:

• Endpoint URLs, HTTP methods, and auth headers
• Required vs optional parameters
• Parameter types and allowed values
• Response shapes and field names
• Pagination patterns (which param name, which response field)
• Rate limits and error formats

Hard Rule: No Guessed Response Schemas

If the official docs do not clearly show the response JSON shape for an endpoint, you MUST tell the user instead of guessing.

• Do NOT assume field names from nearby endpoints
• Do NOT infer nested JSON paths without evidence
• Do NOT treat "likely" fields as confirmed outputs
• Do NOT accept implementation guesses as valid just because they are defensive

If a response schema is unknown, the validation must explicitly call that out and require:

1. sample responses from the user,
2. live test credentials for verification, or
3. trimming the tool/block down to only documented fields.

Step 3: Validate Tools

For every tool file, check:

Tool ID and Naming

• Tool ID uses snake_case: {service}_{action} (e.g., x_create_tweet, slack_send_message)
• Tool name is human-readable (e.g., 'X Create Tweet')
• Tool description is a concise one-liner describing what it does
• Tool version is set ('1.0.0' or '2.0.0' for V2)

Params

• All required API params are marked required: true
• All optional API params are marked required: false
• Every param has explicit required: true or required: false — never omitted
• Param types match the API ('string', 'number', 'boolean', 'json')
• Visibility is correct:
  • 'hidden' — ONLY for OAuth access tokens and system-injected params
  • 'user-only' — for API keys, credentials, and account-specific IDs the user must provide
  • 'user-or-llm' — for everything else (search queries, content, filters, IDs that could come from other blocks)
• Every param has a description that explains what it does

Request

• URL matches the API endpoint exactly (correct base URL, path segments, path params)
• HTTP method matches the API spec (GET, POST, PUT, PATCH, DELETE)
• Headers include correct auth pattern:
  • OAuth: Authorization: Bearer ${params.accessToken}
  • API Key: correct header name and format per the service's docs
• Content-Type header is set for POST/PUT/PATCH requests
• Body sends all required fields and only includes optional fields when provided
• For GET requests with query params: URL is constructed correctly with query string
• ID fields in URL paths are .trim()-ed to prevent copy-paste whitespace errors
• Path params use template literals correctly: `https://api.service.com/v1/${params.id.trim()}`

Response / transformResponse

• Correctly parses the API response (await response.json())
• Extracts the right fields from the response structure (e.g., data.data vs data vs data.results)
• All nullable fields use ?? null
• All optional arrays use ?? []
• Error cases are handled: checks for missing/empty data and returns meaningful error
• Does NOT do raw JSON dumps — extracts meaningful, individual fields
• Every extracted field is backed by official docs or live-verified sample payloads

Outputs

• All output fields match what the API actually returns
• No fields are missing that the API provides and users would commonly need
• No phantom fields defined that the API doesn't return
• optional: true is set on fields that may not exist in all responses
• When using type: 'json' and the shape is known, properties defines the inner fields (tool outputs only — block outputs do not support properties)
• When using type: 'array', items defines the item structure with properties (tool outputs only)
• Field descriptions are accurate and helpful

Types (types.ts)

• Has param interfaces for every tool (e.g., XCreateTweetParams)
• Has response interfaces for every tool (extending ToolResponse)
• Optional params use ? in the interface (e.g., replyTo?: string)
• Field names in types match actual API field names
• Shared response types are properly reused (e.g., XTweetResponse shared across tweet tools)

Barrel Export (index.ts)

• Every tool is exported
• All types are re-exported (export * from './types')
• No orphaned exports (tools that don't exist)

Tool Registry (tools/registry.ts)

• Every tool is imported and registered
• Registry keys use snake_case and match tool IDs exactly
• Entries are in alphabetical order within the file

Resolved-Secret Provenance and Model Input

For every request field, determine whether it is ordinary API input, model-visible text/structured content, opaque model input, or a value persisted into Sim-owned durable storage.

Treat model-input provenance as opt-in. Require official documentation or an unambiguous local execution path proving that the exact field reaches an AI model. If the evidence is ambiguous, leave the integration unchanged; do not infer a model boundary merely from natural-language, search, extraction, or "AI-powered" marketing terminology.

• AI-consumed text/structured fields use request.modelInput with mode: 'project' and a minimal exact selector; nested/JSON-string adapters preserve shape through applyProjected
• Ordinary external URLs, domains, resource IDs, and control fields retain normal request semantics unless the exact field is proven model-visible; an AI-backed provider or later model processing of the referenced resource is not sufficient evidence
• Serialized content proven to be sent directly to an external model is selected by request.modelInput, projected before the existing formatter parses it, and has deterministic formatter behavior when a whole-value placeholder is invalid for the serialized grammar
• Actual inline/raw AI-consumed bytes owned by a registered in-process operation use operation.modelInput with privateInputPaths (or mode: 'private-provenance'), and the operation calls validateOpaqueModelInputProvenance before model egress; storage keys, paths, signed URLs, and ordinary remote URLs are not treated as byte provenance, while tracked stored bytes are authorized independently at the owning model-egress boundary
• Persisted workspace-file contents are checked with the shared provenance guard only when their bytes or decoded content cross into a model/tool-result boundary; ordinary file APIs remain unchanged. Unsupported secret-bearing file paths are rejected at file_write
• Sim-owned durable writes and internal execution handoffs that can enter workflows/models use field-scoped operation.secretProvenance; the owning operation validates the exact selection and scope, strip private metadata, and persist, import, or propagate it at the owning boundary
• Private provenance is never attached to external URLs; registered in-process operations preserve it through operation.modelInput / operation.secretProvenance, while proven model-visible external fields use request projection and other external inputs remain unchanged
• No tool performs raw secret plaintext/source substitution or serializes plaintext provenance
• No transformResponse or tool-local helper blanket-sanitizes ordinary third-party results; only execution-scoped, activated Sim provenance is projected at shared model/log boundaries
• Private headers/envelopes are produced and stripped by the shared tool executor, never hand-rolled or returned as functional output
• Every added provenance hook has a concrete Sim {{...}} resolution path and a later persistence/model/log crossing; there is no generic handling for arbitrary filenames, metadata, provider results, or API payloads
• Diagnostic projection is applied only to values carrying execution-scoped provenance; ordinary provider responses, filenames, URLs, and errors are unchanged
• Tests cover named {{NAME}} projection, unproven identical public text, nested and serialized shape handling, unchanged ordinary external inputs, malformed/incomplete metadata, headerless legacy requests, metadata stripping, and durable legacy/stale/scope cases when applicable

Treat a missing or bypassed model, durable, or internal-execution provenance boundary as critical. Do not fix it with a tool-specific string replacer or by sanitizing every provider result; repair the shared request, in-process operation, persistence, or re-entry boundary that owns the data.

Step 4: Validate Block

Block ↔ Tool Alignment (CRITICAL)

This is the most important validation — the block must be perfectly aligned with every tool it references.

For each tool in tools.access:

• The operation dropdown has an option whose ID matches the tool ID (or the tools.config.tool function correctly maps to it)
• Every required tool param (except accessToken) has a corresponding subBlock input that is:
  • Shown when that operation is selected (correct condition)
  • Marked as required: true (or conditionally required)
• Every optional tool param has a corresponding subBlock input (or is intentionally omitted if truly never needed)
• Every subBlock id is unique (duplicates collide silently; the last definition wins). blocks.test.ts fails a duplicate within one condition unless the copies are a basic/advanced mode-swap pair, one basic plus trigger-mode copies, or all carry canonicalParamId. The only sanctioned cross-condition reuse is the hosted-key apiKey pair (add-hosted-key skill)
• The tools.config.tool function returns the correct tool ID for every possible operation value
• Each subBlock (or its canonicalParamId) is named exactly after the tool param it fills. A required user-only param that is only renamed in tools.config.params fails bun run apps/sim/scripts/check-block-registry.ts origin/staging; remap only optional or user-or-llm params

SubBlocks

• Operation dropdown lists ALL tool operations available in tools.access
• Dropdown option labels are human-readable and descriptive
• Conditions use correct syntax:
  • Single value: { field: 'operation', value: 'x_create_tweet' }
  • Multiple values (OR): { field: 'operation', value: ['x_create_tweet', 'x_delete_tweet'] }
  • Negation: { field: 'operation', value: 'delete', not: true }
  • Compound: { field: 'op', value: 'send', and: { field: 'type', value: 'dm' } }
• Condition arrays include ALL operations that use that field — none missing
• dependsOn is set for fields that need other values (selectors depending on credential, cascading dropdowns)
• SubBlock types match tool param types:
  • Enum/fixed options → dropdown
  • Free text → short-input
  • Long text/content → long-input
  • True/false → switch (a Yes/No dropdown only when the tool needs a third "unset" state)
  • Credentials → oauth-input with correct serviceId
• Dropdown value: () => 'default' is set for dropdowns with a sensible default
• Every short-input, long-input, code, and selector subBlock has a placeholder — including password fields (Enter your API key). Formatted values show the shape (2023-01-01T00:00:00Z); optional fields with a server default name it. See add-integration → Step 3

Advanced Mode

• Optional, rarely-used fields are set to mode: 'advanced':
  • Pagination tokens / next tokens
  • Time range filters (start/end time)
  • Sort order / direction options
  • Max results / per page limits
  • Reply settings / threading options
  • Rarely used IDs (reply-to, quote-tweet, etc.)
  • Exclude filters
• Required fields are NEVER set to mode: 'advanced'
• Fields that users fill in most of the time are NOT set to mode: 'advanced'

WandConfig

• Timestamp fields have wandConfig with generationType: 'timestamp'
• Comma-separated list fields have wandConfig with a descriptive prompt
• Complex filter/query fields have wandConfig with format examples in the prompt
• All wandConfig prompts end with an explicit Return ONLY the <format> instruction so the generated value can be pasted directly into the field
• wandConfig.placeholder describes what to type in natural language

Tools Config

• tools.access lists every tool ID the block can use — none missing
• tools.config.tool returns the correct tool ID for each operation
• Type coercions are in tools.config.params (runs at execution time), NOT in tools.config.tool (runs at serialization time before variable resolution — coercing there destroys dynamic references like <Block.output>)
• tools.config.params handles:
  • Number() conversion for numeric params that come as strings from inputs
  • Boolean / string-to-boolean conversion for toggle params
  • Empty string → undefined conversion for optional dropdown values
  • SubBlock ID → tool param name remapping only for optional or user-or-llm params

Block Outputs

• Outputs cover the key fields returned by ALL tools (not just one operation)
• Output types are correct ('string', 'number', 'boolean', 'json', 'file', 'file[]')
• type: 'json' outputs describe inner fields in the description string: 'User profile (id, name, username, bio)' or '[{address, status, type}]' for arrays
• Do NOT add a properties: {...} field on block outputs. Block-level OutputFieldDefinition (from @sim/workflow-types/blocks) only accepts { type, description?, condition?, hiddenFromDisplay? }. Nested properties is a tool-level construct (OutputProperty) — adding it to a block output will fail TypeScript at build time
• No opaque type: 'json' with vague descriptions like 'Response data'
• Outputs that only appear for certain operations use condition if supported, or document which operations return them

Block Metadata

• type is snake_case (e.g., 'x', 'cloudflare')
• name is human-readable (e.g., 'X', 'Cloudflare')
• description is a concise one-liner
• longDescription provides detail for docs
• docsLink points to 'https://docs.sim.ai/integrations/{service}'
• category is 'tools'
• bgColor uses the service's brand color hex
• icon references the correct icon component from @/components/icons
• authMode is set correctly (AuthMode.OAuth or AuthMode.ApiKey)
• Block + meta are registered in blocks/registry-maps.ts (BLOCK_REGISTRY / BLOCK_META_REGISTRY) alphabetically

BlockMeta

• {Service}BlockMeta is exported in the same file as the block
• Has at least 7 templates, each with icon, title, prompt, modules, category, and tags
• Prompts describe concrete use cases, not generic descriptions of what the service does
• alsoIntegrations is set on any template whose prompt references another service
• skills present (3–5 mainstream, 2–3 niche), each grounded in tools.access — flag any skill implying an unsupported action
• Each skill is real, not hallucinated — web-search and confirm it maps to a popular use case attested online (vendor use-case pages, official docs describing the workflow, reputable "top automations" articles); rewrite/remove any you cannot source
• Each skill has a kebab-case name (≤64 chars, unique), a one-line description, and markdown content with # Title + ## Steps + an output/guidance section

Block Inputs

• inputs section lists all subBlock params that the block accepts
• Input types match the subBlock types
• When using canonicalParamId, inputs list the canonical ID (not the raw subBlock IDs)

Dynamic Selectors

• Every remote selectorKey is classified in the browser-safe manifest and has exactly one server attachment
• The manifest allowlists the minimal active dependsOn context and matches list/search/detail, pagination, scope, and stale-time behavior
• Canonical basic/advanced and trigger/action modes project only their active values; exact {{KEY}} references remain unresolved in the browser
• Stored credentials are bound to the actor, workspace, and trusted provider/service
• Each attachment declares and enforces a fixed, credential-bound, or explicitly reviewed user-controlled destination policy
• Provider results are explicitly projected to safe option fields; secrets, tokens, credential IDs, context values, and raw upstream errors do not enter responses, logs, or query keys
• No selector provider module, provider fetch, or OAuth-token request runs in the browser, and no selector-only provider route remains

Step 5: Validate OAuth Scopes (if OAuth service)

Scopes are centralized — the single source of truth is OAUTH_PROVIDERS in lib/oauth/oauth.ts.

• Scopes defined in lib/oauth/oauth.ts under OAUTH_PROVIDERS[provider].services[service].scopes
• lib/auth/connectors/providers.ts (buildConnectorProviders) uses getCanonicalScopesForProvider(providerId) — NOT a hardcoded array
• Block requiredScopes uses getScopesForService(serviceId) — NOT a hardcoded array
• No hardcoded scope arrays in lib/auth/connectors/providers.ts or block files (should all use utility functions)
• Each scope has a human-readable description in SCOPE_DESCRIPTIONS within lib/oauth/utils.ts
• No excess scopes that aren't needed by any tool