Adding A Provider API Feature

Adding a provider API feature

Use this when exposing a new provider API capability through Pydantic AI — prompt caching, strict/structured tool calling, thinking/reasoning effort, service tier, safety settings, logprobs, cache breakpoints, and the like. The output is a change that is consistent with how sibling providers already expose the same concept, defaults deliberately, and gates support with a capability flag.

Not for: adding a new model id (that's add-new-model), a bug fix, or a refactor.

The one rule that prevents the most rework

Before designing anything, find the existing cross-provider abstraction that governs this capability and let its shape decide the API. Most "how should I expose this?" questions are already answered by an abstraction the codebase has — reaching for a new provider-specific knob when one exists is the single most common thing maintainers reject. When a feature routes through an existing abstraction, the abstraction's shape pre-decides the API surface, the opt-out, and often the default.

The tell that you skipped this: you find yourself listing 2-3 "options" for how a user controls the feature. If one of those options duplicates an existing cross-provider control, it isn't a real option — the existing abstraction wins.

Step 0 — Enumerate sibling precedent

For the capability you're adding, list how every provider that already has an analog exposes it, and name the governing existing abstraction. It is one of:

• a per-tool flag — ToolDefinition.strict: bool | None (tools.py), resolved in models/__init__.py::_customize_tool_def;
• a shared ModelSettings field — thinking, service_tier (settings.py), each with per-model resolvers mapping to native concepts;
• a provider-prefixed {Provider}ModelSettings field — anthropic_cache, openai_prompt_cache_key, groq_reasoning_effort;
• a message-stream marker — CachePoint in UserPromptPart.content (messages.py);
• a ModelProfile capability flag — openai_supports_strict_tool_definition, bedrock_supports_prompt_caching.

Read the actual sibling implementations and any review threads on the PRs that added them (gh pr view <n> --comments). Contributors who skipped this got redirected: raw google_tool_config → use strict (#5366); string-prefix model detection → use a profile flag (#4604). Only open a design fork if no existing abstraction covers the capability.

Step 1 — Pick the API shape

1. An existing cross-provider abstraction covers it → reuse it. Add a provider mapping (a _translate_* resolver, a JsonSchemaTransformer subclass, a CachePoint translation). Do not add a provider-prefixed knob for something the shared abstraction already expresses.
2. No shared abstraction, but ≥3 providers now have the concept → promote to a shared ModelSettings field with a deliberately narrow common vocabulary and per-model resolvers, keeping per-provider fields underneath as precedence-winning escape hatches (the service_tier promotion, #4926). Don't delete provider knobs; deprecate only genuinely-misnamed ones with a TODO(v3).
3. The provider's native values can't be expressed by the shared enum (family-disjoint value sets, platform-only request shaping) → a {provider}_* knob is justified, and it coexists with and outranks the unified field (groq_reasoning_effort, #5797).
4. Choose the locus by "can the user naturally point at it?": a boundary inside the message stream → a marker (CachePoint); a structural region (system prompt, tool defs, whole-request setting) → a setting (#3363).
5. Wire it into every request path the provider has (chat and responses APIs); put a setting shared across a provider's APIs on the base settings class (#3678).
6. Type it. Reuse the provider SDK's own types where they exist; type knobs as Literal, never extra_body or untyped **kwargs (models/AGENTS.md; "kwargs are a big no no… I'd rather be repetitive but type safe" — DouweM, #3457).

Step 2 — Default: default-on (silent) vs opt-in

Default the feature on only when enabling it cannot change observable behavior and cannot cost the user — a pure, backward-compatible improvement (caching only lowers cost; a validation mode that needs no schema rewrite and can't reject a previously-valid request). A backward-compatible, purely-better default is welcome and needs no opt-in.

Keep it opt-in when it:

• changes observable output or wire behavior;
• is a preview feature (provider may change semantics);
• can raise cost — and choose the default value so a shared field never silently upgrades anyone to a pricier tier (service_tier 'auto' vs 'default', #4926);
• can hit provider limits at scale (auto-promoting strict silently broke agents with >20 tools; reverted in #5580);
• applies lossy schema rewrites (OpenAI/Anthropic strict transforms).

When the default isn't obvious, decide it with a live probe, not an opinion (#5897 flipped a default on 0/5 → 5/5 recovery). Beware "automatic" language — verify whether it means a Pydantic AI default or just provider-side management of an opt-in feature.

Step 3 — Capability gating

Detect support with a provider-prefixed ModelProfile flag set in Provider.model_profile() — never inline isinstance/model-name checks (profiles/AGENTS.md). Gate at the layer(s) that actually vary:

• model/family → profile flag (google_supports_strict_tool_definition, bedrock_supports_prompt_caching);
• per-schema → the JsonSchemaTransformer.is_strict_compatible signal (default conservative unless the mode needs no rewrites);
• SDK version → probe the SDK shape and degrade with a UserWarning (#5580, botocore strict param);
• unknowable client-side → defer to the runtime API error.

Unsupported → silently ignore the setting (best-effort so as many requests as possible succeed), documented in the docstring — never hard-error. Conflicting user settings → UserWarning, not UserError. Profiles are layered: developer-keyed base + thin provider overlay resolved per family, with the provider overlay layering last (#5934, #6231).