Langchain Model Inference

What it does

When your agent uses it

• Initializing chat models
• Routing across providers
• Iterating AIMessage content
• Choosing a structured-output method

Example prompts

• “langchain model inference”
• “ChatAnthropic”
• “ChatOpenAI”
• “/langchain-model-inference”

Requirements

• Python 3
• A credential in ANTHROPIC_API_KEY
• A credential in OPENAI_API_KEY
• Compatibility (from SKILL.md): Designed for Claude Code
• Pre-approved tools (allowed-tools): Read, Write, Edit, Bash(python:*), Bash(pip:*), Grep

Workflow steps

5 steps, taken from the step headings in SKILL.md.

1. 1Initialize a chat model with explicit, version-safe defaults
2. 2Iterate AIMessage.content as typed blocks, not strings
3. 3Route across providers with a factory, not a conditional
4. 4Count tokens correctly during streaming
5. 5Pick the right with_structured_output method

What it can do on your machine

Read from SKILL.md and the folder at commit cfae287. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves these tools, so the agent can use them without asking each time:

  • Read
  • Write
  • Edit
  • Bash(python:*)
  • Bash(pip:*)
  • Grep

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Shell commands in SKILL.md call:

  • pip

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  Links to these hosts (documentation or services it may open):

  • python.langchain.com
  • blog.langchain.com

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names these keys or tokens, usually read from environment variables:

  • ANTHROPIC_API_KEY
  • OPENAI_API_KEY
  • GOOGLE_API_KEY

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

• Compatibility

  Designed for Claude Code

  From compatibility in the SKILL.md frontmatter.

Context cost

Langchain Model Inference loads about 2.7k tokens when it runs, and up to ~7.4k if it reads all its reference files. Until then it costs about 121 tokens; SKILL.md has 784 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~121

When it runs · the whole SKILL.md, loaded when a task matches

~2.7k

With references · SKILL.md plus every file in references/, read only if the agent opens them

~7.4k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from jeremylongshore/tons-of-skills-marketplace at commit cfae287, republished under its MIT licence (© jeremylongshore). 784 words, ~2,683 tokens.

name

langchain-model-inference

description

Invoke Claude, GPT-4o, and Gemini through LangChain 1.0 without tripping on the content-block, token-accounting, and structured-output quirks that silently break production code. Use when initializing chat models, routing across providers, iterating AIMessage content, or choosing a structured-output method. Trigger with "langchain model inference", "ChatAnthropic", "ChatOpenAI", "with_structured_output", "AIMessage content blocks", "langchain routing".

allowed-tools

Read, Write, Edit, Bash(python:*), Bash(pip:*), Grep

compatibility

Designed for Claude Code

version

2.7.0

license

MIT

author

Jeremy Longshore <jeremy@intentsolutions.io>

tags

saas, langchain, langgraph, python, langchain-1.0, model-inference

from langchain_anthropic import ChatAnthropic
from langchain_openai import ChatOpenAI

claude = ChatAnthropic(
    model="claude-sonnet-4-6",
    temperature=0,
    max_tokens=4096,
    timeout=30,       # seconds. Default is None — hangs forever on provider stall.
    max_retries=2,    # Retries, not attempts. See P30 in pain catalog.
)

gpt4o = ChatOpenAI(
    model="gpt-4o",
    temperature=0,
    timeout=30,
    max_retries=2,
)

from langchain_core.messages import AIMessage

def extract_text(msg: AIMessage) -> str:
    """Safe on both provider shapes. Works for streaming deltas too.

    Handles both dict blocks (provider-native) and typed block objects
    (LangChain 1.0 wrappers) — which Gemini, OpenAI tools, and future
    SDK versions may return.
    """
    if isinstance(msg.content, str):
        return msg.content
    parts = []
    for block in msg.content:
        # Block may be a dict (provider-native) or a typed object (1.0 wrapper)
        block_type = block.get("type") if isinstance(block, dict) else getattr(block, "type", None)
        if block_type == "text":
            parts.append(block["text"] if isinstance(block, dict) else block.text)
    return "".join(parts)

from langchain_core.language_models import BaseChatModel

# Version-safe defaults applied to every model the factory builds.
# Callers can override via **kwargs.
_SAFE_DEFAULTS = {"timeout": 30, "max_retries": 2}

def chat_model(provider: str, **kwargs) -> BaseChatModel:
    defaults = {**_SAFE_DEFAULTS, **kwargs}  # caller's kwargs win
    if provider == "anthropic":
        return ChatAnthropic(model="claude-sonnet-4-6", **defaults)
    if provider == "openai":
        return ChatOpenAI(model="gpt-4o", **defaults)
    if provider == "gemini":
        from langchain_google_genai import ChatGoogleGenerativeAI
        return ChatGoogleGenerativeAI(model="gemini-2.5-pro", **defaults)
    raise ValueError(f"Unknown provider: {provider!r}")

async for event in claude.astream_events({"input": "..."}, version="v2"):
    if event["event"] == "on_chat_model_stream":
        chunk = event["data"]["chunk"]
        if hasattr(chunk, "usage_metadata") and chunk.usage_metadata:
            meter.record(chunk.usage_metadata["input_tokens"],
                         chunk.usage_metadata["output_tokens"])

from pydantic import BaseModel, ConfigDict

class Plan(BaseModel):
    model_config = ConfigDict(extra="ignore")  # P53 — models add helpful extra fields
    steps: list[str]
    estimated_minutes: int

structured = claude.with_structured_output(Plan, method="json_schema")
plan = structured.invoke("Plan a 3-step deploy")

© jeremylongshore, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 5 other files (references) in skills/.curated/langchain-model-inference of jeremylongshore/tons-of-skills-marketplace.

• SKILL.md
• references/content-blocks.md
• references/one-pager.md
• references/provider-quirks.md
• references/structured-output-methods.md
• references/token-accounting.md

Open the folder on GitHubat commit cfae287