Agentsop Prompt History Inspect

Prompt-History Inspect — First Move Before Anything Else

"The prompt you wrote is not the prompt the model received." — Operating axiom for every framework that templates, injects few-shots, appends tool definitions, or wraps system messages.

1. 何时激活 (When to activate)

Activate this skill the moment an LM call surprises you, BEFORE any other debug move.

Do NOT activate when:

• You are authoring a new prompt for the first time (no rendered prompt exists yet).
• The bug is clearly outside the LM call (retriever returned empty, API key invalid, network down).
• The framework hasn't even been called yet (e.g., import error, schema validation pre-call).

The trigger is universal across the stack. Any framework that templates a prompt — DSPy, LangChain, LangGraph, CrewAI, LlamaIndex, Aider, Guidance, Outlines — has a layer between "what you wrote" and "what the model received." This skill is the first-line probe into that gap.

2. 核心心智模型 (Core mental model)

┌──────────────────┐    ┌──────────────────┐    ┌──────────────────┐
│ What you wrote   │ ≠  │ What was rendered│ ≠  │ What the LM saw  │
│ (template,       │    │ (after few-shot  │    │ (after provider  │
│  signature, etc) │    │  injection, tool │    │  reformatting,   │
│                  │    │  defs appended,  │    │  message squash, │
│                  │    │  system msg,     │    │  token truncation)
│                  │    │  history, etc)   │    │                  │
└──────────────────┘    └──────────────────┘    └──────────────────┘
       layer 1                layer 2                  layer 3
       (your code)         (framework render)      (provider transport)

Three mental shifts the agent must internalize:

1. The rendered prompt is the ground truth, not your source code. Frameworks silently inject system messages, append tool JSON-schemas, deduplicate messages, summarise history, truncate context, reorder fields. The only trustworthy artefact is what was sent on the wire.

2. Inspect first, change nothing. Premature prompt edits hide the bug. If you "fix" the prompt before seeing the rendered output, you're optimising against a hallucination of the problem. The discipline is: dump → diff against expectation → identify which layer diverged → fix at that layer.

3. The inspect command is framework-specific but the SOP is universal. DSPy gives you inspect_history(n=1). LangChain gives you set_debug(True). Raw SDKs give you OPENAI_LOG=debug or HTTPX event hooks. You learn one cheat sheet (§7) once and the SOP applies everywhere.

The 30-second test. Before you do anything else, you should be able to print the exact final prompt text in <30 seconds. If you can't — you're not set up to debug LMs. Fix that first.

3. SOP 工作流 (SOP workflow)

A four-step ritual. Do them in order. Do not skip ahead.

Step 1 — Dump the rendered prompt (≤30 sec)

Look up the framework on the §7 cheat sheet. Run the inspect command. Get the actual text of:

• The system message (if any)
• The user message
• Any few-shot demos that were injected
• Tool / function definitions (if any)
• Assistant prefill (if any)
• The model's raw response

For DSPy: dspy.inspect_history(n=1) [dspy.ai/api/utils/inspect_history/]. For LangChain: from langchain.globals import set_debug; set_debug(True) [python.langchain.com/api_reference/core/globals/langchain_core.globals.set_debug.html]. For raw SDKs: export OPENAI_LOG=debug or ANTHROPIC_LOG=debug [github.com/openai/openai-python, github.com/anthropics/anthropic-sdk-python].

Step 2 — Diff against expectation

Write down (mentally or in a scratch file): what did I expect the prompt to contain? Then compare to the dump. Look for:

• Extra content you didn't intend (auto-injected system message, hidden few-shot, tool JSON-schema bloat).
• Missing content you did intend (your variable interpolation rendered as None / empty / {var} literal).
• Wrong order (assistant message before system, tool call before tool result).
• Truncation (your 10k-token context was clipped to 4k; few-shot ate your real input).
• Encoding (raw HTML/JSON characters un-escaped, leaking template markers like {% if %}).

Step 3 — Identify the divergence layer

Now classify the bug into one of three layers (mental model §2):

• Layer 1 (your code): variable not interpolated, wrong key passed to chain, signature field misnamed → fix in your code.
• Layer 2 (framework render): framework's template logic injected something unwanted, demos chosen by optimizer don't match task, tool definitions inflated past budget → fix at the framework config (different prompt template, fewer demos, smaller tool set, different module).
• Layer 3 (provider transport): provider's message squashing, role coercion, token cap, finish_reason="length" → fix at provider config (different model, raise max_tokens, restructure to avoid role coercion).

Step 4 — Fix at the correct layer, then re-dump

Apply the minimum change at the identified layer. Re-run. Re-dump. Confirm the prompt now matches expectation. Then check whether the bug is fixed.

Critical anti-pattern: do not skip step 4's re-dump. Many "fixes" change Layer 1 when the real bug is Layer 2 — the prompt still looks broken on re-dump, even if the visible symptom changed.

4. 操作模型 (Operational model — Trigger / Action / Output / Evidence)

OP-1 · Inspect last N DSPy LM calls

• Trigger: A DSPy Predict / ChainOfThought / ReAct returned wrong output, OR a compile run hung mid-trial.
• Action: import dspy; dspy.inspect_history(n=1) — increase n to see the last few calls. For per-LM history: lm.inspect_history(n=3).
• Output: Prints system / user / assistant blocks plus the raw response for the last N LM invocations.
• Evidence: dspy.ai/api/utils/inspect_history/, DSPy docs "Debugging & Observability" [dspy.ai/tutorials/observability/]. Note: inspect_history shows only LM calls, not retriever/tool calls — for those, wire mlflow.dspy.autolog() [github.com/stanfordnlp/dspy issue #784 for n-parameter quirks].

OP-2 · LangChain global debug / verbose

• Trigger: A LangChain chain or agent produced unexpected output; you want to see every LM call's prompt and response.
• Action:

  from langchain.globals import set_debug, set_verbose
  set_debug(True)     # most verbose — full prompt + response + chain internals
  # OR for less noise:
  set_verbose(True)   # prompt + response only, no chain internals

  Scope: process-global. Toggle off when done.
• Output: Console-printed prompt/response for every LM call in the process.
• Evidence: python.langchain.com/api_reference/core/globals/langchain_core.globals.set_debug.html, python.langchain.com/api_reference/langchain/globals/langchain.globals.set_verbose.html, debugging guide [python.langchain.com/v0.1/docs/guides/development/debugging/].

OP-3 · LangGraph time-travel state inspection

• Trigger: A LangGraph thread produced a wrong final answer; you need to see the state (and the LM input) at step k, possibly fork from there.
• Action:

  history = list(graph.get_state_history(config))   # newest → oldest
  for snap in history:
      print(snap.config["configurable"]["checkpoint_id"], snap.values)
  # Replay from a checkpoint:
  graph.invoke(None, config={"configurable": {
      "thread_id": "...", "checkpoint_id": "<chosen id>"}})
  # Fork by modifying state:
  graph.update_state(config, {"some_field": "new value"})

• Output: Full per-step state list (including messages, the inputs the LM saw); ability to replay or fork.
• Evidence: langchain-ai.github.io/langgraph/concepts/time-travel/, docs.langchain.com/oss/python/langgraph/use-time-travel. Caveat: replay re-executes nodes — LLM calls and API requests fire again and may produce different results [time-travel docs]. To inspect without re-cost, read state from history only; do not invoke.

OP-4 · CrewAI step-by-step inspection

• Trigger: A CrewAI agent picked the wrong tool, looped, or delivered the wrong output; you need per-step visibility.
• Action: Two layers:

  Agent(..., verbose=True)             # prints agent thoughts + tool I/O
  Crew(..., verbose=True)
  # Per-step structured capture:
  def my_step(step_output):
      print("STEP:", step_output)      # AgentAction / AgentFinish / observation
  Agent(..., step_callback=my_step)

• Output: Every agent action, tool input, tool observation, and final answer printed or captured to your callback.
• Evidence: docs.crewai.com/en/concepts/agents (step_callback parameter), docs.crewai.com/en/observability/overview. CrewAI's internal log is thin — for production observability also wire mlflow.crewai.autolog() or Langtrace [docs.crewai.com/how-to/langtrace-observability].

OP-5 · Aider show last diff and full prompts

• Trigger: Aider applied a wrong edit, mis-identified files, or behaved differently than expected.
• Action:

  /diff           # see exactly what Aider just changed
  /tokens         # see how big the context actually got
  /ls             # see which files are in chat vs read-only

  Plus CLI: aider --verbose prints the full prompt sent to the model on each turn.
• Output: Last-turn diff, current token budget, file set, and (with --verbose) the rendered prompt.
• Evidence: aider.chat/docs/usage/commands.html, aider.chat/docs/usage/tutorials.html. Combine with git diff HEAD~1 for verification — Aider auto-commits, so the diff is also in git history.

OP-6 · Raw OpenAI / Anthropic SDK debug logging

• Trigger: You're calling client.chat.completions.create(...) (OpenAI) or client.messages.create(...) (Anthropic) directly, and the response is wrong. No framework templating layer to blame.
• Action:

  # OpenAI:
  export OPENAI_LOG=debug   # also: OPENAI_LOG=info for less verbose
  # Anthropic:
  export ANTHROPIC_LOG=debug

  Both SDKs use the stdlib logging module — the env var sets the logger level.
• Output: Full request/response bodies (and headers) printed to stderr.
• Evidence: github.com/openai/openai-python README §Logging, github.com/anthropics/anthropic-sdk-python README §Logging.
• CRITICAL security caveat: at debug level, the Authorization header (including the API key) is printed in plaintext [github.com/openai/openai-python issue #1196, issue #1082]. Never commit a debug-level log file. Strip keys before sharing.

OP-7 · HTTPX event-hook logger (full control)

• Trigger: You need the full raw HTTP request body (e.g., to verify what additional_drop_params or extra_body did), and OPENAI_LOG=debug is too noisy or formats badly.
• Action: Pass a custom httpx.Client with event hooks to the SDK:

  import httpx, json
  from openai import OpenAI
  def log_request(req):
      print("REQUEST:", req.method, req.url)
      print(json.dumps(json.loads(req.content), indent=2))
  client = OpenAI(http_client=httpx.Client(event_hooks={"request": [log_request]}))

• Output: Clean, structured dump of the exact JSON body sent over the wire.
• Evidence: Simon Willison's TIL [til.simonwillison.net/httpx/openai-log-requests-responses], httpx event-hooks docs [encode/httpx discussion #3073]. Works for both OpenAI and Anthropic SDKs (both accept http_client=).

OP-8 · Tool-definition bloat probe

• Trigger: Tokens / latency spiked, model "got dumber" after adding tools, or context-window error fires before your real input.
• Action: Dump the prompt (any of OP-1–7), then count tokens in the tool-definitions block alone. Most frameworks serialise each tool's JSON-schema (description, parameters, types, enums) into the prompt — 10 tools easily costs 2k+ tokens before any user input.
• Output: Concrete token cost per tool; identification of which tools to drop or compress.
• Evidence: Reproducible by inspect-dump; community pattern documented in agent-framework debugging threads (e.g., github.com/microsoft/semantic-kernel discussion #1239 on getting the raw prompt).

OP-9 · Production trace inspection (LangSmith / LangFuse / MLflow / Phoenix)

• Trigger: Bug surfaced in production; you don't have a repro locally; you need to inspect a real user's thread.
• Action: Open the trace in the configured platform (LangSmith / LangFuse / Phoenix / MLflow), navigate to the failing run, click into the LM call, view the rendered prompt and response. This is the production-equivalent of OP-1–7.
• Output: Reviewed prompt for the failing run, ready for diff against expectation (Step 2 of SOP).
• Evidence: Use the corresponding skill (langsmith, phoenix, langfuse) for the platform-specific UI. This skill enforces the first-move SOP; those skills provide the UI.

OP-10 · Save the rendered prompt as a fixture

• Trigger: You've found the bad rendered prompt and now need to reproduce / regression-test.
• Action: Save the dumped prompt text to tests/fixtures/prompt-bug-<id>.txt. Add a test that asserts the next render does not contain the bad pattern. (For LangChain, snapshot the prompt.format(**inputs) output; for DSPy, snapshot what inspect_history printed.)
• Output: A regression test that pins the prompt rendering, not just the model output.
• Evidence: Same principle as snapshot testing in UI — the rendered prompt is the artefact to pin.

5. 困境决策案例 (Dilemma cases)

Case A — "Framework template silently truncated my context"

困境 (Dilemma): A RAG pipeline retrieves 10 passages and asks the LM to synthesize. Outputs miss obvious facts that are in the retrieved passages. User's first instinct: "the model is bad / the retriever is bad / let me re-rank."

约束 (Constraints):

• 10 passages, ~600 tokens each = ~6k tokens of context.
• Framework: a custom LangChain MapReduceDocumentsChain wrapper.
• Model: GPT-4o, 128k context — no obvious context-budget issue.

决策步骤 (Decision steps):

1. Refuse to re-rank, refuse to swap the model, refuse to "fix" the prompt. First move is inspect (§3 Step 1).
2. Enable set_debug(True) from langchain.globals [python.langchain.com/api_reference/core/globals/langchain_core.globals.set_debug.html]. Re-run.
3. In the dumped prompt: notice the prompt only contains 3 passages, not 10. The framework's map-reduce splits and the reduce prompt got only the per-chunk summaries — and the chunk-summarizer dropped facts.
4. Layer classification (§3 Step 3): Layer 2 — framework render. The bug is the map-reduce decomposition, not the model or the retriever.
5. Fix: switch to a stuff chain (all 10 passages in one prompt — fits in 128k) or write better chunk-summary prompts. Re-dump to confirm all 10 passages now appear.

结果 (Outcome): Wrong layer would have been: a week of re-ranker tuning. Right layer: 10 minutes of template fix. Visible only via rendered-prompt dump.

可提取的操作 (Extractable operation): When a RAG pipeline misses obvious retrieved facts, dump the prompt and count how many retrieved chunks actually appear. The framework probably dropped some.