Debug

Debug Skill

Use this skill when debugging any issue in the MCP Gateway Registry: timeouts, 5xx errors, UI failures, broken flows, unexpected behavior. Invoke this skill proactively whenever normal debugging is failing or going in circles, or use it from the start for any non-trivial issue. We are testing whether this approach should be the default for all debugging.

Expert Personas

Before starting, announce that you are consulting the following expert panel. Each persona challenges assumptions and contributes their domain expertise:

• SRE/Infrastructure Engineer - Thinks about: single-worker bottlenecks, event loop blocking, connection pooling, DNS resolution, proxy timeouts, container resource limits, startup ordering.
• Frontend/Browser Security Engineer - Thinks about: same-origin vs cross-origin, preflight behavior, cookie scope, session validity after restarts, browser caching of error responses, service worker interception.
• Backend/Python Engineer - Thinks about: asyncio event loop blocking, synchronous HTTP calls on async workers, Pydantic model_dump dropping extra fields, DB write ordering, exception swallowing in try/except.
• Senior Staff Engineer (Skeptic) - Questions EVERY theory before implementation. Asks: "If that were true, why does X work?" and "Was this working before our change?" Forces the team to prove hypotheses before writing code.

State which persona is "speaking" when presenting a theory or counter-argument. The Skeptic must challenge every theory before any code change is proposed.

First Principles Debugging

Step 1: Reproduce and observe (do NOT theorize yet)

• What exact error does the user see? (HTTP status, error message, timing)
• What was the user doing? (which page, which action, which persona)
• Does the same operation work via curl from the command line?
• If curl works but browser doesn't: the difference is timing or session, NOT the backend logic.

Step 2: Check what the worker is DOING right now

This is the single most important step. The registry runs a single uvicorn worker. If a request times out, the worker is busy doing something else.

docker compose logs registry --since=30s 2>&1 | tail -30

Read what the logs say at the EXACT timestamp of the failure. Do NOT filter by keywords related to your bug. Look at EVERYTHING the worker is doing. Common blockers:

• GoDaddy ANS API pagination (hundreds of sequential HTTP GETs)
• Federation sync (AgentCore, peer registries)
• Health checks pinging 30+ MCP servers
• Vector index processing (embedding generation, index updates)
• Security scans (yara, spec analyzers take 3-8 seconds each)

If the worker is busy with a background task, your request is queued. The fix is to make that task non-blocking, NOT to change the endpoint code.

Step 3: Is this new or pre-existing?

Before touching ANY code, answer:

• Was this working before on the current branch?
• Was this working on main / the last release tag?
• Did a recent restart, rebuild, or config change cause it?

If the issue exists on the baseline tag with zero changes, it is pre-existing and unrelated to whatever PR you are working on. Do NOT fix pre-existing issues in a PR scoped to something else unless the user explicitly asks.

Step 4: If you see something suspicious in the logs, ask HOW it connects

When you see an error or warning in the logs, do NOT immediately assume it is the root cause. Ask:

• Is this error on the code path of the failing request, or is it a different background operation?
• Does this error BLOCK the worker, or is it caught and handled?
• Would fixing this error actually change the user-visible symptom?

Example: seeing FAISS index not initialized in logs does NOT mean FAISS is the problem. It might be a non-fatal warning from a background task that runs in parallel. Trace the actual call stack of the failing request.