Porting Agent To Hub

Porting an Agent to a Hub-Native Package

Take an agent that exists only as in-repo Python and make it a published package a user can install from the Agent Hub and use in Agent UI v2 immediately.

The email agent is the template — hub/agents/email/python/ + hub/agents/email/npm/ + .github/workflows/release_agent_email.yml. It is the only agent in the repo that clears the bar; every phase below points at the file in it you are mirroring.

This skill is the porting flow. For adjacent work use: gaia-build-agent (a NEW agent) · agent-hub-release (cutting the release) · adding-eval-scorecard (the scorecard mechanics) · integrate-hub-agent (consuming one from an app).

The Iron Rule

Generalize before you document.

Docs, SPEC, SKILL and SCORECARD written against behavior that is about to change are wasted work — and a scorecard is meaningless until the capability is stable. Phases run in order. Do not jump to the parity kit because it looks mechanical.

Phase 0 — Decide whether it should ship at all

Not every agent should be ported. Before any work, get a verdict: PORT / MERGE INTO <target> / DISCARD (keep as in-repo example) / DEFER.

Check for a more general agent that already covers the use case — in in the other hub agents, and in ChatAgent's profiles. Duplicating a capability into the catalog is worse than not shipping.

Signals it is not a catalog agent: category: examples / security_tier: experimental; absent from setup.py's AGENT_WHEEL_PACKAGES; no gaia.agent entry point; a module docstring saying it exists to validate some other feature; a named successor already in flight.

Phase 1 — Capability-truth audit

Read the manifest's description, tags, tools_count and interfaces:, then prove each claim against the code. Assume the manifest is lying until checked — in the 2026-07 fleet audit it was wrong for most agents, in both directions.

• Does every advertised verb have a reachable tool? (An agent advertising CSV analysis that composes no file mixin cannot open a file.)
• Does every conversation_starter have a tool behind it? Agent UI shows these as first-run prompts, so a starter for a missing capability means the user's first click fails. Worse than a stale description — check these first.
• Does tools_count match the real @tool surface plus composed mixins? It lives in two places — gaia-agent.yaml and the build_registration() call in __init__.py. Check both; the registry copy is what the UI reads.
• Do the agent's own defaults work end to end? (A language="python" default that fails a TypeScript-only validator is a mis-scope, not a bug.)
• Does interfaces: claim a mode nothing serves — and is the dependency for that mode even declared? api_server/mcp_server need amd-gaia[api] in both pyproject.toml and the manifest's python.dependencies. Most agents declare the interface against plain amd-gaia, so the mode is not merely unimplemented, it is uninstallable — and it fails on a clean install, not in CI. word-count is the package that gets this right.
• Are preflights checking the right thing? docker --version tests the binary; the agent needs the daemon. A preflight can be present and still wrong.

Record every gap. This list is the port's actual scope.

Phase 2 — Generalize and harden

• Close every Phase 1 gap: implement the advertised capability, or narrow the manifest to the truth. Both are valid; shipping the mismatch is not.
• Remove single-instance / single-machine assumptions — an author's own service schema baked into a system prompt is the canonical case.
• Inject configuration instead of hardcoding paths, URLs, models, ports.
• Fail loudly (CLAUDE.md): no except …: pass, no default-to-empty, no swallowed retry. Errors name what failed, what to do, and where to look.
• Declare and preflight external dependencies (a daemon, a binary, credentials, VRAM). Check the service, not just the binary on PATH.

Phase 3 — Behavioral tests

The fleet-wide failure mode: tests import the agent, construct it, and assert a tool name appears in the registry. They never call a tool.

Every @tool needs a test that invokes it, over a fixture harness (temp FS, mocked network, temp scratchpad DB) with _TOOL_REGISTRY isolation. Cover the cold state a new user is in — empty index, empty DB, first run.

Phase 4 — Eval: corpus first, then scorecard

Two halves, and the expensive one is not the code:

• The oracle (not automatable): a labelled, human-curated ground-truth corpus plus a deterministic fixture harness so the eval needs no live service and no LLM judge. Email's lives in tests/fixtures/email/ — _stub_inbox.mbox plus a per-task ground truth (action_items_, briefing_, drafting_, followups_, longthread_) driven through FakeGmailBackend. ls it; there is no single ground_truth.json.
• The mechanism: the adapter, SCORECARD.md, the scorecard_gate.py wiring and a refresh workflow → use adding-eval-scorecard.

Pick the metric before starting, and prefer deterministic exact-match over a judge. Where no honest metric exists, say so in the scorecard rather than reporting a number that means nothing.

Beware the measurement trap: an agent_type in the scenario corpus may name a ChatAgent prompt profile, not your package. A green scenario can be measuring something else entirely.