Datadog Data Source Generator

Generate a Datadog data source (input → generation → PR)

This skill takes a Datadog v2 OpenAPI operation from nothing to a review-ready PR. It owns the whole flow — the earlier version assumed tfgen had already generated and committed; this version runs generation itself, so it also owns branching and committing.

Phase 1: INPUT        Phase 2: GENERATION                    Phase 3: PR
collect params  ──▶   slice_and_annotate.py → slice     ──▶  classify scenario
(routes, name,        tfgen generate → .go + test             evaluate vs goldens + risks
 cardinality,         make docs / make build                  draft PR body
 description,         branch off master + commit               open PR with gh
 overwrite)           (gate on the RunReport first)            report back

Each phase has its own reference subdirectory under references/. Read the reference for a phase before running it.

Operating principles — run it fast and hands-off

tfgen is deterministic and well-tested. This skill is a thin wrapper around it, not an audit. Three rules keep runs fast and honest:

1. Trust the generator. If tfgen produced the files and make build passed, treat the output as correct. Do not re-derive its decisions, re-verify field-by-field, or second-guess the scenario it emitted. Report what happened — don't prove the code right.
2. Never fix — report and stop. On ANY failure (spec/annotation error, generation failure, make build/make docs failure, or a red CI check) do not edit the spec, patch the generated code, retry, or work around it. Quote the error verbatim, say plainly why the data source could not be generated, and stop. For a failed run, that report is the deliverable.
3. Keep analysis light. The only judgment that matters is a quick scan for material runtime risks (references/pr/risk-heuristics.md) — minutes, not a line-by-line review, and not a golden diff. If nothing clearly applies, say so and move on.

The one rule that overrides everything: never overclaim "verified"

A green RunReport and a clean build prove only that code was generated and compiles — not that it works at runtime. A plural data source has built and reported created cleanly yet returned 0 rows live (read-after-write lag / silent-empty trap; see references/pr/risk-heuristics.md). So:

• Only use "verified", "working", "confirmed", or "replays green" if a cassette actually replayed green.
• If generation/build succeeded but no cassette has replayed, say exactly that: "generated and builds cleanly; runtime behavior not yet verified — see testing guide."
• When in doubt, describe what was checked, not what you assume.

This is the single most common way to write a misleading PR here. Guard against it in every section you draft, and never let the confidence of having generated the code leak into runtime claims.

Phase 1 — Input

Goal: produce a complete, validated parameter set for slice_and_annotate.py. Nothing is generated in this phase — you are only deciding what to generate.

Collect, confirming each with the user:

• Spec path — default: curl the upstream v2 spec to a temp file (see references/input/collecting-inputs.md); an explicit path or $DATADOG_OPENAPI_V2_SPEC overrides. A local copy must exist before you can discover routes.
• Read group (routes) — the operationIds. If the user names a resource/service and a spec is available, inspect it and propose candidate GET routes; otherwise take operationIds directly. Validate every operationId against the spec.
• Cardinality + scenario — singular (by-id / both / search-only) vs plural, derived from which GETs exist and what the user wants.
• Artifact name — default derived from the resource (snake_case, no datadog_ prefix); validate ^[a-z][a-z0-9_]*$, ≤64.
• TF description — default Use this data source to retrieve information about an existing Datadog <thing>. (plural: …existing Datadog <thing>s.); only ask if they want a custom one.
• Overwrite — auto-detect whether a hand-written datadog_<name> data source already exists; if so, find its constructor and ask whether to retire it (--overwrites). Otherwise additive.

End the phase by echoing the full parameter set back and getting a go-ahead.

Details: references/input/collecting-inputs.md.

Phase 2 — Generation

Goal: turn the parameter set into committed generated files on a fresh branch. Do not open a PR here.

1. Preconditions. gh auth status authenticated. If HEAD is master, that's fine — this phase creates the branch. Ensure bin/tfgen exists (make tfgen-build if not).
2. Build the slice. Call slice_and_annotate.py with the phase-1 params; capture the printed slice path (stdout is only the path). See references/generation/slice-and-annotate.md.
3. Generate. Run tfgen generate --spec "$SLICE" --report -, capturing the RunReport JSON. See references/generation/running-tfgen.md.
4. Gate on the report — before committing. Stop if summary.failed > 0 or any diagnostics[].severity == "error". Quote the failing artifact + diagnostics verbatim, say plainly why it couldn't be generated, and stop — do not edit the spec, retry, or fix anything (principle 2). Leave the working tree uncommitted. That report is the deliverable; nothing is committed. warning/info do not gate — carry them into the PR risk section.
5. Docs + build. Run make docs (creates docs/data-sources/<name>.md) and make build to confirm it compiles. Use make targets, never raw go. If either fails, quote the output and stop (principle 2) — do not attempt to fix the generated code.
6. Branch + commit. Create the branch off master and commit the generated .go, test, and docs files. Carry forward to Phase 3: the RunReport, the known scenario/cardinality, the slice path, and the branch name.

Details: references/generation/slice-and-annotate.md, references/generation/running-tfgen.md.