Cubic Audit

cubic-audit

Periodic audit of the cubic AI-review configuration (cubic.yaml at the repo root). Two deliverables, in order:

1. Rule review: are the custom review rules the highest-signal set for this repo, correctly scoped, within cubic's limits, and factually accurate against the code?
2. Learnings reconciliation: which accumulated cubic learnings should be promoted into durable rules, and which are obsolete / duplicate / contradictory and should be cleaned up in the cubic UI.

Two modes. Deliverable 2 needs the cubic learnings MCP (tools list_learnings / get_learning). That server is connected per-account in the cubic app, not declared in the repo's .mcp.json, so a teammate running this skill may not have it. Probe for it in step 1: if it is reachable, run the full audit (both deliverables). If not, run rules-only mode (deliverable 1 plus the cubic.yaml validation in step 4), and tell the user the learnings reconciliation was skipped because the cubic MCP is not connected, with a one-line pointer on how to enable it (connect the cubic integration in the cubic app, then re-run). Never block the whole skill on the missing MCP.

Cubic docs (re-check if a limit below seems wrong): https://docs.cubic.dev/configure/cubic-yaml and https://docs.cubic.dev/ai-review/custom-agents

Language: all output (chat, cubic.yaml text, commit messages) is US English, regardless of the language used to invoke the skill.

Open source: cartography is a public CNCF project. Never reference clients, internal tickets, or private deployments in cubic.yaml, commits, or learnings recommendations.

Hard constraints (bake these in)

• Max 5 enabled custom rules per repo. Order matters; beyond the limit only the first N take effect. To add one you must merge or drop one. cubic.yaml is already at the cap (5 custom_rules), so every "promote" recommendation must name what it merges into or replaces.
• 10,000 characters per rule, and that budget includes the resolved content of any file_paths. Attaching a large doc (AGENTS.md, a schema.md) is counterproductive: it truncates to the least-relevant top and starves the prompt. Prefer self-contained prompts; only link a file if it is small (<2k) and review-focused.
• cubic.yaml merges across levels: repo > org > UI > built-in defaults.
• No MCP tool deletes or edits a learning. Learnings cleanup is a manual action in the cubic UI. This skill produces the list; a human actions it.
• No em-dashes / en-dashes anywhere in cubic.yaml or commits. Run grep -n '—\|–' cubic.yaml before declaring done.
• Honor the repo's worktree/path and git rules in AGENTS.md (CLAUDE.md may be a symlink to it; read AGENTS.md directly). Commit/push only when explicitly asked; commits use --signoff (DCO is required on this repo); never open a PR without per-PR authorization.

Workflow

0. Context

• Confirm the repo root (or worktree). Read cubic.yaml.
• Derive the GitHub slug: git remote get-url origin -> owner/repo (this repo: cartography-cncf/cartography).

1. List learnings (cubic MCP) -- skip if unavailable

The cubic MCP server is namespaced by an opaque id and is connected per-account in the cubic app, not in the repo's .mcp.json. Probe first: run ToolSearch for list_learnings get_learning cubic. If it returns no matching tool, the cubic MCP is not connected in this environment: skip the rest of this step and all of steps 2 and 5's cleanup table, run rules-only mode (steps 3-4), and state in the report that learnings reconciliation was skipped (connect the cubic integration in the cubic app and re-run to enable it). Do not fabricate learnings from memory.

If the tool is found, call list_learnings(owner, repo).

The result is large and is persisted to a file (the tool message returns its path). Do not read it raw. Extract a compact view and read 100% of it:

# f = the persisted list_learnings result file path from the tool message
awk '
/^## / { cat=$0; sub(/^## /,"",cat); next }
/^### / { title=$0; sub(/^### /,"",title); desc=""; getline; while($0=="") getline; while($0!="" && $0 !~ /^- \*\*/){ desc=desc $0 " "; getline } next }
/^- \*\*Confidence\*\*/ { c=$0; sub(/.*: /,"",c) }
/^- \*\*Source\*\*/     { s=$0; sub(/.*: /,"",s) }
/^- \*\*Updated\*\*/    { u=$0; sub(/.*: /,"",u); printf "[%s] %s | %s | %s | %s\n    %s\n", cat, title, c, s, u, desc }
' "$f" > /tmp/cubic_learnings_compact.txt
wc -l /tmp/cubic_learnings_compact.txt

Read the compact file fully (chunk it). Never pipe the awk through head: it silently drops the tail and you will miss whole categories.

2. Bucket the learnings

Most learnings are negative / suppression ("Do not flag X", "Treat Y as intentional", "Before flagging Z, check W"). These are false-positive corrections that already apply automatically. Do NOT turn suppression learnings into rules: a rule says "flag", a suppression says "do not".

• Promote candidates: positive / prescriptive ("always do X"), general, recurring (appears 2+ times or echoes a known convention), and not already covered by an existing rule. Expect only 1-3 per audit. Since the config is at the 5-rule cap, each promotion must fold into an existing rule's description rather than add a sixth.
• Cleanup candidates:
  • Contradictory: conflicts with a rule or with repo reality. Highest priority, since these suppress real findings.
  • Obsolete: references a removed/renamed module, deleted helper, or a label/pattern that no longer exists in the code.
  • Duplicate: near-identical pairs (often the same rule reworded, or one with a hyphen vs an arrow).
  • Low-value: vague entries under ~75% confidence that are platitudes. Optional prune.

Verify before recommending deletion. Never trust the learning text alone: grep the code for the referenced symbol / dir / feature. Patterns that pay off:

git grep -n 'def from_node_schema'             # the signature a rule quotes is correct?
git grep -lI 'aws_handle_regions' -- '*.py'    # decorator still referenced?
ls cartography/intel/<module> 2>/dev/null      # module still exists / not renamed?