Consult Perf Knowledge

Consult Performance Knowledge

<!--
SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

Turn the current audited performance finding into one documented configuration proposal. Write the reasoning record; do not edit a deployment manifest, deploy anything, or run AIPerf.

Inputs

Require:

• EXP_ROOT and the zero-based current optimization iteration;
• exact EXP_ROOT/user_workload.yaml path and SHA256 supplied by the parent;
• exact active benchmark-plan path, SHA256, and series ID returned by perf-analyzer;
• current DEPLOY_ROOT/deployment_ledger.json;
• current successful DEPLOY_ROOT/smoke_test_artifact.json;
• current DEPLOY_ROOT/applied_manifests/deploy.yaml;
• current DEPLOY_ROOT/benchmark/benchmark_audit.json;
• current DEPLOY_ROOT/benchmark/benchmark_summary.json;
• current DEPLOY_ROOT/benchmark/performance_analysis.json;
• prior deployment and benchmark artifacts; and
• EXP_ROOT/analysis/hypothesis-backlog.jsonl and EXP_ROOT/analysis/challenger-reviews.jsonl when present.

Treat the current deploy-iter-<NNN> as the source iteration and <NNN + 1> as the candidate iteration. Never create the next deployment-iteration directory.

After every consultation, whatever its outcome, append one record to EXP_ROOT/analysis/hypothesis-backlog.jsonl (the proposal or non-proposal and its source evidence), creating the file on first use. When recording an ask, append it to EXP_ROOT/analysis/asks.jsonl per run-artifacts.md, deduplicating against existing entries first.

Read The Applicable Knowledge

Always read:

• agent-docs/rules/benchmarking/evidence-eligibility.md;
• agent-docs/rules/benchmarking/comparison-uncertainty.md;
• agent-docs/rules/benchmarking/series-boundaries.md;
• agent-docs/rules/optimization/evidence-before-spend.md;
• agent-docs/rules/optimization/one-variable.md;
• agent-docs/rules/verification/config-engagement.md;
• agent-docs/rules/verification/implausible-speedup.md;
• agent-docs/rules/verification/overlap.md;
• agent-docs/rules/verification/stack-verdict.md;
• agent-docs/guides/knob-tuning/tuning-hierarchy.md;
• all three files under agent-docs/guides/model-sizing/;
• agent-docs/guides/knob-tuning/dynamo.md; and
• only the active engine guide: agent-docs/guides/knob-tuning/vllm.md, agent-docs/guides/knob-tuning/sglang.md, or agent-docs/guides/knob-tuning/tensorrt-llm.md.

Additionally read:

• agent-docs/guides/rate-matching/matching.md for a disaggregated allocation decision;
• agent-docs/rules/benchmarking/concurrency-grid.md when interpreting a capacity or concurrency series;
• agent-docs/rules/benchmarking/proxy-workload-selection.md when the audit identifies a recipe proxy;
• agent-docs/rules/benchmarking/benchmark-isolation.md when the audit reports an isolation limitation; and
• agent-docs/references/reference-repos.md before consulting current framework or Kubernetes source or official documentation.

Do not load guides for inactive engines. Verify version-sensitive flags and defaults against the active image, checked out source, generated help, or official documentation. Treat evidence transferred across a model, engine version, hardware class, topology, or workload as an explicit assumption.

Validate The Decision Inputs

Proceed with a proposal only when:

• the smoke test succeeded;
• the benchmark audit status is valid or valid_with_recovery;
• the plan, audit, summary, and analysis identify the same active benchmark series;
• the summary and performance analysis refer to the current candidate and executed workload;
• the exact successful source manifest exists;
• target-fixed model, framework, precision, hardware, workload, and SLO constraints are known; and
• every direct comparison used as decision evidence is valid and same-series.

Record resource or placement differences as limitations. A valid absolute characterization may support a proposal without a prior reference, but it cannot establish a gain or loss. If an input or claimed comparison is missing, inconsistent, invalid, or non-comparable, write a blocked consultation. If the inputs are valid but no defensible lever meets the evidence gate, write no-proposal.

Establish The Performance Finding

Record:

• the primary objective or failed SLO and target operating region;
• absolute current metrics and client-visible symptoms;
• comparisons with the series baseline, previous valid iteration, best prior result per objective, and relevant same-series history when available;
• run count, repeat rationale when applicable, the measured noise floor of the active benchmark series (per comparison-uncertainty.md), and AIPerf confidence intervals or coefficient of variation only when deliberate repetitions made them useful;
• proxy limitations, missing metrics, resource or placement differences, or other uncertainty; and
• whether any surprising prior result needs engagement or plausibility rechecking.

AIPerf establishes client-visible behavior, not a router, scheduler, transfer, or backend root cause. State internal mechanisms as hypotheses unless separate engagement or runtime evidence supports them.

Enforce The Evidence Gate

A proposed candidate must cite at least three distinct evidence categories, and one must be AIPerf profiler data:

1. AIPerf profiler data and audited analysis tied to the objective or SLO.
2. Dynamo or active-engine source, official documentation, or performance guidance.
3. Same-series benchmark history from prior candidates.
4. Model architecture details relevant to the mechanism.
5. Hardware speed-of-light or roofline analysis when the mechanism concerns compute, memory, or communication limits.

Count categories, not citations. Multiple AIPerf metrics still count as one category. For each item, record the exact path or citation, observation, what it supports, and its limitation. Do not invent category 5 when no applicable analysis exists. If fewer than three categories qualify, use no-proposal and name the missing evidence.

Calibrate Exploration Versus Exploitation

Before generating a hypothesis, determine whether a broad or narrow knob adjustment is needed.

• Prefer a broad move while any materially applicable, plausibly higher-upside lever family remains untested or has not been ruled out by current evidence.
• Prefer a narrow move only after the broad landscape has been screened and valid measurements show a demonstrated, meaningful signal in the selected family.
• Return to exploration when a narrow change is invalid, inconclusive, within the measured noise floor of the active benchmark series, reverses the expected direction, or reveals a different limiting regime.

Maintain one persistent search-calibration ledger for the engagement at EXP_ROOT/analysis/search-calibration.md instead of regenerating a scan for every hypothesis; each iteration's knowledge-consult.md records only the delta applied to it. The ledger is the authoritative family table. When submitting a stop-request, record in knowledge-consult.md the ledger path and the SHA256 of the ledger state being submitted, plus — whenever any granted budget is non-null — the derived budget consumption (wall clock from manifest.yaml's session start; failed deploys from the deployment ledgers' failed_attempts records; GPU-hours from GPU allocation time (per deployment ledger: allocated_at to torn_down_at, or to now if live, times its gpus_requested, summed across deployments)); do not modify the ledger while that validation is pending. Before each hypothesis, update the ledger by delta, re-reviewing every row whose evidence regime changed (a topology adoption, new variance data, an answered ask). The ledger explicitly covers:

1. deployment topology and fit, including model fit, parallelism, replication, aggregated versus disaggregated serving, disaggregated rate matching, GPU allocation, and placement or fabric constraints;
2. every Category 2 family in tuning-hierarchy.md: CUDA graphs; admission, batching, prefill scheduling, and workspace; speculative decoding; KV-cache dtype and capacity; engine backend or autotuner selection; Dynamo routing and prefix reuse; KVBM or engine KV offload; and frontend, transport, and pod resources; and
3. Local Planner only when its conditional lane applies.

For each family, record its coverage as tested, ruled-out, not-applicable, untested-promising, deferred, or reopened-by-new-evidence, plus its expected upside — recorded BOTH as a quantitative estimate and on the ordinal scale with cutoffs DERIVED per engagement, recorded for readers of the ledger: low (below the primary objective series' measured minimum detectable effect — indistinguishable from noise), medium (above the MDE but below the engagement's practical-significance threshold), high (above that threshold). The practical-significance threshold is the user's stated smallest-delta-that-matters when the interview captured one; otherwise DEFAULT it to twice the measured MDE and say so. Record both derived cutoffs and their derivation in the ledger header (mde:, practical_significance: lines above the family table) — and the evidence for that disposition. A deferred row's cost-estimate-vs-remaining-budget numbers live in its Evidence and reason cell. A ruled-out row must cite a measurement, a sourced hard constraint, a confirmed incompatibility, or an explicit operator decision; expected upside below the minimum detectable effect is deferred (still visible, and stackable under a documented one-variable exception), never ruled-out. Compare all applicable families for potential benefit and information value before choosing one.

During exploration, prefer an independently testable change that crosses into a different high-impact family or tests a coarse, documented operating regime. Do not keep adjusting the same knob in single-digit or otherwise near-neighbor increments while a plausible higher-impact family remains untested. An immediate adjacent adjustment is appropriate only when current evidence shows that family dominates the objective or a broad scan finds no plausible higher-upside alternative; record that exception.