Warp Eval

Warp evaluation

Purpose

Collect reproducible evidence about how a narrow seam in an existing codebase would behave in NVIDIA Warp. Report facts; the user decides.

Name Warp as the option under evaluation in the first line, state that no adoption recommendation will follow, and do not treat the triggering performance request as authorization to experiment.

Measured evaluations produce warp-evaluation-report/: the report, one independently applicable diff per solution, the drivers, and raw results. Never modify production code. Exits before measured work create no directory.

Hard rules

These override any local reasoning.

1. Default objectives are latency, throughput, and peak or retained memory. Count maintainability, ergonomics, packaging, extensibility, autodiff or new functionality only when the user names it; otherwise report them as constraints or costs, not benefits.
2. Correctness is a gate. If the incumbent is buggy or its contract unclear, abort that comparison until an independent oracle or clarified contract exists. Report the defect without prescribing a response.
3. Never predict savings from source shape or another workload. Materialized memory comes from profiler or allocator evidence, not source a compiler may fuse.
4. "It can be written in Warp" is a hypothesis, never a proven opportunity.
5. Gates are exact, never adjacent or analogous. Fire a gate only when every condition in its definition is established. Gate D requires a maintained implementation confirmed to execute with CUDA on an NVIDIA GPU; a fast native CPU library is a baseline, not Gate D.
6. Label every claim as observed fact, measurement, hypothesis or unknown.
7. Time the user-visible stage, not the kernel. Include Warp's cold import/init/JIT in the real process regime, transfers, launches, Python launch loops, structure build/refit, allocation, conversion, validation, compaction and synchronization. Report each cost and the end-to-end difference.
8. No measured work without explicit authorization. After stage 1, stop and ask before profiling, environment changes, prototyping, benchmarking or GPU use.
9. Authorized measured work includes Warp. If profiling exposes no gate, continue through the strongest in-project baseline, minimum Warp prototype and end-to-end comparison. If Warp is out of scope, stop before profiling.
10. Never infer environment intent. NVIDIA deployment and ownership of an optional compiled dependency are product decisions. Abort only on a stated constraint; otherwise ask once and stop (AWAITING INTENT).
11. Warp measurements are CUDA-only and synchronized. Resolve an explicit NVIDIA CUDA device; discard CPU resolution and dispatch-only timing.
12. Never recommend or rank adoption options. Report facts, measurements, hypotheses and unknowns per seam and regime.
13. Stop at the report. No rollout or production edits.

Requirements

Static screening requires the target repository and its stated product constraints. Measured work additionally requires explicit authorization, an NVIDIA CUDA GPU, target-project dependencies and a representative workload.

Limitations

• Warp CPU kernels are serial and outside this skill's evaluation scope. "Run it on the CPU instead" is not a performance fallback.
• Mesh and geometry queries compute in float32/int32 even behind a float64 public API. Absolute error grows with coordinate magnitude.
• Warp does not follow semver. Feature releases can break APIs, only the newest feature line is maintained, deprecations run roughly four monthly releases.
• latest docs track development, not the shipped release. Pin to the target project's own Warp version; if it has none, use the current stable and say which.
• Kernel-only builtins are not resolvable from Python — hasattr(wp, "mesh_query_point") is False on a release that has it. Probe the stub file or that version's docs before concluding a builtin is absent.
• enable_backward=False (kernel, module or global) removes adjoint codegen. If nothing differentiates through the seam, set it before measuring compile cost.

Output Format

Use this exact shape for an early exit: ABORT — Gate <letter>: <cited fact>; <why the scoped Warp evaluation cannot proceed>. Do not name a preferred alternative. Reporting rules: references/evidence-and-reporting.md. A delivered directory follows the template's fixed order: schema and provenance, authorization and evaluation state, stage census, one B<n> evidence section per seam/regime, caveats, then environment/reproduction. solutions/, benchmarks/ and results/ contain every linked artifact.

Examples

Gate exit: ABORT — Gate A: deployment.md requires one implementation with AMD, Apple and NVIDIA parity; a Warp-specific path cannot satisfy this scope.

Surviving candidate: Name the seam and pattern, label inferred facts as assumptions, state that no gate has fired, preview the profile/baseline/Warp prototype/benchmark scope and its cost, then ask the separate intent and authorization questions from references/authorization-checkpoint.md.

Inputs

Required: the target repository and a performance, memory or scale problem with a candidate seam. Optional: explicit deployment/packaging constraints, existing profiles or logs, representative datasets and acceptance criteria. Prompt constraints take precedence over repository policy/configuration, then existing logs. User corrections override inference. Never substitute an assumption for a stated fact or measurement.

Available scripts

Use run_script("scripts/validate_report_schema.py", args=["warp-evaluation-report"]) when supported; otherwise invoke the script with Python and the report directory.

Troubleshooting

• No representative workload: mark the scope INCOMPLETE and name the missing artifact; do not invent data or fire Gate F.
• Warp resolves to CPU or no CUDA device: discard the run and stop before correctness or timing claims.
• Report validation fails: fix the report or referenced artifact; never waive the schema error.

Instructions

Every stage before the last can end the evaluation. Stop as soon as a gate fires; do not gather evidence that cannot change the scoped facts.

1. Read the code, derive the contract, check the gates

• Identify a candidate and its metric with references/target-patterns.md.
• Derive devices/residency, dtypes/shapes, sizes, frequency, process lifetime, gradients and packaging from the repository. Infer before asking.
• State the inferred contract in one line and invite correction. Unknown hardware, counts, sizes and tolerances are assumptions, never measurements. Every inference remains open to correction and cannot satisfy a gate that requires a stated fact or measurement.
• Check Gates A–E before profiling. Check Gate F now only if representative evidence already exists; otherwise carry it into stage 2. Every gate uses only the exact boundaries in references/rejection-gates.md.

• Gate F can fire in stage 1 only from representative evidence that already exists — a supplied profile, structural bound, or arithmetic on figures the user quoted. If that evidence does not exist, Gate F remains open until stage 2 profiling; inferred values never fire it.
• Gates A and E need a stated constraint. A CPU implementation, another accelerator, no Warp dependency, or a small dependency list proves nothing.
• When A/E are unresolved and a pattern survives, ask whether an optional NVIDIA path is acceptable: named extra, soft import, existing fallback, default install unchanged. Every affirmative answer must say explicitly that Warp will be prototyped and benchmarked; conditions constrain only that Warp scope. A negative or undecided answer means ABORT.
• Do not ask when another gate fired, the repository answers, or no pattern matched. No pattern means no profiling.
• If a candidate survives, combine any intent question with the authorization checkpoint — early findings, exact scope, stages, resource cost — then stop. Stage 2 requires both settled intent and explicit authorization.