Basin Perf Investigation

Investigate Basin Performance

Reproduce the reported workload, establish a fair reference, and explain the cost with measurements. An investigation can finish with a supported diagnosis and ranked next steps. When the user requests a speedup, carry promising fixes through numerical verification and fresh timing. Do not turn an investigation alone into a solver redesign or publication of benchmark results.

This is a project-local skill for Basin's Rust optimization library and its workspace. Run commands from the repository root. Read AGENTS.md, the relevant benchmark source, and the target implementation. Consult CONTRIBUTING.md before changes to architecture, public APIs, dependencies, or platform support.

Establish the comparison

Keep the user's solver, variant, workload, and reference. If none is specified, choose a representative existing case and state the choice. For a regression, compare against a known earlier revision.

Before choosing kernel optimizations for an implementation comparison, establish whether the competitor uses the same algorithm variant. If it differs, retain the user's competitor as the practical performance target and identify a matching reference for implementation-cost comparisons. Prefer author-maintained reference code or an established library implementing the same variant. Record versions or commits and material differences; use primary documentation or source to resolve ambiguous reference semantics.

When whole-solver trajectories differ, start the implementation comparison with kernels on identical inputs. If no matching reference is available, document that limitation and use matched before/after work to support implementation speedups without attributing the whole competitor gap to implementation overhead.

Before timing, run each contestant once and record:

• Problem dimensions, data, starting point, scalar type, bounds, and constraints. Check constraint signs, residual scaling, and whether the reported objective is a sum of squares or half that sum.
• Algorithm settings: line search, memory size, initial simplex or population, trust-region radii, scaling, restart rules, and random seeds as applicable.
• Actual stopping conditions, including implicit solver stops, adapter tolerance floors, and the meaning of each budget. Equal tolerance values or iteration caps do not establish equal work across libraries.
• Returned objective, feasibility, gradient or residual norm when relevant, termination reason, iterations, and objective, derivative, and constraint evaluations. Validate the returned point independently outside the timer. Distinguish current and best-so-far state, and count verification calls separately. Budget exhaustion alone does not establish convergence; distinguish budget-limited output from verified success and invalid non-finite results.

Choose the measurement that answers the question:

• Implementation cost: matched work, such as a kernel on identical inputs or the same solver steps. Verify evaluation counts and numerical results; equal iterations alone can hide different line-search or inner-solver work.
• Solver effectiveness: time and evaluations to a common accuracy and feasibility target, or quality achieved within a common budget. Retain failed cases and success rates. Use paired seeds and multiple starts when stochastic variation matters. Convergence traces help explain differing trajectories.

Use both views when fewer evaluations could explain the speed difference. For non-solver components, establish equivalent outputs and the relevant work unit instead of imposing solver convergence criteria.

Benchmark and profile

Read Benchmarking Basin when selecting a harness or preparing timings. It maps the existing benchmarks and verification probes, explains Basin's counter and solver-lifecycle semantics, and gives commands for reference comparisons and before/after baselines. Check those semantics before using a state's cost_evals() as a measure of objective calls.

Establish an uninstrumented baseline before changing production code. Record versions, features, workload, timing boundaries, and sample uncertainty. Isolate historical builds from the user's checkout, and alternate prebuilt contestants when drift could obscure a small difference. Match numerical work and thread settings before attributing a gap to implementation cost.

Read Profiling Basin when collecting a profile. It provides a focused Criterion flamegraph command, optimized build settings, stack-validation guidance, and fallback options. Keep profiling and allocation instrumentation separate from timing builds; profile percentages alone cannot establish a speedup.