Testlib Cpp Judging

Testlib C++ Judging

Scope

Use this skill when a task involves testlib.h, a special judge, a custom checker or scorer, an input validator, a deterministic test generator, or a testlib interactor. It is intentionally C++-only and header-only. Do not create a Python harness for the ordinary local judging workflow.

This skill owns concrete Testlib APIs, streams, verdicts, command lines, and templates. When evaluator-facing evidence has triggered independent contract or oracle work, use checker and local evaluation to choose checker/scorer/interactor architecture, reconstruct legality and objectives, and validate the evaluator itself. Do not enter that route merely because a previously specified generator or validator is being implemented.

Start Here

1. Read the testlib usage guide before writing checker, validator, generator, or interactor code.
2. Copy the bundled testlib.h into the working directory beside the C++ sources, or add its directory to the compiler include path.
3. Put #include "testlib.h" before other includes.
4. Select exactly one registration function for each executable.
5. For ordinary offline judging, use the minimal commands below rather than building a separate evaluation framework.

Choose the Executable Role

Generate Data as Reproducible Code

Testlib includes input-generation support; it is not limited to checkers and validators. When a local problem-finding campaign needs randomized, batch, or maximum-scale data, implement a problem-specific generator.cpp instead of hand-authoring large inputs or copying many variants. Parameterize the relevant case family, size, density/bias, structure, and seed tag with opt; use rnd for all randomness. The full command line determines Testlib's deterministic seed, so preserve that command exactly.

Define the coverage families and their expected oracle, invariant, or failure target before generating volume. Use one deterministic generator invocation per case, or a small coded batch driver that records every invocation in a manifest. Validate every intended-valid generated case with an independent validator before running the solution, and retain the generator command, input hash, validator result, and any failing seed and artifacts. Follow Checker and Local Evaluation for the complete coverage-to-retention workflow.

Critical Checker Contract

Always invoke a checker in this exact order:

checker <input-file> <contestant-output> <standard-answer>

After registration, the mapping is:

• inf: input file
• ouf: contestant or participant output
• ans: standard or jury answer

Never swap ouf and ans. Even a checker that does not use the standard answer still needs the third file argument; pass an empty placeholder file if necessary.

Minimal Local Judging Flow

Run these commands from a directory containing testlib.h, solution.cpp, checker.cpp, case.in, and case.ans:

g++ -std=c++17 -O2 solution.cpp -o solution
g++ -std=c++17 -O2 -I. checker.cpp -o checker

timeout 2s ./solution < case.in > case.out
solver_status=$?
if [ "$solver_status" -ne 0 ]; then
  echo "solution failed with status $solver_status" >&2
  exit "$solver_status"
fi
./checker case.in case.out case.ans
checker_status=$?
echo "$checker_status"
exit "$checker_status"

If the problem has validator.cpp, compile it and validate the input before running the solution:

g++ -std=c++17 -O2 -I. validator.cpp -o validator
./validator < case.in || exit 1

Capture solver_status before running the checker; a timeout, signal, or other nonzero solution exit is a solver failure and must not be relabeled as an output verdict. Capture checker_status immediately after the checker. For a verdict-only checker under the default local Testlib configuration, status 0 means accepted and nonzero means rejection or judge failure. A points checker using quitp instead returns a partial-points status (default 7) that a points-aware runner must parse separately. Preserve checker diagnostics and any reported points with the status.