Kitaru Tests Release

Kitaru Tests, CI, and Release Workflow

Use this when adding, moving, or debugging tests beyond the basic commands and safety rules in tests/AGENTS.md, or when changing CI/release behavior.

Unit and Contract Tests

• Build small typed stubs or SimpleNamespace objects instead of booting unrelated runtime state.
• Assert one behavior or contract at a time.
• Include regression coverage for bug fixes.
• Keep tests independent of working-directory state unless the test creates that state explicitly.
• Avoid shared mutable module-level state and wall-clock ordering assumptions.

V2 has no primed_zenml fixture or test_phase* example suite. Do not copy those v1 patterns into new tests.

CLI Tests

• Install the surface with uv sync --extra cli --extra worker.
• Call src/kitaru/cli/app.py::main with an explicit argument list.
• Assert the returned integer exit code, such as main(["--help"]) == 0; successful calls do not raise SystemExit(0).
• Use capsys and assert stable structured or plain-text contracts.
• Prefer lightweight stubs for remote resources instead of starting the full server.
• Exercise offline help, version, schema, and scaffold commands without reading local configuration.
• Run just cli-artifact-smoke after optional-dependency, entrypoint, or packaging changes.

Keep CLI tests focused on argument parsing, command dispatch, output contracts, and the specific resource interaction under test.

Server and SDK Tests

Follow the four server-resource surfaces in tests/AGENTS.md: service tests, ASGI REST tests, shared repository contracts, and PostgreSQL end-to-end tests. Add SDK round-trip coverage under tests/client/ when a public resource changes.

Use the PostgreSQL-backed tests for transaction, locking, migration, or cross-request behavior that an in-memory fake cannot prove. Run docker compose up -d db before those tests and just migration-check after schema changes.

Task and Worker Tests

• Task subprocess contracts live under tests/task/.
• Worker lifecycle and handler contracts live under tests/worker/.
• Keep subprocess tests bounded and assert structured receipts, exit behavior, and redaction.
• Use the existing worker fakes rather than starting unrelated services.
• When changing dynamic task dependencies or adding a first-party Python distribution, compare the worker's package-age allowlist with every PyPI unit in release/release-units.toml, including non-default packages. Exercise an exact pin under an older exclude-newer cutoff with a candidate or published wheel, and cover the fallback when the installed uv lacks --exclude-newer-package.

Default Plugin Packages

• Read plugins/DEVELOPMENT.md for the package map, local candidate-image rehearsal, version preparation, dry-run dispatch, PyPI Trusted Publisher setup, publish workflow, and verification commands.
• Standalone adapter distributions also live under plugins/packages/, but agent projects install them directly. Keep default-catalog = false in the release inventory and keep them out of the server default catalog.
• Run just plugin-artifact-smoke after changing plugin package metadata, default definitions, requirement pins, or release installation paths.
• The smoke builds Kitaru and every selected plugin as wheels, installs them into a clean environment, loads each configured package entrypoint, and verifies idempotent default registration.
• CI runs plugin distributions as a package matrix. Keep the matrix aligned with plugins/packages/ and the choices in .github/workflows/release-plugins.yml.
• Plugin workflow dispatches are dry-runs. Package tags publish from reviewed commits reachable from develop or the unit's matching maintenance branch. Dependent plugins wait for core PyPI availability; other core jobs can continue.
• Kitaru release dry-runs build plugin-owned candidate images from plugins/candidate-wheels; production release Dockerfiles continue to install exact versions from PyPI.
• Commit plugins/candidate.Dockerfile and plugins/docker-compose.candidate.yml. Do not commit generated files under plugins/candidate-wheels/.
• Keep production release Dockerfiles unchanged when a plugin change only needs local candidate-wheel testing.
• Register a self-contained in-progress plugin with the CLI --script source. Use an exact package source when the test must cover wheel installation or package imports.

MCP Tests

• Synchronize with uv sync --frozen --extra mcp and run just test tests/mcp.
• Keep handler tests typed and bounded; use resource-shaped fake KitaruAPIClient objects unless a protocol or real-server contract requires deeper integration.
• Test capability filtering through MCPServer.list_tools(), not by calling decorated functions alone.
• Treat tests/mcp/snapshots/metrics.json and src/kitaru/mcp/registry.py as the tool-inventory authorities. Do not hardcode copied inventory counts in instructions.
• Run just mcp-schema-check after any input/output model, registry, annotation, description, or MCP SDK change. Snapshot changes require explicit MCP API review.
• Build the wheel and run just mcp-wheel-smoke after launcher, packaging, lifecycle, or optional-import changes.
• Preserve stable request-ID forwarding, mixed-version refusal, bounded preflight reads, and text/structured response parity where the existing contracts require them.

Bug Fix Workflow

Every bug fix should include a regression test that would have caught the original problem:

1. Write or update the test so it captures the broken behavior.
2. Run it and observe the expected failure when practical.
3. Make the code change.
4. Rerun the targeted test.
5. Rerun the broader relevant suite.

If code changes after a successful test run, run the affected tests again.

Feature Completion Checklist

When adding a new CLI command, MCP tool, SDK resource, task, or worker capability:

• Add or update focused tests for the changed surface.
• Update offline CLI registration metadata for CLI commands.
• Update examples/example-coverage.yaml and run just example-coverage-audit when examples are added, removed, renamed, or publicly documented.
• Review analytics coverage and add events only through the current v2 analytics paths.
• Run the relevant CLI artifact, MCP schema, MCP wheel, migration, OpenAPI, or package-build checks for the changed contract.