Dotagents QA

dotagents QA

Prove the requested behavior in Docker without touching host agent homes, caches, plugin registries, or credentials. Treat Docker as the safety boundary and the changed or released behavior as the test plan.

1. Define the contract

Before commands, state:

• the exact subject: local checkout, packed local build, or published version;
• the commands and semantics at risk;
• default-global scope, explicit-project scope, or both;
• the harnesses involved;
• the fixture and evidence that constitute a pass.

For a published release, verify its registry metadata and install that exact version. Never silently test the local checkout instead. If you fix a discovered defect, report the release failure and packed-local-build pass separately.

Read the relevant references before acting:

• Docker, non-root setup, exact release installation, and baseline commands: references/docker-sandbox.md
• Ordinary install/sync behavior: references/core-agentic-qa.md
• Real plugins, full lifecycle, native clients, and user/global scope: references/release-plugin-matrix.md
• Plugin adapters and automated proof: references/plugin-runtime.md
• Harness details: Claude, Copilot, Codex, OpenCode, Pi, Cursor, Grok

Planning is part of acting: read the relevant references before proposing a command sequence, not only before executing it.

2. Establish isolation

Use the repo QA image. Rebuild it when the Dockerfile, pnpm version, or required current harness versions changed.

Run package and runtime work as a non-root user. Keep these inside Docker or disposable directories:

export HOME=/sandbox/home
export DOTAGENTS_STATE_DIR=/sandbox/state
export DOTAGENTS_HOME=/sandbox/user-agents
export COPILOT_HOME=/sandbox/copilot-home
export CODEX_HOME=/sandbox/codex-home
export CLAUDE_CONFIG_DIR=/sandbox/claude-home

Mount the host checkout read-only and copy it into the container without .git, node_modules, dist, coverage, or *.tsbuildinfo. Never point user-scope commands or client CLIs at the host's real home.

Do not forward auth for file, validation, marketplace, or resource-discovery checks. Copy credentials only for a separately authorized model-backed proof, scrub them afterward, and never retain them in /qa-out.

3. Run baseline validation

For a local plugin/runtime change, normally run:

pnpm install --frozen-lockfile
pnpm check
pnpm qa:example
pnpm qa:plugins

Add focused regression tests for every confirmed logic bug. A failing baseline is a finding; do not bypass it silently. If a check is irrelevant or unavailable, record why.

4. Exercise complete lifecycles

Use the CLI from a fresh fixture and inspect output plus files:

dotagents --project init --agents claude,codex,opencode,pi
dotagents --project add <source> [name]
dotagents --project list --json
dotagents --project doctor
dotagents --project install

Inspect agents.toml, agents.lock, canonical skills/plugins, ownership markers, marketplaces, manifests, harness projections, and warnings. Delete representative managed artifacts, run dotagents --project sync, and prove exact repair. Run dotagents --project remove <name> and prove canonical and generated cleanup.

Invoke every lifecycle command named by the contract with the intended scope. In particular, run install explicitly even though add also installs, and assert the config and lockfile paths directly rather than inferring them from list output.

For scope-selection changes, create both a project config and isolated global config, then prove unqualified commands mutate only global state from inside that configured repository. Prove --project mutates only project state, both --global and --user select the global paths, --global --user executes once, and either alias combined with --project fails before creating or changing files. Also cover --project init outside Git.

Test hook migration explicitly: create an executable, marker-delimited legacy post-merge hook containing unrelated lines around the bare install command; prove --project doctor diagnoses it and --project doctor --fix changes only the managed command to explicit project scope while preserving the unrelated content and executable mode.

Cover an unambiguous skill source as well as plugins so plugin-first discovery still falls back correctly. Use --all only with a controlled small catalog.

For global plugins, test the unqualified default plus both --global and legacy --user. Isolate HOME and DOTAGENTS_HOME; inspect global Claude, Codex, OpenCode, and Pi paths; repair with one spelling and remove with another.