Aif QA Check

QA Check — Execute Test Cases

Runs the test-cases.md artifact produced by /aif-qa and records execution status in qa-check.md. In agent mode, it also maintains reusable cross-QA execution memory under the QA root:

• agent-context.md — curated, current, reusable non-sensitive setup facts for future automated QA runs
• agent-history.md — append-only reusable learnings extracted from prior runs, not a per-run audit log

The skill is stack-free and agent-free: it does not assume a framework, package manager, browser tool name, or agent runtime. In agent mode, it uses the appropriate available execution surface for each case: live browser automation, CLI commands, project test runners, HTTP/API calls, database-safe read checks, file/document inspection, or other non-destructive automated verification.

Modes

If no mode is provided, ask the user which mode to run.

Workflow

Step 0: Load Config

FIRST: Read .ai-factory/config.yaml if it exists to resolve:

• Paths: paths.description, paths.architecture, paths.qa (default: .ai-factory/qa)
• Language:
  • language.ui for AskUserQuestion prompts, progress messages, summaries, and next-step guidance
  • language.artifacts for the persisted qa-check.md artifact
  • language.technical_terms for human-readable technical terminology style in the artifact
  • If language.artifacts is missing, use language.ui
  • If both are missing, use en
• Git: git.enabled for branch resolution

If config.yaml does not exist, use defaults:

• DESCRIPTION.md: .ai-factory/DESCRIPTION.md
• ARCHITECTURE.md: .ai-factory/ARCHITECTURE.md
• QA artifacts: .ai-factory/qa/
• ui_language: en
• artifact_language: en
• technical_terms_policy: keep
• Git enabled: true

Store:

• ui_language = language.ui || "en"
• artifact_language = language.artifacts || language.ui || "en"
• technical_terms_policy = language.technical_terms || "keep"
• git_enabled = git.enabled when present, otherwise true
• qa_root = resolved paths.qa || ".ai-factory/qa/"
• qa_agent_context_path = <qa_root>/agent-context.md
• qa_agent_history_path = <qa_root>/agent-history.md

All AskUserQuestion prompts, user-visible explanations, per-case summaries, and final summaries MUST be written in ui_language.

The persisted qa-check.md artifact MUST be written in artifact_language.

Templates define structure, not language. Use the canonical English template in templates/QA-CHECK.md. If artifact_language is not en, translate headings, labels, status text, comments you author, and explanatory text to artifact_language before saving. Preserve checkbox syntax, test case IDs (TC-001), commands, paths, config keys, URLs, selectors, package names, API names, branch names, and raw error messages.

For artifact_language = ru, write human-readable prose, headings, statuses, summaries, and agent-authored comments in Russian. Preserve user wording except mandatory redaction of sensitive values before writing.

Apply technical_terms_policy while writing artifacts:

• keep — keep common technical terms such as browser, selector, viewport, endpoint, payload, regression, and fixture when clearer
• translate — translate human-readable technical terms where a natural target-language term exists
• mixed — translate ordinary prose terms while keeping code, infrastructure, and ecosystem terms unchanged

Step 0.1: Load Project Context

Read the resolved description path if it exists.

Read the resolved architecture path if it exists.

Read .ai-factory/skill-context/aif-qa-check/SKILL.md — MANDATORY if the file exists. Treat it as project-level overrides for this skill.

Read <paths.qa>/agent-context.md if it exists. Treat it as reusable, user-approved QA execution context for future automated runs: stable environment classes, canonical local/staging URLs, login route, test account role, allowed non-sensitive usernames, required seed data patterns, stable selectors, field identifiers, known redirects, startup prerequisites, safe command patterns, reusable test-filter conventions, environment notes, and service startup instructions.

Read <paths.qa>/agent-history.md if it exists. Treat it as append-only cross-QA memory from prior /aif-qa-check runs. Before asking the user or attempting automated execution, search it for reusable prior answers to the same blocker so the skill does not repeat avoidable mistakes.

agent-context.md and agent-history.md are global to all QA sets under paths.qa. They MUST NOT become logs for a specific QA plan. Do not write branch names, branch slugs, QA target paths, current artifact_dir, TC-* mappings, per-run summary counts, exhaustive command transcripts, assertion counts from a single run, one-off file checks, or plan-specific labels such as a numbered QA folder name. Write those details only to <paths.qa>/<branch-slug>/qa-check.md.

Only promote information into agent-context.md or agent-history.md when it is likely to help future unrelated QA runs. Prefer stable patterns over concrete one-off run facts. Example: Laravel backend checks use php artisan test --filter=<TestClass> is reusable; 05-profile-formula-recommendations ran ProfileReaderTest with 4 tests / 13 assertions is run-specific and belongs only in qa-check.md.

Never write global negative capability or scope claims unless they are true for the whole project. Do not write statements like browser automation is not required, browser URL is not used, all QA is backend/CLI, or these cases are backend-test/cli/file-docs to agent-context.md or agent-history.md when that conclusion came from one QA plan. Instead, keep it branch-specific in qa-check.md, or phrase a reusable conditional rule such as For backend-only QA cases, browser automation is not required; browser/UI cases still require Browser or Playwright MCP.

Never treat either file as a source of secrets. If either file contains an apparent password, token, cookie, authorization header, one-time code, or other secret, ignore that value for execution and redact it the next time the file is rewritten or appended to.

Step 0.2: Parse Arguments and Resolve Branch

Parse $ARGUMENTS:

1. Detect mode: first word matching human or agent; remove it from arguments.
2. Detect branch: remaining text, if any, is the branch label.
3. If no mode was provided, ask in ui_language which mode to run:
   • Human-guided QA
   • Automated-agent QA

Resolve the working branch:

If git_enabled = false or the repository is not a git work tree:
  If branch was provided in arguments → use it as the resolved branch label
  Otherwise → set resolved_branch = "manual"
If git_enabled = true and the repository is a git work tree:
  If branch was provided in arguments → use it as the resolved branch
  Otherwise → run: git branch --show-current

Store:

• resolved_branch
• artifact_dir = <resolved paths.qa>/<branch-slug>
• test_cases_path = <artifact_dir>/test-cases.md
• qa_check_path = <artifact_dir>/qa-check.md

Compute branch-slug with the exact same algorithm as /aif-qa:

1. Replace every character not in [A-Za-z0-9._-] with -, collapse consecutive -, trim leading/trailing -, and use branch if empty. Then MUST truncate safe_slug to the first 40 ASCII characters. Because the normalized safe_slug alphabet is [A-Za-z0-9._-], byte length and character length are identical.
2. Run git hash-object --stdin <<< "<resolved_branch>" and take the first 8 hex characters as hash8.
3. Combine: branch-slug = "<safe_slug>-<hash8>".

Step 1: Verify QA Inputs

Check for <artifact_dir>/test-cases.md.

If it is missing, STOP and ask in ui_language whether to:

1. Run /aif-qa test-cases <resolved_branch> first
2. Cancel

Read test-cases.md. Extract test cases by IDs (TC-001, TC-002, etc.), titles, priority, type, execution surface when present, preconditions, steps, expected results, and test data. Preserve their order.

Compute source binding metadata before creating or modifying qa-check.md:

• source_digest — deterministic digest of the full test-cases.md content using git hash-object --no-filters <test_cases_path> when available, or git hash-object --stdin over the exact file content.
• case_digests — deterministic per-case digest for each extracted TC-NNN, computed from that case's canonical block including title, priority, type, preconditions, steps, expected result, and test data.
• tested_revision — when git_enabled = true and the repository is a git work tree, run git rev-parse HEAD and record the resolved commit SHA.
• worktree_digest — when git_enabled = true and the repository is a git work tree, record a deterministic digest of the current working tree state so dirty-tree QA cannot be reused after local changes without a commit.
• manual_build_id — when git_enabled = false or the repository is not a git work tree, ask the user for an explicit build/version identifier before creating or resuming results. Do not accept an empty identifier.

Canonicalize each per-case digest input exactly:

1. Extract the raw Markdown block from the line containing the case's TC-NNN identifier through the line before the next TC-NNN block or end of file.
2. Normalize line endings from CRLF or CR to LF.
3. Remove trailing spaces and tabs from each line.
4. Trim leading and trailing blank lines from the extracted block.
5. Preserve the original field order, bullet markers, indentation, internal blank lines, Markdown punctuation, and all remaining whitespace. Do not sort fields, collapse whitespace, or rewrite bullets.
6. Wrap the normalized block with raw block boundaries before hashing: BEGIN TC-NNN\n<normalized block>\nEND TC-NNN\n.
7. Hash the wrapped block with git hash-object --stdin when available, or another stable SHA-1/SHA-256 digest if git is unavailable.

Compute worktree_digest exactly when git is enabled and a git work tree exists:

1. Capture git status --porcelain=v1 --untracked-files=all.
2. Capture git diff --binary HEAD --.
3. Exclude qa_check_path from the status, diff, and untracked-file digest inputs so saving qa-check.md does not stale its own results. Do not exclude test_cases_path; source changes are also tracked by source_digest.
4. For each untracked file listed by porcelain status except qa_check_path, append UNTRACKED <path> <content-digest> where <content-digest> is git hash-object --no-filters <path> when the file is readable.
5. Normalize line endings in the combined input to LF and hash it with git hash-object --stdin.
6. If the filtered work tree input is clean, record the digest of the canonical string clean\n.

If mode = agent, perform Step 1.1 before creating or modifying qa-check.md. Existing qa-check.md may be inspected read-only during this gate.

If qa-check.md exists, read it and resume from existing statuses only after comparing stored binding metadata to the current binding metadata:

• If tested_revision changed, mark every prior result as Stale and unchecked, preserve prior comments/evidence as historical context, and require retest. Do not count stale pass/fail/block statuses as current.
• If worktree_digest changed, mark every prior result as Stale and unchecked, preserve prior comments/evidence as historical context, and require retest. Do not count stale pass/fail/block statuses as current.
• If manual_build_id changed, treat it the same as a tested revision change.
• If source_digest changed, compare case_digests. Preserve current status only for cases whose per-case digest is unchanged and whose tested revision/manual build id and worktree digest are unchanged.
• If an existing case's digest changed, mark that case Stale, unchecked, and require retest.
• If a new case appears, add it as unchecked Pending.
• If a prior case no longer exists in test-cases.md, keep its historical entry marked Stale or move it to an artifact-language "Stale / Removed Cases" section; never count it as current.
• Do not overwrite prior pass/fail comments unless the user explicitly chooses to retest that case.

If qa-check.md does not exist, create it from templates/QA-CHECK.md using the extracted test cases and source binding metadata. Every case starts unchecked and Pending.

Step 1.1: Agent Mode Capability and Safety Preflight

Agent mode is user-only (disable-model-invocation: true) because it can perform live browser actions, shell commands, local service calls, and other checks with meaningful side effects.

Before executing any case or writing qa-check.md in agent mode:

1. Detect available execution capabilities from the current runtime tools:
   • shell/CLI command execution
   • project test runners and build tools available through shell
   • live browser automation through in-app Browser or Playwright MCP
   • HTTP/API checks available through existing project tooling or safe shell commands
   • file, document, and repository inspection through Read/Grep/Glob
2. Classify each test case by the evidence surface it requires: browser-ui, cli, backend-test, api, file-docs, database-read, hybrid, human, or unknown. If the case has an explicit Execution surface: field, use it unless the case text clearly contradicts it; record any override in agent-history.md.
3. Choose the least invasive capability that can produce concrete evidence for that case. Browser automation is REQUIRED for browser/UI-observable cases and hybrid cases whose expected result depends on rendered web behavior. Do not substitute CLI commands, unit tests, static inspection, or backend checks for a browser/UI case; those may be recorded only as supporting evidence.
4. If a case requires browser automation, prefer the in-app Browser capability if available. If in-app Browser is not available, use Playwright MCP if available.
5. If a browser/UI case requires live browser automation and neither in-app Browser nor Playwright MCP is available, keep only that case unchecked, set status to Blocked, ask the user in ui_language to enable a live browser capability, and append this blocker to <paths.qa>/agent-history.md. Continue with other cases that can be verified through CLI, tests, API, or file/document checks.
6. Use <paths.qa>/agent-context.md, <paths.qa>/agent-history.md, test-cases.md, change-summary.md, project docs, current browser state, and repository scripts to determine the target URL, service startup commands, test commands, and environment.
7. For browser/UI cases that require authentication or specific user state, resolve a browser test identity before marking the case blocked:
   • First search agent-context.md, agent-history.md, test data, seeders/factories/fixtures, docs, local database-safe reads, and existing test users for a matching non-sensitive test identity.
   • If the environment is clearly local, test, or disposable development, and repository tooling provides a safe way to create or seed a synthetic test user/state, create the minimal disposable fixture needed for the case. Record the fixture identifier and setup command/evidence in qa-check.md.
   • If creating a fixture requires a password, role, tenant, workspace, feature flag, seed command, or any unclear setup decision, ask the user before blocking. Ask whether to provide an existing test account or authorize creating a disposable fixture.
   • If the user provides or authorizes disposable test credentials for a local or test environment, use them for this run. Store credentials in agent-context.md only if the user explicitly says they are reusable test-only credentials and safe to persist. Otherwise store only the username/role and where to retrieve the secret.
   • Never store production credentials, personal credentials, cookies, session tokens, one-time codes, or shared secrets in agent-context.md or agent-history.md.
8. If a user-suppliable prerequisite is missing, ask the user before blocking:
   • application URL or login URL
   • test account role or non-sensitive username
   • where to find credentials without exposing them in artifacts
   • authorization to create a disposable local/test browser fixture and the desired initial state
   • required startup state, seed data, feature flag, tenant, organization, or workspace
   • ambiguous login mechanism or authentication provider
   • stable selector, field identifier, or page route needed to continue
   • exact project command, test filter, service startup command, or safe API endpoint needed to continue
9. Save reusable non-sensitive answers to <paths.qa>/agent-context.md immediately only when the answer is general enough to help future QA sets. Append the redacted answer summary to <paths.qa>/agent-history.md only when it captures a reusable lesson; otherwise keep it in the branch-specific qa-check.md evidence/comment.
10. Determine and display the target environment, including URL when applicable and inferred environment class: local, development, staging, test, production, or unknown.
11. Do not execute browser actions, API writes, CLI commands with side effects, database writes, destructive commands, payment, permission, email, notification, data export, or other external-side-effect steps against production or unknown targets without explicit user authorization immediately before execution. Record the authorization decision and target class in qa-check.md. Append it to agent-history.md only when it creates a reusable cross-QA policy or recurring authorization lesson; do not persist broad production authorization for future runs.
12. Inspect each case and proposed command for destructive, irreversible, payment, permission, email, notification, data export, database mutation, service restart, deployment, filesystem deletion, or other external-side-effect risk. Require explicit per-case authorization before executing any such case.
13. Prefer read-only and test-scoped commands. Do not run destructive shell commands such as rm, git reset, git checkout --, database resets, migrations against shared environments, deploy commands, or cleanup commands that delete user data unless the user explicitly authorizes the exact action.
14. If authorization is denied or unclear, leave the case unchecked, set status to Blocked, and write the blocker to qa-check.md without executing the risky action. Append the decision to agent-history.md only when it is a reusable cross-QA lesson, not merely a case-specific denial.

Redaction is mandatory for agent comments/evidence and all human-entered comments/evidence:

• Redact credentials, cookies, authorization values, session tokens, API keys, bearer tokens, basic auth values, one-time codes, and personal secrets.
• Redact sensitive URL parameters such as token, access_token, refresh_token, id_token, code, secret, password, passwd, pwd, auth, key, api_key, session, sid, and jwt.
• Replace redacted values with [REDACTED] before writing comments or evidence to qa-check.md.
• Do not paste raw browser storage, cookies, request headers, authorization headers, or token-bearing URLs into the artifact.