Uat Testing

UAT Testing

End-to-end UAT workflow: analyze → generate test cases → set up environment → execute → report.

Workflow Overview

Phase 1: Discovery
  ├─ Read spec/requirements (if provided)
  ├─ Analyze git branch diff against base
  └─ Identify what was built/changed

Phase 2: Environment Setup
  ├─ Determine target: local or production/staging?
  ├─ Local path:
  │   ├─ Check for .env.local → ask user if missing
  │   └─ Start dev server (npm run dev or equivalent)
  ├─ Prod/staging path:
  │   └─ Get application URL, verify reachable
  └─ Ask user for test account credentials (both paths)

Phase 3: Test Case Generation
  ├─ Write uat-test-cases.md (see references/test-case-template.md)
  ├─ Cover: happy path, errors, persistence, auth, responsive
  └─ Present to user for review before execution

Phase 4: Test Execution
  ├─ Start dev server (framework-appropriate command)
  ├─ Authenticate via the env-var auth script (never type passwords)
  ├─ Execute test cases with Playwright MCP tools
  ├─ Take screenshots as evidence
  └─ If a test fails → document the failure, continue testing

Phase 5: Reporting
  ├─ Write uat-results.md (see references/results-template.md)
  ├─ Include: summary table, detailed results, screenshots, fixes
  └─ Present overall PASS/FAIL verdict

Phase 1: Discovery

Finding the Spec

Look for feature context in this priority order:

1. User-provided spec: If user gives a path to a spec, requirements doc, or thread — read it first
2. Spec directory: Check specs/[branch-name]/spec.md or specs/[feature-id]/spec.md
3. Tasks/todo: Check tasks/todo.md or similar for what was planned
4. PR description: If a PR exists, read it via gh pr view

Analyzing Branch Changes

# What branch are we on?
git branch --show-current

# What changed vs base branch?
git log main..HEAD --oneline
git diff main..HEAD --stat

# What files were modified?
git diff main..HEAD --name-only

Read the changed files to understand what was built. Focus on:

• New components or pages (user-facing features)
• Modified API routes or backend logic (behavior changes)
• Schema changes (data model additions)
• Config changes (new env vars needed)

Output

Summarize findings to the user:

• "This branch adds [feature] based on [spec]"
• "Key changes: [list of modified areas]"
• "I'll generate test cases covering [scope]"

Phase 2: Environment Setup

Determine Target Environment

Ask the user (or infer from context) whether this is local or production testing:

Are we testing locally (I'll start the dev server) or against a deployed environment (provide the URL)?

Local Environment Path

Check Environment Variables

# Check if env file exists
ls -la .env.local .env 2>/dev/null

If .env.local does not exist, ask the user:

I don't see a .env.local file. Do you have an environment file or credentials I should use? Common needs:

• Database connection (e.g., Convex URL/deploy key)
• Auth provider keys (e.g., Clerk publishable/secret key)
• API keys (e.g., AI model keys, external service keys)

Please provide the file path or paste the required variables.

Start Dev Server

Detect the framework and start the dev server:

If the project needs multiple services (e.g., Next.js + Convex), start all of them.

Verify the server is reachable before proceeding:

curl -s -o /dev/null -w "%{http_code}" http://localhost:3000

Production / Staging Environment Path

No server startup or env file needed. Ask the user for:

Please provide:

1. Application URL (e.g., https://app.example.com)
2. Any test data considerations (is there a staging org/business to use?)

Verify the URL is reachable:

curl -s -o /dev/null -w "%{http_code}" https://app.example.com

Request Test Account (Both Environments)

Authentication uses the env-var auth script, never interactive password entry (see Phase 4 and references/agent-guide.md → Auth Handling). So ask the user to put the credentials in a gitignored env file, not to paste the password into chat:

For UAT I authenticate with a small script that reads the test creds from a gitignored env file — I never type your password into a login field. Please either point me at an existing env file (e.g. .env.local) or create .env.uat (gitignored) with:

UAT_BASE_URL=https://app.example.com   # or http://localhost:3000
UAT_USER_EMAIL=test@example.com
UAT_USER_PASSWORD=...

Also tell me: any 2FA/MFA steps, and which business/org/team to use after login (if applicable). Don't paste the password in chat — the env file keeps it out of the conversation.

Phase 3: Test Case Generation

Read references/test-case-template.md for the full template format.

Generation Rules

1. Map spec to test cases: Each user story or requirement gets at least one test case
2. Prioritize: Critical (P1) = core happy path; High (P2) = error handling; Medium (P3) = edge cases
3. Be specific: Each step should be a concrete action ("Type 'hello' in the input field"), not vague ("Test the input")
4. Include expected data: Use actual values from the seeded test data or spec examples
5. Cover persistence: Always include a "reload page and verify" test case for stateful features
6. Cover auth boundaries: If the feature has role-based access, test with different roles

Writing the File

Write uat-test-cases.md to the spec directory (e.g., specs/[feature-id]/uat-test-cases.md) or to the project root if no spec directory exists.

Present the test cases to the user for review before executing. Ask:

I've generated [N] test cases in [file path]. Review them and let me know:

• Any test cases to add or remove?
• Any priority adjustments?
• Ready to proceed with execution?