Test Suite Prioritizer

# Recent commit changes
git diff --name-only HEAD~1 HEAD

# Uncommitted changes
git diff --name-only

# Changes in current branch vs main
git diff --name-only main...HEAD

# Files in last N commits
git diff --name-only HEAD~N HEAD

src/auth/login.py
src/payment/processor.py
src/models/user.py
tests/test_auth.py

Changed: src/auth/login.py
Covered by:
  - tests/test_auth.py::test_login_success
  - tests/test_auth.py::test_login_failure
  - tests/integration/test_user_flow.py::test_complete_registration
  - tests/e2e/test_authentication_flow.py::test_full_auth_cycle

Changed: src/payment/processor.py
Covered by:
  - tests/test_payment.py::test_process_payment
  - tests/test_payment.py::test_refund
  - tests/integration/test_checkout.py::test_complete_purchase

Priority Score = Base Score × Change Impact Factor

Base Score factors:
- Direct coverage of changed file: +10 points
- Indirect coverage (imports changed file): +5 points
- Integration test covering changed area: +3 points
- Test in same module: +2 points

Change Impact Factor (multiply):
- Critical path code (auth, payments): ×1.5
- Recently failed test: ×1.3
- High historical failure rate: ×1.2
- Core business logic: ×1.2
- UI/cosmetic changes: ×0.8

# Example: test_login_success
base_score = 10  # Directly tests login.py (changed)
impact_factor = 1.5  # Auth is critical path
priority_score = 10 × 1.5 = 15

# Example: test_user_flow
base_score = 5 + 3  # Imports login.py + integration test
impact_factor = 1.2  # Core business logic
priority_score = 8 × 1.2 = 9.6

# Example: test_ui_styling
base_score = 2  # Same module as changed file
impact_factor = 0.8  # UI/cosmetic
priority_score = 2 × 0.8 = 1.6

# Test Execution Priority List

## Summary
- Total tests analyzed: 45
- High priority (score ≥ 10): 8 tests
- Medium priority (score 5-9): 15 tests
- Low priority (score < 5): 22 tests
- Estimated time for high priority tests: ~2 minutes

## Recommended Execution Order

### Priority 1: Critical Tests (Run First)

1. **tests/test_auth.py::test_login_success** (Score: 15.0)
   - Directly tests modified file: src/auth/login.py
   - Critical path: Authentication
   - Reason: Core security functionality changed

2. **tests/test_auth.py::test_login_failure** (Score: 15.0)
   - Directly tests modified file: src/auth/login.py
   - Critical path: Authentication
   - Reason: Error handling for authentication changed

3. **tests/test_payment.py::test_process_payment** (Score: 15.0)
   - Directly tests modified file: src/payment/processor.py
   - Critical path: Payment processing
   - Reason: Payment logic modified, high business impact

### Priority 2: High Impact Tests

4. **tests/integration/test_user_flow.py::test_complete_registration** (Score: 9.6)
   - Indirect coverage: Imports src/auth/login.py
   - Integration test: End-to-end user flow
   - Reason: Validates auth changes in realistic scenario

5. **tests/integration/test_checkout.py::test_complete_purchase** (Score: 9.6)
   - Indirect coverage: Imports src/payment/processor.py
   - Integration test: Full checkout flow
   - Reason: Validates payment changes with real workflow

6. **tests/test_models.py::test_user_model** (Score: 7.0)
   - Related file: src/models/user.py modified
   - Dependency: Auth and payment depend on user model
   - Reason: Foundation for changed functionality

### Priority 3: Supporting Tests

7. **tests/test_validators.py::test_email_validation** (Score: 4.0)
   - Indirect dependency: Used by auth module
   - Reason: Input validation for authentication

8. **tests/e2e/test_authentication_flow.py::test_full_auth_cycle** (Score: 3.9)
   - E2E coverage: Complete authentication flow
   - Reason: Comprehensive validation but slow to execute

[... remaining tests with lower scores ...]

## Quick Commands

Run high priority tests only (8 tests, ~2 min):
```bash
pytest tests/test_auth.py::test_login_success \
       tests/test_auth.py::test_login_failure \
       tests/test_payment.py::test_process_payment \
       tests/integration/test_user_flow.py::test_complete_registration \
       tests/integration/test_checkout.py::test_complete_purchase \
       tests/test_models.py::test_user_model \
       tests/test_validators.py::test_email_validation \
       tests/e2e/test_authentication_flow.py::test_full_auth_cycle

pytest tests/test_auth.py tests/test_payment.py::test_process_payment

### Step 5: Handle Special Cases

Adjust prioritization for specific scenarios.

**Time-Constrained Testing:**

```markdown
## 1-Minute Quick Check (Top 3 Critical Tests)

pytest tests/test_auth.py::test_login_success \
       tests/test_auth.py::test_login_failure \
       tests/test_payment.py::test_process_payment

These cover the most critical changed functionality.

## Post-Refactor Validation

Changed file: src/utils/helpers.py (used by 25 modules)

Priority:
1. All tests directly importing helpers.py (18 tests)
2. Integration tests using helper functions (8 tests)
3. Tests in dependent modules (25 tests)

Reason: Wide-reaching refactor requires comprehensive validation.

## Note: Flaky Tests Detected

The following tests have high failure rates but low priority scores:
- tests/test_cache.py::test_concurrent_access (30% failure rate)
- tests/test_api.py::test_rate_limiting (25% failure rate)

Recommendation:
- Fix flaky tests before relying on prioritization
- Consider quarantining or mocking timing-dependent code

# .github/workflows/ci.yml example
jobs:
  quick-tests:
    name: Critical Tests (Fast Feedback)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run critical tests
        run: |
          # Run tests with score ≥ 15 first
          pytest -v tests/test_auth.py tests/test_payment.py

  full-tests:
    name: Complete Test Suite
    runs-on: ubuntu-latest
    needs: quick-tests  # Only run if critical tests pass
    steps:
      - uses: actions/checkout@v2
      - name: Run all tests
        run: pytest

# Pre-commit hook (.git/hooks/pre-commit)
#!/bin/bash

# Get changed files
changed_files=$(git diff --cached --name-only)

# Run prioritized tests for changed files
if echo "$changed_files" | grep -q "src/auth/"; then
    pytest tests/test_auth.py || exit 1
fi

if echo "$changed_files" | grep -q "src/payment/"; then
    pytest tests/test_payment.py || exit 1
fi

# pytest conftest.py - custom marker
import pytest

def pytest_collection_modifyitems(config, items):
    # Mark tests based on priority
    for item in items:
        if "test_auth" in item.nodeid or "test_payment" in item.nodeid:
            item.add_marker(pytest.mark.critical)

# Run only critical tests
# pytest -m critical

# Example: Analyze test failure history from CI
# tests with >10% historical failure rate get boosted priority

Historical Failures:
- test_concurrent_access: 12 failures / 100 runs = 12% → Priority boost ×1.2
- test_database_connection: 5 failures / 100 runs = 5% → No boost