Test Tagging

Step 3: Classify each test method

For each test method without traits, analyze:

1. Method name -- names containing Invalid, Fail, Error, Throw, Reject, BadInput, Null, None, Nil, Negative, raises_, _throws_, _returns_error suggest negative
2. Assertion type -- Assert.ThrowsException / Assert.Throws / Should().Throw() / pytest.raises / expect(fn).toThrow / assertThrows / assert.Error(t, err) / expect { ... }.to raise_error / #[should_panic] / XCTAssertThrowsError / Should -Throw / EXPECT_THROW suggest negative
3. Input values -- null / None / nil / undefined, "", 0, -1, int.MaxValue / sys.maxsize / Number.MAX_SAFE_INTEGER / math.MaxInt64 / i32::MAX, empty collections suggest boundary
4. Setup complexity -- minimal setup with basic assertions suggests smoke; external dependencies (file/db/net/env) suggest integration
5. Comments and names -- references to issue numbers or "regression" / "bug" / "fix for #..." suggest regression
6. Timing assertions -- Stopwatch, BenchmarkDotNet, elapsed-time checks; pytest-benchmark fixtures; benchmark.js; JMH @Benchmark; go test -bench; criterion.rs; XCTMetric; Google Benchmark; kotlinx-benchmark suggest performance
7. Feature centrality -- tests on primary public API entry points or critical user workflows suggest critical-path
8. Security patterns -- validates auth, checks permissions, sanitizes input, tests for injection, handles tokens/secrets suggest security
9. Parallel/async constructs -- per-language concurrency primitives (see Trait Taxonomy table) suggest concurrency
10. Fault injection -- simulates failures, tests retries, timeouts, or circuit breakers suggest resilience
11. State mutation -- deletes external records, drops resources, modifies shared/global state suggest destructive
12. Full-stack flow -- test spans entry point through data layer to final response, covering a complete user scenario suggest end-to-end
13. Config/settings -- loads configuration, tests missing keys, validates options, checks environment variables suggest configuration
14. Known instability -- test has skip / ignore annotations with comments about flakiness, or names contain "flaky" / "intermittent" suggest flaky
15. Default -- if the test verifies a normal success path, tag positive

When in doubt between positive and negative, read the assertion: if it asserts success -> positive; if it asserts failure -> negative.

Step 4: Apply trait attributes (or report only)

If the loaded language extension declares auto-edit for the framework, add the appropriate attribute to each test method. Place trait attributes adjacent to the existing test attribute. Examples:

MSTest:

[TestMethod]
[TestCategory("negative")]
[TestCategory("boundary")]
public void Parse_NullInput_ThrowsArgumentNullException() { ... }

xUnit:

[Fact]
[Trait("Category", "positive")]
[Trait("Category", "critical-path")]
public void CreateOrder_ValidItems_ReturnsConfirmation() { ... }

NUnit:

[Test]
[Category("regression")]
[Category("negative")]
public void Calculate_OverflowInput_ReturnsError() // Fix for #1234
{ ... }

pytest:

@pytest.mark.negative
@pytest.mark.boundary
def test_parse_none_input_raises_value_error():
    ...

JUnit 5:

@Test
@Tag("positive")
@Tag("critical-path")
void createOrder_validItems_returnsConfirmation() { ... }

TestNG:

@Test(groups = {"negative", "boundary"})
public void parse_nullInput_throwsIllegalArgumentException() { ... }

RSpec:

it "rejects null input", :negative, :boundary do
  ...
end

Pester:

It 'Rejects null input' -Tag 'negative','boundary' {
    ...
}

Kotest:

@Tags(Negative, Boundary)
class ParserSpec : StringSpec({
    "rejects null input" { ... }
})

Swift Testing:

@Test(.tags(.negative, .boundary))
func parseNullInputThrows() throws { ... }

Catch2:

TEST_CASE("Parse null input throws", "[negative][boundary]") { ... }

If the loaded language extension declares report-only for the framework (Go standard testing, plain Jest/Vitest without convention, Rust without project-specific cfg, plain XCTest, plain GoogleTest, plain Mocha), do NOT modify source files. Instead emit a Markdown table mapping each test to its suggested tags, and recommend a project-wide convention the team can adopt (build tags, file suffix, describe-block prefix, GoogleTest filter prefix, test-plan grouping, etc.).

If the loaded language extension declares convention-based (e.g., Go //go:build integration, *_integration_test.go, GoogleTest INTEGRATION_* prefix), only emit canonical edits when the user has confirmed the project's convention. Otherwise treat as report-only.

Step 5: Generate trait summary

After tagging, produce a summary table:

## Trait Distribution

| Trait         | Count | % of Total |
|---------------|-------|------------|
| positive      |    42 |      53.8% |
| negative      |    22 |      28.2% |
| boundary      |     8 |      10.3% |
| critical-path |    12 |      15.4% |
| smoke         |     3 |       3.8% |
| regression    |     5 |       6.4% |
| integration   |     4 |       5.1% |
| end-to-end    |     2 |       2.6% |
| performance   |     1 |       1.3% |
| security      |     3 |       3.8% |
| concurrency   |     2 |       2.6% |
| resilience    |     1 |       1.3% |
| destructive   |     1 |       1.3% |
| configuration |     2 |       2.6% |
| flaky         |     1 |       1.3% |
| **Total tests** | **78** | -- |

Note: Percentages exceed 100% because tests can have multiple traits.

Include observations such as:

• Ratio of positive to negative tests
• Whether critical-path tests exist for key public APIs
• Any tests that could not be confidently classified (list them for manual review)

Validation

• Every test method has at least one trait classification (positive or negative at minimum) — in the report for report-only frameworks, or as an attribute for auto-edit frameworks
• No invented trait values outside the taxonomy table
• Existing trait attributes were preserved, not duplicated
• The trait summary table was generated
• For auto-edit frameworks, the project still builds / tests still discover after changes (dotnet build / pytest --collect-only / mvn test-compile / go vet ./... / cargo check --tests / npm run test:list / Invoke-Pester -PassThru -Skip / equivalent)
• For report-only frameworks, no source files were modified
• For convention-based frameworks, edits were applied ONLY when a project convention was confirmed

Common Pitfalls