Pathling YAML Exclusions

Pathling YAML test exclusions

Pathling runs two YAML conformance suites, each with its own exclusion baseline:

An exclusion is not a mute button. It is an assertion about how a case currently fails.

The baseline polices itself

Excluded cases are still executed. The runner asserts the case produces the recorded outcome, and only then reports it as skipped. If an excluded case starts passing, the build fails with:

Excluded test passed when expected outcome was error

That is the machine-checkable done-signal for feature work: implement a feature, and every exclusion it obsoletes turns the build red until it is cleaned up. You cannot silently leave a stale exclusion behind.

Prefer error or failure. An explicitly-null outcome opts the case out of verification entirely, which is how baselines rot. Use it only when running the case is itself the problem (a hang, an OOM), and say so in the comment.

UnsupportedFhirPathFeatureError is handled separately — such cases are skipped regardless of outcome.

Rule anatomy

excludeSet:
  - title: Global exclusions        # required
    comment: |                      # optional
      Why this block exists.
    exclude:
      - title: Unsupported resources # required
        type: feature                # convention only — see taxonomy below
        id: "#2418"                  # issue this is tracked under
        outcome: failure             # defaults to "error"
        comment: |                   # why, and what would resolve it
          ...
        expression:                  # matcher — see below
          - "^StructureDefinition"

Matchers

A rule may carry several matchers; they are OR-ed. A case is excluded by the first rule whose matchers match it.

Prefer any with a complete expression string for surgical exclusions — it is the easiest to verify and the hardest to over-match. Reach for expression regexes only when a genuine family of cases shares a shape, and remember they are unanchored: "toQuantity" also matches convertsToQuantity.

Fields that do not work

Two fields are parsed but never applied. Do not rely on them, and do not add new uses:

• glob on an exclude block — documented as scoping a block to certain test files, but never read. Every block applies to every case file. A rule you believe is scoped to one corpus file will silently mask matching cases everywhere.
• desc on a rule — declared as a matcher but never converted to a predicate. Use any, which already matches descriptions.

Separately, the system property au.csiro.pathling.test.yaml.exclusionsOnly is read and logged but never applied — it does not filter anything. Ignore it.

disabled: true on a rule, by contrast, does work: it suppresses the rule entirely, so the cases it was masking run unexcluded. See §3 below for using it, and the disabledExclusions property, to test whether a rule is still earning its place.

Type taxonomy

type is a free-form string with no validation. Current usage in fhirpath-js/config.yaml:

Use feature, not new-feature. When you touch a block containing new-feature, migrate it.

wontfix deserves scrutiny. It currently absorbs two different things: genuinely invalid test cases, and unimplemented capabilities mislabelled as invalid. For example a wontfix block titled "Parse error — not valid FHIRPath syntax" contains ('a'|'b'|'c').join(','), (1|2|3).sum() and (1|2|3).aggregate($this+$total, 0) — all valid FHIRPath, and all really feature gaps. If a wontfix entry you encounter is actually a capability gap, reclassify it and give it an issue id.

The sweep

Run this whenever a feature lands, and whenever the build reports an excluded test passing.

1. Find the rules in scope

Search by every handle the feature has — function name, operator, expression fragment, and the issue number:

cd fhirpath/src/test/resources
rg -n 'toQuantity' fhirpath-js/config.yaml fhirpath-ptl/config.yaml
rg -n '#2391' fhirpath-js/config.yaml fhirpath-ptl/config.yaml

An empty result is not evidence the feature is fully working. Most open FHIRPath issues own no exclusion at all — the corpus may simply not cover the feature. When nothing matches, say so explicitly in your report rather than concluding there was nothing to do.