Anonymizer

Before You Start

Do not explore the workspace first. The workflow's data-inspection step shows you what you need.

Goal

Anonymize a text dataset using NeMo Anonymizer in the way the user describes:

$ARGUMENTS

The output is a single runnable Python script that builds an AnonymizerConfig, previews results on a few rows, inspects failures and quality metrics, optionally scores output with LLM-as-judge evaluation (Replace and Rewrite modes), and (on user approval) runs the full pipeline. The script is the durable artifact — the user keeps it for re-runs, version control, and production.

Workflow

Read references/interactive.md and follow it. Anonymization is high-stakes, so there is no autopilot mode. Even when the user says "you decide" or "be opinionated", ask the minimum questions needed to choose risk_tolerance and phrase privacy_goal. The user must make those choices based on their regulatory and business context.

Rules

• Always preview before running the full pipeline. Preview is cheap; a full run can be expensive and slow.
• If result.failed_records is non-empty after preview, fix that before tweaking strategy. Dropped rows are a model/provider/infra problem (rate limits, auth, etc.), not a config problem. Strategy knobs won't help. See docs/troubleshooting.md "Did the run actually complete cleanly?" or the published troubleshooting guide.
• Ask the user which mode. Briefly describe both: Replace detects entities and replaces each in place (faster, cheaper, keeps shape); Rewrite transforms the full text to also remove inferable identifiers (more expensive, may restructure). Use the data shape as a hint — free-text with implicit identifiers (clinical notes, biographies, depositions) leans Rewrite; structured records / log lines lean Replace — but the user picks.
• For cross-record consistency (same value → same replacement everywhere), use Hash, not Substitute. Substitute is consistent within a row only.
• In Replace mode, default to Substitute if the user hasn't specified a strategy. It's the most general-purpose choice and matches the bulk of production usage.
• Annotate is for inspection, not production. Its output keeps the original entity text and is not privacy-safe. Use it during iteration to confirm detection is working, then switch.
• Evaluation is opt-in and runs as a separate step (Replace and Rewrite modes). After preview() / run(), call anonymizer.evaluate(result) to score the output with LLM-as-judge. Entity coverage always runs in both modes — it reports detection recall over the judge's unique candidate values (entity_coverage + missed_entities). On top of that: Replace Substitute adds three quality judges (type fidelity, relational consistency, attribute fidelity); Rewrite adds the holistic privacy/quality/style judge. Detection validity is opt-in via EvaluateConfig(compute_detection_validity=True) (off by default). Evaluation is diagnostic — it scores quality, it does not change the anonymized output.
• Always set AnonymizerInput.data_summary, even briefly. It is the single cheapest quality lever and it improves both detection and rewrite.
• Never claim privacy guarantees. Anonymizer is best-effort. Outputs may need human review depending on risk_tolerance. Tell the user this when you finalize.

Usage Tips and Common Pitfalls

• Detect.entity_labels=None (the default) is permissive — the augmenter LLM may invent labels not in DEFAULT_ENTITY_LABELS. Setting an explicit list switches to strict mode where only the listed labels are detected.
• Label terminology: A default label is present in DEFAULT_ENTITY_LABELS, while any label not present in this list is called a non-default label. An explicit label set is supplied through entity_labels and may contain either kind.
• Detect.entity_label_examples provides configured positive examples for detection. For a default label, configured examples are appended to its built-in examples. Every non-default label referenced by entity_label_examples must also appear in the explicit entity_labels set. To keep every default while adding one, use entity_labels=[*DEFAULT_ENTITY_LABELS, "clinical_facility"]. Configured examples are soft detection guidance: they do not limit detection to the listed value formats, and they are not passed to substitution or evaluation. Keep configured examples concise and lists short to limit prompt growth, token cost, latency, and context-window pressure. Use synthetic values because configured examples are included in prompts, exported builders, provider requests, and explicitly enabled raw message traces.
• Detect.excluded_entity_labels removes specific label types from the active detection scope and final results. Use it when a label type is systematically noisy for your data or should never be anonymized (e.g. Detect(excluded_entity_labels=["occupation", "gender"])). Exclusions take precedence over labels and examples; configured examples for excluded labels are ignored with a warning. If exclusions empty the default or explicit label set, Detect raises a ValueError.
• GLiNER is zero-shot — entity labels are natural-language concept names (e.g. "clinical_facility", "internal_project_codename"), not codes or enum values. Any concept you can name in English is a label GLiNER can detect.
• Rewrite.instructions is a dead field today — it exists on the model but the rewrite engine never reads it. Do not use it. Put rewriter guidance in privacy_goal.protect / privacy_goal.preserve instead.
• risk_tolerance only applies to Rewrite mode, not Replace.
• PrivacyGoal.protect and .preserve must each be 10–1000 chars and at least 3 words. Be specific (categories, named identifiers, structural facets); avoid generic phrasing like "preserve meaning".
• Validator pool is the only model role with built-in load-spreading. Set entity_validator: [a, b, c] in models.yaml if rate limits drop rows. Other roles (rewriter, evaluator, etc.) are single-alias.
• Self-hosted GLiNER2: When detection must stay local (PHI on-prem, air-gapped, latency), run the reference server from a source checkout with python tools/serve_gliner.py. The server is not installed by pip install nemo-anonymizer. Add a provider with endpoint: http://localhost:8001/v1, then route entity_detector through a gliner-pii-detector alias whose model is fastino/gliner2-privacy-filter-PII-multi, whose provider points to that endpoint (the bundled name is local-gliner2), and whose skip_health_check is true. Match any custom --port or --host in the provider endpoint. model_configs is a complete model pool, not an overlay. Copy src/anonymizer/config/default_model_configs/models.yaml and change only the provider name or endpoint as needed, keeping the LLM entries. See docs/concepts/self-hosting-gliner.md or the published self-hosting guide.
• The evaluation judges use their own model roles (entity_coverage_judge, detection_validity_judge, replace_type_fidelity_judge, replace_relational_consistency_judge, replace_attribute_fidelity_judge, rewrite_judge), configured in the evaluate section of models.yaml. They are not consumed by preview() / run(), so a config that anonymizes fine can still fail validation at evaluate() if those roles are unset. Defaults ship in src/anonymizer/config/default_model_configs/evaluate.yaml (entity_coverage_judge defaults to nemotron-super).
• Verdict columns are null when the judge was unavailable — None means "unscored", never a pass. entity_coverage is a 0–1 float (1.0 = no missed candidate values or no PII found) or None; missed_entities lists unique candidate values the anonymizer failed to detect. Replace verdict columns (type_fidelity_valid, etc.) are True / False / None. Rewrite detection_valid is a 0–1 float fraction (or None if unscored). Inspect verdicts per record with evaluated.display_record(i).
• EvaluateConfig has one knob today: compute_detection_validity (default False). Plain anonymizer.evaluate(result) runs entity coverage + the mode's quality judges; pass EvaluateConfig(compute_detection_validity=True) only to additionally score detection validity (an internal-facing tag-precision metric).