Phi Prompt Guard

Source: https://github.com/aipoch/medical-research-skills Contributed by: @ndu-bioinfo

PHI Prompt Guard

A behavioral skill that instructs the agent to refuse, redact, or redirect when the live prompt — or an action the agent is about to take — would push more Protected Health Information (PHI) into the LLM context window. Because this is a pure in-context behavioral skill with no pre-submit filter or middleware, it cannot literally prevent PHI that a user has already pasted from reaching the model; what it can do is reduce further PHI propagation after detection, avoid agent-initiated actions that would pull additional PHI in, and steer the user toward de-identified or synthetic inputs.

Intended for contexts where the model provider has not been approved to receive PHI under a Business Associate Agreement (BAA) or equivalent organizational authorization — any data placed in the prompt is sent to a third-party API outside the organization's control and may be cached or logged depending on vendor terms.

When to Use

• The user pastes (or is about to paste) clinical, patient, or specimen data into the conversation.
• The agent is about to run a database client (psql, mysql, mongo, duckdb, sqlite3, bq, snowsql, redis-cli, clickhouse-client, cqlsh) or a dump tool (pg_dump, mysqldump, mongodump) against an environment that may contain PHI.
• The agent is about to read a file whose contents may contain identifiers (lab reports, EHR exports, accession-keyed CSVs).
• The user pastes a clinical document by type — H&P, SOAP note, discharge summary, progress note, op note, path report, radiology report, HL7 v2 message, FHIR resource JSON, CCD/CDA — these are PHI-by-construction even when no trigger keyword is present.
• The user attaches an image or screenshot of an EHR, lab portal, chart, or clinical document. Multimodal models will OCR identifiers off the image — treat the attachment as a PHI paste.
• Discussion involves keywords such as: patient, clinical, accession, phi, hipaa, mrn, medical record, health plan, social security.
• The environment is unknown — assume production with PHI by default.

Workflow

For each user prompt:

1. Assess whether the prompt — or the action the agent is about to take in response — could expose PHI to the LLM context.
2. Proceed normally if no PHI risk is present.
3. If [PHI-OK] is present, treat it as a user attestation of synthetic / test data and proceed (see The [PHI-OK] Attestation below). Override only when the surrounding context still strongly resembles real, operational PHI.
4. If PHI is present without [PHI-OK], do not echo or summarize it. Tell the user which identifier category was detected and offer two paths: redact and re-submit, or re-submit with [PHI-OK] if the data is synthetic.
5. If the agent's intended action would pull PHI in (database client, dump tool, file read against a possibly-clinical source), generate the command for the user to run locally rather than executing it.
6. Offer a safer alternative where possible: schema-only inspection, aggregate query, a generic clinical answer using year-level dates, or a de-identified template the user can fill in locally.

Key Features

• Runtime PHI detection in the live prompt — covers obfuscated dates, cross-turn identifiers, foreign formats, indirect identifiers, and prose-form PHI.
• Action-level guardrails for database clients, dump tools, and file reads that could stream PHI into context; instructs the agent to generate the command for the user to run locally rather than executing it.
• Full HIPAA Safe Harbor 18-identifier taxonomy plus the 90+ age aggregation rule.
• [PHI-OK] attestation protocol that lets users work freely with synthetic/test data without false refusals or lecturing.
• Concrete decision rules: "schema is safe, data is not"; "error messages are usually safe"; "unknown environment ⇒ assume production with PHI"; "generate, don't execute".
• Pure in-context behavioral skill — no scripts, no external services, portable across agent frameworks.

Dependencies

• None (documentation-only behavioral skill).

Example Usage

Scenario 1 — User pastes a clinical row

"Why won't this row parse? Jane Doe, MRN 0044812, DOB 1985-03-15, dx SMA" (synthetic — illustrative only)

Correct agent behavior:

1. Detect category: name + MRN + DOB.
2. Do not echo the PHI.
3. Reply: "That row contains three HIPAA identifiers (name, MRN, DOB). Re-submit with the identifier columns redacted, or append [PHI-OK] if these are synthetic fixtures."

Scenario 2 — User asks for a SQL query against a clinical DB

"Pull the last 10 lab results from clinical.results so I can debug the parser."

Correct agent behavior:

1. Recognize psql/bq/etc. would stream PHI into context.
2. Generate the SQL but do not execute: SELECT * FROM clinical.results ORDER BY collected_at DESC LIMIT 10;
3. Instruct: "Run this in your own terminal and paste only de-identified columns or the parser error message back here."

Scenario 3 — Synthetic test fixture with [PHI-OK]

"Compute age-at-diagnosis: DOB 03/15/1985, dx 2020-06-01 [PHI-OK]"

Correct agent behavior: answer directly — Age at diagnosis = 35. No refusal, no redaction, no lecture.

Scenario 4 — [PHI-OK] misused on an EHR-shaped paste (override)

"Patient Jonathan A. Smith, DOB 1962-08-22, MRN 88440213, admitted 2024-11-04 to Memorial Hospital, NYC, for stage IV pancreatic adenocarcinoma. Current meds: gemcitabine + nab-paclitaxel. ECOG 2. Help me write the discharge summary. [PHI-OK]"

Correct agent behavior: override the attestation. The paste clusters multiple operational-PHI signals (real-looking full name, named hospital, exact admission date, current regimen, plausible MRN format) into something that reads as a real chart rather than a fixture. Reply: "This reads as a real chart despite [PHI-OK]. Confirm it is synthetic and I'll proceed, or re-submit with name / MRN / hospital / exact dates removed and I'll draft a discharge-summary template you can fill in locally."

Implementation Details

• Pure in-context behavioral skill — no scripts, no external services, no persisted state.
• Inputs: the live prompt plus any tool output the agent is considering.
• Outputs: (a) a refusal-with-redirect when PHI is detected, (b) a generated-but-unexecuted command when an action would pull PHI in, or (c) a normal task response when [PHI-OK] is present or no PHI is present.
• Each prompt is evaluated independently, but when the current prompt completes an identifier started in an earlier turn (e.g., last name now + first name two turns ago, or MRN tail now + MRN head earlier), the assembled identifier still counts as PHI and the current prompt must be treated as a PHI prompt.

Instructions

Your Responsibilities (the agent)

1. Detect PHI in the live prompt, including obfuscated dates, cross-turn identifiers (e.g., last name now + first name two turns ago), foreign date / phone / ID formats, indirect identifiers, and prose-form PHI like "the patient born in March 1985 with SMA…". Do not process, repeat, summarize, or store the PHI. Warn the user and offer two paths: redact, or re-submit with [PHI-OK] if it is synthetic.
2. Avoid pulling additional PHI in via actions. Before running a database client, dump tool, or file read whose output may contain PHI, stop and generate the command for the user to run in their own terminal rather than executing it yourself.
3. Honor the [PHI-OK] attestation by default. When present in the prompt, treat identifier-looking values as synthetic / test data and proceed with the task without redacting or appending unsolicited HIPAA commentary. Override only when the surrounding context still strongly resembles real, operational PHI — see The [PHI-OK] Attestation below for the judgment heuristics.

The 18 HIPAA Identifiers (keep out of the LLM context whenever possible)

1. Names
2. Geographic data smaller than state (street, city, county, ZIP — ZIP3 only for large areas)
3. Dates (except year) tied to an individual — DOB, admission, discharge, death
4. Phone numbers
5. Fax numbers
6. Email addresses
7. Social Security numbers
8. Medical record numbers (MRN)
9. Health plan beneficiary numbers
10. Account numbers
11. Certificate / license numbers
12. Vehicle identifiers and serial numbers (including license plates)
13. Device identifiers and serial numbers
14. Web URLs
15. IP addresses
16. Biometric identifiers (fingerprints, voiceprints)
17. Full-face photographs and comparable images
18. Any other unique identifying number, characteristic, or code (accession numbers, specimen IDs)

Also PHI under Safe Harbor: ages over 89, and any dates or date elements indicative of such an age (must be aggregated as 90+).

Re-identification risk (not literally one of the 18 Safe Harbor identifiers, but treat with the same caution): combinations of otherwise-non-PHI attributes — e.g., rare disease + small geography + age, or rare disease + sex + procedure date — that can uniquely identify an individual even after the 18 direct identifiers are removed. Small-cell aggregates count too: a count of 1 or 2 in a county × diagnosis × age-band cell is effectively an identifier; suppress or coarsen cells below k = 5 before they enter context.