OpenMed hit 755 tokens per second redacting clinical PHI on a Mac. Local-first is finally fast enough.

755 tokens per second. On a Mac. While redacting clinical identifiers as the text streams by.

That is the number Maziyar Panahi posted for OpenMed privacy-filter v2 (Nemotron, MLX 8-bit): a 13,000-token file, 1,152 PII hits across 22 categories, nothing left the machine. OpenMed 1.5 ships batch and streaming PII redaction in nine languages, Apache 2.0 licensed, with millions of PyPI downloads in under a year.

I work with teams that cannot send patient or customer PHI to a third-party LLM API, no matter how good the model is. For years the tradeoff was: local inference for compliance, cloud inference for speed. This release chips at that gap.

What OpenMed actually provides

OpenMed is not one model. It is a local-first healthcare stack:

The marketing site at openmed.life positions against cloud APIs that bill per character and require sending clinical text to vendor regions. OpenMed downloads once and runs in your process: laptop, VPC, or air-gapped box.

The API surface (quick start)

from openmed import deidentify

result = deidentify(
    "Patient Pedro Almeida, MRN 88421, admitted to St. Mary's on 03/12/2024.",
    model_name="OpenMed/privacy-filter-nemotron-mlx-8bit",
    method="mask",
)
print(result)

Batch and streaming paths exist for document pipelines:

from openmed import BatchProcessor

processor = BatchProcessor(operation="deidentify", batch_size=16)
outputs = processor.run(documents)

The repo documents speedups (up to ~3.3x throughput on CPU batch vs one-doc-at-a-time on MLX paths). Your hardware and model choice will move those numbers.

Why 755 tok/s matters operationally

Cloud redaction APIs force a workflow break: export note, call API, wait, re-import sanitized text, hope spans are right. Latency and privacy review sit on every new feature.

On-device streaming at hundreds of tokens per second means:

• Inline redaction before text hits an LLM summarizer you do control
• Real-time dictation or scribe pipelines with a PHI gate in the middle
• Batch overnight jobs on a Mac Studio instead of a GPU cluster rental

That is the difference between "we have a compliance exception" and "we wired redaction into the product path."

What local does and does not solve

Panahi's demo closes the data transmission gap. It does not close governance.

LinkedIn comments on the launch were right to push back: "100% offline" removes vendor API risk, not HIPAA program risk. You still need:

• Role-based access on the device running the model
• Audit logs for who ran de-identification and on what charts
• Validation that span boundaries match your policy (especially merged entities)
• Clinical review of downstream model outputs, not just inputs

Local is the new perimeter. Perimeter security still exists.

On-device PHI redaction is necessary for many healthcare AI products. It is not sufficient for compliance sign-off.

OpenMed in a typical architecture

A pattern I would prototype for a clinic client:

Clinical note (EHR export or scribe)
        ↓
OpenMed deidentify() stream
        ↓
Sanitized text → internal LLM (summaries, coding suggestions)
        ↓
Human review gate → EHR write-back

No third-party API sees raw PHI. Summarization quality still depends on the LLM you choose behind the gate. Redaction quality depends on OpenMed's span detection. Test both on your note formats (discharge summaries look nothing like psychiatry progress notes).

Ecosystem context

OpenMed sits next to other "ship AI without exfiltrating data" moves: on-device embeddings, local voice agents, VPC-hosted models. The common buyer question is shifting from "can we use AI?" to "where does the data sit when inference runs?"

If you are evaluating build vs buy for clinical NLP,

OpenMed's PII notebook is a practical starting point. Run it on synthetic notes first. Never benchmark on real patient charts in a shared dev environment.

When I would recommend it

Good fit:

• Small clinic or startup with strict no-cloud-PHI policy
• Pre-processing layer before an internal or contract LLM
• Mobile or field apps that must redact on device

Pause and design harder:

• Multi-site health systems with existing Epic/Cerner integration contracts
• Workflows needing FDA-style validation evidence (OpenMed is tooling, not a cleared device)
• Languages or ID formats outside the supported PII catalog

If you want help sketching a local-first PHI gate before you bolt an LLM onto clinical workflows, book a free discovery call.