Running Openmed Ondevice

What it does

When your agent uses it

• The user wants to deploy OpenMed at the edge
• Run NER/de-id on Apple Silicon
• Target iPhone/iPad/Mac
• Quantize a clinical model to int8/4-bit

Example prompts

• “/running-openmed-ondevice”

Requirements

• Python 3

Workflow steps

5 steps, taken from the first numbered list in SKILL.md.

1. 1Pick the backend for the target (table above).
2. 2Convert the HF/OpenMed model with the matching convert() /
3. 3Quantize if size/latency demands it (MLX 4/8-bit, CoreML int8, WebGPU
4. 4Run locally: MLX artifacts go straight through analyze_text /
5. 5Verify outputs against the full-precision model before shipping

What it can do on your machine

Read from SKILL.md and the folder at commit 34d7b8c. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Shell commands in SKILL.md call:

  • python
  • pip

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  Links to these hosts (documentation or services it may open):

  • github.com
  • apple.github.io
  • onnx.ai
  • onnxruntime.ai
  • huggingface.co

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Running Openmed Ondevice loads about 2k tokens when it runs. Until then it costs about 172 tokens; SKILL.md has 601 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~172

When it runs · the whole SKILL.md, loaded when a task matches

~2k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from maziyarpanahi/openmed at commit 34d7b8c, republished under its Apache-2.0 licence (© maziyarpanahi). 601 words, ~2,018 tokens.

name

running-openmed-ondevice

description

Run OpenMed models fully on-device with the MLX (Apple Silicon), CoreML (iOS/macOS), or ONNX/WebGPU (cross-platform/browser) backends, including convert-quantize-run workflows. Use when the user wants to deploy OpenMed at the edge, run NER/de-id on Apple Silicon, target iPhone/iPad/Mac, export to ONNX or WebGPU, quantize a clinical model to int8/4-bit, run with no network, or pick between MLX/CoreML/ONNX. Covers the mlx/coreml/onnx extras, the convert() functions and python -m convert CLIs, quantization, loading a local MLX artifact through analyze_text, OpenMedMLXLanguageModel/generate_text, and the on-device-only PHI guarantee (nothing leaves the host).

license

Apache-2.0

metadata.project

OpenMed

metadata.category

deployment-ops

metadata.pairs

adjacent

metadata.version

1.0

pip install "openmed[mlx]"

# Convert a HF token-classification model to an OpenMed MLX artifact, 8-bit:
python -m openmed.mlx.convert --model OpenMed/<some-ner-model> --output ./mlx_ner --quantize 8

import openmed

# Run NER/de-id through the normal API — pass the local artifact dir as model_name.
# The loader auto-detects the MLX backend from the artifact (or set backend explicitly).
result = openmed.analyze_text(
    "Patient received 75mg clopidogrel for NSTEMI.",
    model_name="./mlx_ner",          # local MLX artifact directory
    output_format="dict",
)

# Force MLX via config if you prefer to be explicit:
from openmed.core.config import OpenMedConfig
cfg = OpenMedConfig(backend="mlx")   # None=auto-detect, "mlx", or "hf"

from openmed.mlx.lm import generate_text, OpenMedMLXLanguageModel

text = generate_text(
    messages=[{"role": "user", "content": "Summarize: chest pain, troponin elevated."}],
    model_name="OpenMed/laneformer-2b-it-q4-mlx",   # resolves to a local MLX-LM artifact
    max_tokens=128,
)

llm = OpenMedMLXLanguageModel("OpenMed/laneformer-2b-it-q4-mlx")
out = llm.generate(prompt="...", max_tokens=64, temp=0.0)

pip install "openmed[coreml]"
python -m openmed.coreml.convert --model OpenMed/<some-ner-model> --output model.mlpackage --quantize int8

from openmed.coreml.convert import convert
convert(
    "OpenMed/<some-ner-model>",
    "model.mlpackage",
    compute_units="cpuAndNeuralEngine",   # "all" | "cpuAndNeuralEngine" | "cpuOnly"
    compute_precision="float16",          # float16 for Neural Engine, float32 for CPU
    quantize="int8",                      # emits an int8-palettized sibling .mlpackage
)

pip install "openmed[onnx]"
python -m openmed.onnx.convert --model OpenMed/<some-ner-model> --output ./onnx_out

from openmed.onnx.convert import convert
res = convert("OpenMed/<some-ner-model>", "./onnx_out", include_webgpu=True, opset=18)
# Emits model.onnx (fp32) and model.webgpu.onnx (fp16) + an export manifest.

© maziyarpanahi, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in skills/running-openmed-ondevice of maziyarpanahi/openmed.

Open the folder on GitHubat commit 34d7b8c