batch-processing-clinical-text
适合脱敏大规模语料的可复现、可恢复处理管道。
需访问大量敏感临床文本,注意合规与隐私。
在本地对大量临床笔记运行 NER/去标识化,支持分片、断点续传和 JSONL 输出。
在本地批量跑 NER、PII 提取或去标识化,支持分片、断点续传和追加式 JSONL 输出,能处理中等到大量的临床笔记语料。适合需要把成千上万篇文档做脱敏或信息抽取、要求可重启和不会一次性占满内存的场景。特点是可恢复的批处理 runner:有进度回调、无 PHI 日志、文档分块和容错,结果可以边跑边流出。
▸ 展开 SKILL.md 英文原文
Run large-scale batch NER, PII extraction, or de-identification over many clinical notes on-device with OpenMed, with sharding, checkpointing, resumability, and append-only JSONL output. Use when the user needs to process a corpus or folder of notes, de-identify a dataset, run NER over thousands of documents, build a resumable batch pipeline, or stream results to JSONL without holding everything in memory. Covers process_batch / BatchProcessor / BatchItem / BatchResult, the operation= selector (analyze_text | extract_pii | deidentify), iter_process streaming, the PHI-safe on_progress callback, chunking long documents, and no-PHI logging. Produces a resumable batch runner over an OpenMed model.
帮我安装这个 skill:https://raw.githubusercontent.com/maziyarpanahi/openmed/master/skills/batch-processing-clinical-text/SKILL.mdcurl -fsSL "https://raw.githubusercontent.com/maziyarpanahi/openmed/master/skills/batch-processing-clinical-text/SKILL.md"# Batch processing clinical text `openmed.processing` runs OpenMed over many documents efficiently, with progress tracking, per-item error isolation, and streaming. It runs **fully on-device**: the corpus, the model, and the output never leave the host. This skill shows a **resumable** runner — sharded, checkpointed, append-only JSONL — that you can restart without reprocessing. ## When to use this skill For corpora, folders, or datasets — anything beyond a handful of notes. For a single note, just call `openmed.analyze_text` / `deidentify` directly (`extracting-clinical-entities`, `deidentifying-clinical-text`). For an always-on HTTP service, see `serving-openmed-rest-api`. ## Quick sta