batch-processing-clinical-text

仓库创建 2025年10月4日最近提交 3 小时前SkillHot 收录 3 小时前
▸ 精选理由

适合脱敏大规模语料的可复现、可恢复处理管道。

▸ 风险提示

需访问大量敏感临床文本,注意合规与隐私。

这个 Skill 做什么

在本地对大量临床笔记运行 NER/去标识化,支持分片、断点续传和 JSONL 输出。

在本地批量跑 NER、PII 提取或去标识化,支持分片、断点续传和追加式 JSONL 输出,能处理中等到大量的临床笔记语料。适合需要把成千上万篇文档做脱敏或信息抽取、要求可重启和不会一次性占满内存的场景。特点是可恢复的批处理 runner:有进度回调、无 PHI 日志、文档分块和容错,结果可以边跑边流出。

▸ 展开 SKILL.md 英文原文

Run large-scale batch NER, PII extraction, or de-identification over many clinical notes on-device with OpenMed, with sharding, checkpointing, resumability, and append-only JSONL output. Use when the user needs to process a corpus or folder of notes, de-identify a dataset, run NER over thousands of documents, build a resumable batch pipeline, or stream results to JSONL without holding everything in memory. Covers process_batch / BatchProcessor / BatchItem / BatchResult, the operation= selector (analyze_text | extract_pii | deidentify), iter_process streaming, the PHI-safe on_progress callback, chunking long documents, and no-PHI logging. Produces a resumable batch runner over an OpenMed model.

垂直行业批处理去标识化可恢复大规模通用
4.7k
Stars
581
Forks
40
仓库内 Skill
积累中
7 日增星
安装 / 使用
给你的 Agent 一句话(通用)
帮我安装这个 skill:https://raw.githubusercontent.com/maziyarpanahi/openmed/master/skills/batch-processing-clinical-text/SKILL.md
或 curl 直取 SKILL.md
curl -fsSL "https://raw.githubusercontent.com/maziyarpanahi/openmed/master/skills/batch-processing-clinical-text/SKILL.md"
SKILL.MD 节选查看完整文件 ↗
# Batch processing clinical text

`openmed.processing` runs OpenMed over many documents efficiently, with progress
tracking, per-item error isolation, and streaming. It runs **fully on-device**:
the corpus, the model, and the output never leave the host. This skill shows a
**resumable** runner — sharded, checkpointed, append-only JSONL — that you can
restart without reprocessing.

## When to use this skill

For corpora, folders, or datasets — anything beyond a handful of notes. For a
single note, just call `openmed.analyze_text` / `deidentify` directly
(`extracting-clinical-entities`, `deidentifying-clinical-text`). For an
always-on HTTP service, see `serving-openmed-rest-api`.

## Quick sta
via SKILL·HOT · 数据来自 GitHub 公开信息 · 原文版权归作者所有