evaluating-with-leakage-gates
把关注点从F1转为残余泄露，适用于发布前合规审查
评估通常需要含真实PHI的黄金集，操作时需严格管控数据访问。
用泄露优先的G1a–G8门控评估去标识化/NER模型是否存在残余PHI泄露。
把模型放到以泄露为优先的 G1a–G8 审核门里跑，核心目标是找出任何残留的 PHI，而不是只看总体 F1。通常用合成金标集合做评估，决定模型是 RELEASABLE 还是 QUARANTINED，并强制每类直接标识符的召回下限或要求关键泄露为零。会产出带证据的门控报告，方便合规放行决策。
▸ 展开 SKILL.md 英文原文
Evaluate an OpenMed de-identification or clinical NER model against the leakage-first release gates G1a through G8, which gate releases on residual PHI leakage rather than on F1. Use when the user wants to run the OpenMed eval harness on a synthetic golden set, decide whether a de-id model is RELEASABLE or QUARANTINED, enforce direct-identifier recall floors, require zero critical leakage, fit calibration thresholds, or produce a signed gate report. Trigger on "release gate", "leakage", "is this model safe to ship", "G1a", "G3", "quarantine", "recall floor", or "calibration thresholds" in an OpenMed de-id context.
帮我安装这个 skill：https://raw.githubusercontent.com/maziyarpanahi/openmed/master/skills/evaluating-with-leakage-gates/SKILL.mdcurl -fsSL "https://raw.githubusercontent.com/maziyarpanahi/openmed/master/skills/evaluating-with-leakage-gates/SKILL.md"# Evaluating with Leakage Gates OpenMed's release gates answer one question: **did any PHI leak?** A de-id model with a beautiful F1 can still leak a single SSN — and that one leak is a HIPAA breach. So `openmed.eval` gates on *residual leakage* and *per-label recall floors*, not on aggregate F1. The candidate is either `RELEASABLE` or `QUARANTINED`; there is no partial credit. ## When to use this skill - You have a candidate de-id or PII model and need a ship / no-ship decision. - You want to run the benchmark harness over a **synthetic** golden suite. - You need to enforce direct-identifier recall floors and `critical_leakage == 0`. - You need calibration thresholds (`thresholds.json`)