bootstrap

仓库创建 2025年7月8日最近提交 18 天前SkillHot 收录 8 小时前

▸ 精选理由

快速把空白项目变为可验证的初步评测流程，便于迭代。

这个 Skill 做什么

在无标注与评测集时构建初始评估体系与 v0 分级器。

当什么都没有时，30 分钟内搭建起初始评估体系和一个 v0 grader（用 OpenJudge SimpleRubricsGenerator），并给出走向校准评估的路线图。适合新产品或刚发现失效模式、需要从零开始做评估的场景。特别强调真实系统输出+人工标注为标签来源，结果在未校准前要明确标注。

▸ 展开 SKILL.md 英文原文

Use when the user has nothing — no traces, no labels, no eval set — and needs to build a v0 evaluation from scratch. Also use when the user says "I need to start evaluating my app but don't know where to begin," "I want to set up eval for a new product," or has just identified failure modes and needs to turn them into principles. Outputs a v0 grader in 30 minutes using OpenJudge SimpleRubricsGenerator, plus a roadmap to reach calibrated evaluation.

研究检索冷启动评估数据标注评测搭建通用

748

Stars

Forks

仓库内 Skill

+40

7 日增星

安装 / 使用

给你的 Agent 一句话（通用）

帮我安装这个 skill：https://raw.githubusercontent.com/agentscope-ai/OpenJudge/main/skills/eval_pipeline/08-bootstrap/SKILL.md

或 curl 直取 SKILL.md

curl -fsSL "https://raw.githubusercontent.com/agentscope-ai/OpenJudge/main/skills/eval_pipeline/08-bootstrap/SKILL.md"

SKILL.MD 节选查看完整文件 ↗

<HARD-GATE>
NO v0 grader deployed WITHOUT explicitly marking it as uncalibrated.
NO synthetic labels — LLM can generate eval inputs, but labels MUST come from real system output + human judgment.
NO principle without a source label documenting where it came from.
</HARD-GATE>

# Bootstrap

Cold-start an evaluation system when you have nothing. In 30 minutes you get a working
v0 grader and a clear path to a calibrated, trustworthy evaluation.

> **Requires OpenJudge** (`pip install py-openjudge`) for the grader generators
> (`SimpleRubricsGenerator` / `IterativeRubricsGenerator`). The interview, stratification,
> and calibration-roadmap methodology is SDK-independent.

## Checklist

You MUST

via SKILL·HOT · 数据来自 GitHub 公开信息 · 原文版权归作者所有