benchmark-skill

仓库创建 2026年6月26日最近提交 20 天前SkillHot 收录 20 天前
▸ 精选理由

适合验证技能改进是否真实带来收益。

▸ 风险提示

可能调用模型或外部服务产生费用与数据外泄风险

这个 Skill 做什么

设计并运行技能的对照基准试验,生成可复现评估报告。

设计并运行可复现的对照试验来衡量某个 skill 是否真正有用:把模型分成有/无该技能两组跑同一任务,然后产出并保存到 skill-analysis/<skill>/ 的报告和数据。用在想要有证据回答“这项技能值不值得投入”时。它强调可重复性——重复运行会复用磁盘上的同一基线,保持可比性,不随意改动。

▸ 展开 SKILL.md 英文原文

Measure whether a skill actually pays off — design and run a controlled with/without benchmark of a target skill across models, then write the result up under `skill-analysis/<skill>/` as a study of one frozen skill revision. Use when asked to "benchmark the X skill", "is this skill worth it", "measure / evaluate a skill". On a repeat invocation it re-runs the *same* benchmark already on disk rather than redesigning one — so cohorts stay comparable. Not for the improve-a-skill loop that edits and re-measures (that is `/optimize-skill`), and not for benchmarking a website or generic code performance; this measures a *skill's* effect on agent behaviour.

研究检索基准测试评估实验指标通用
0
Stars
0
Forks
11
仓库内 Skill
+0
7 日增星
安装 / 使用
给你的 Agent 一句话(通用)
帮我安装这个 skill:https://raw.githubusercontent.com/Svendsys/agent-skills/master/skills/benchmark-skill/SKILL.md
或 curl 直取 SKILL.md
curl -fsSL "https://raw.githubusercontent.com/Svendsys/agent-skills/master/skills/benchmark-skill/SKILL.md"
SKILL.MD 节选查看完整文件 ↗
# Benchmark a Skill

A benchmark answers one question about a skill: **does it pay off, and where?**
The deliverable is a self-contained study saved under
`skill-analysis/<skill-name>/` — a report grounded in per-agent data, not vibes.
**Saving the study is part of running the benchmark, not an afterthought:** every
run persists its report, data, evidence, and harness under
`skill-analysis/<skill-name>/`, so the result is durable and the next run can build on
it. **If the repo you're working in has no `skill-analysis/` directory yet, create it**
— it is the home for these studies, so set it up before saving the first one. The shape
to match is
[`skill-analysis/TEMPLATE/`](../../skill-analysi
via SKILL·HOT · 数据来自 GitHub 公开信息 · 原文版权归作者所有