meta-eval

仓库创建 2025年7月8日最近提交 18 天前SkillHot 收录 8 小时前

▸ 精选理由

适合想搭建或改进评估管线的团队，覆盖从数据到工具的决策点。

这个 Skill 做什么

指导从零构建 LLM/agent 评估体系并推荐后续评测路径与工具。

从零帮你搭建 LLM/agent 的评估体系：先问清数据形式、标注情况和现有 trace，然后给出可执行的评测路径和工具建议。适合不知道如何开始做模型/代理评测、对比提示或衡量 RAG 准确度的团队。作为入口路由器的特点是基于 data_form 与 label_status 快速给出子流程建议，并在掌握更多背景后细化下一步。

▸ 展开 SKILL.md 英文原文

Use when the user wants to build an evaluation system for an LLM/agent application but doesn't know where to start — they have traces, prompts, RAG pipelines, or nothing at all. Also use when the user mentions evaluation, eval, benchmarking, testing LLM quality, measuring agent performance, assessing RAG accuracy, or wants to compare prompts/models. This skill is the entry router: it asks diagnostic questions then recommends which sub-skill (local workflow) to use next.

研究检索评估流程测评策略入门引导通用

748

Stars

Forks

仓库内 Skill

+40

7 日增星

安装 / 使用

给你的 Agent 一句话（通用）

帮我安装这个 skill：https://raw.githubusercontent.com/agentscope-ai/OpenJudge/main/skills/eval_pipeline/00-meta-eval/SKILL.md

或 curl 直取 SKILL.md

curl -fsSL "https://raw.githubusercontent.com/agentscope-ai/OpenJudge/main/skills/eval_pipeline/00-meta-eval/SKILL.md"

SKILL.MD 节选查看完整文件 ↗

<HARD-GATE>
NO sub-skill recommendation WITHOUT identifying data_form + label_status (these two pick the entry workflow).
ALWAYS give a provisional recommendation once data_form + label_status are known, even if stakes/user_prior are still unknown — then ask the remaining questions to refine the downstream path. Do not withhold the route while waiting on stakes.
</HARD-GATE>

# Meta Eval

Entry router for the eval skill collection. You diagnose what the user has and route
them to the right sub-skill. You don't do evaluation yourself — you're the triage desk.

Each sub-skill is self-contained: it carries inline the data shapes, statistics, and data
principles it needs, so it can be installed

via SKILL·HOT · 数据来自 GitHub 公开信息 · 原文版权归作者所有