advanced-evaluation

仓库创建 2026年1月15日最近提交 3 小时前SkillHot 收录 3 小时前
▸ 精选理由

把学术与业界评估实践落地为可生产化的评估管道和准则,适合模型开发评测团队。

这个 Skill 做什么

构建并实现面向 LLM 输出的评估体系、判定器与偏差缓解方法与流程。

把 LLM-as-a-Judge 的研究和经验落地成可运行的评估体系和判定器,帮你把抽象的评估方法变成可复用的流程。适合需要比较模型输出、做打分、pairwise 比较或缓解 position bias 与评估偏差的场景。重点在偏差缓解、自动化评估流水线和可靠的 rubrics 设计上,提供实用模板。

▸ 展开 SKILL.md 英文原文

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

研究检索评估LLM判官偏差缓解通用
4.4w
Stars
6.5k
Forks
40
仓库内 Skill
积累中
7 日增星
安装 / 使用
给你的 Agent 一句话(通用)
帮我安装这个 skill:https://raw.githubusercontent.com/sickn33/agentic-awesome-skills/main/skills/advanced-evaluation/SKILL.md
或 curl 直取 SKILL.md
curl -fsSL "https://raw.githubusercontent.com/sickn33/agentic-awesome-skills/main/skills/advanced-evaluation/SKILL.md"
SKILL.MD 节选查看完整文件 ↗
# Advanced Evaluation

This skill covers production-grade techniques for evaluating LLM outputs using LLMs as judges. It synthesizes research from academic papers, industry practices, and practical implementation experience into actionable patterns for building reliable evaluation systems.

**Key insight**: LLM-as-a-Judge is not a single technique but a family of approaches, each suited to different evaluation contexts. Choosing the right approach and mitigating known biases is the core competency this skill develops.

## When to Use
Activate this skill when:

- Building automated evaluation pipelines for LLM outputs
- Comparing multiple model responses to select the best one
- Establishing 
via SKILL·HOT · 数据来自 GitHub 公开信息 · 原文版权归作者所有