agent-evaluation

为LLM代理提供行为测试、能力评估与生产可靠性监控的工具集。

给 LLM 代理做一套可量化的测试和基准评估：覆盖行为测试、能力评估、可靠性指标、回归测试和生产监控。适合在把 agent 投产前后做基准、找弱点或持续监控表现时使用。特别之处是把真实世界任务抽成可测指标，帮助发现上线后才会暴露的可靠性问题。

# Agent Evaluation

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks

## Capabilities

- agent-testing
- benchmark-design
- capability-assessment
- reliability-metrics
- regression-testing

## Prerequisites

- Knowledge: Testing methodologies, Statistical analysis basics, LLM behavior patterns
- Skills_recommended: autonomous-agents, multi-agent-orchestration
- Required skills: testing-fundamentals, llm-fundamentals

## Scope

- Does_not_cover: Model training evaluation (loss, perplexity), Fairness and bias testing, User experience testin