research-agent-pitfalls

防护自主研究循环中的常见失败模式，避免自信但错误的结论。

在自动化研究或代理做实证工作时帮你识别并避免那些会让模型“很自信但其实错了”的陷阱。它覆盖常见问题，比如 reward hacking、伪造数值、过拟合自身搜索信号、数据泄露、单次 seed 结论或 LLM-judge 偏差。适用于任何涉及实验数据、数字结论或可重复性验证的场景。特点是列出失败模式并给出具体防护建议，避免草率完成和误导性结论。

---
name: research-agent-pitfalls
description: Guards autonomous research loops against the failure modes that produce confident wrong conclusions: reward hacking, fabricated numbers when code fails, overfitting the agent's own search signal, data leakage, undertuned baselines, single-seed claims, LLM-judge bias, and premature completion. Use whenever work touches an empirical claim - designing an experiment; splitting, deduplicating or normalising data; setting up training runs, sweeps or ablations; filling in a results table, abstract or paper section; comparing against a baseline copied from a paper or repo; choosing which metrics to report; tuning or selecting against a test or held-out