fixing-flaky-tests
强调先复现再修复与根因定位,能显著降低 CI 抖动成本。
指导复现、定位、修复并验证 CI 中间歇性(flaky)测试的流程与准则。
处理 CI 中间歇性失败(flaky)测试的全流程:先在本地复现,再定位根因并修复,最后用 N 次跑验真伪。绝不靠加 sleeps/重试或放大超时来掩盖问题,适用于 Jest、pytest、Playwright 等测试框架,并会给出验证和回归试验的方法。
▸ 展开 SKILL.md 英文原文
Guides an agent through reproducing, root-causing, fixing, and validating flaky tests in the PostHog monorepo. Use when a test fails intermittently in CI but passes on rerun or locally, when `hogli ci:insights` or the debugging-ci-failures skill classifies a failure as a flaky test, when given a GitHub Actions URL for a flaky job, or when asked to deflake, stabilize, or fix a flaky Jest, pytest, or Playwright test. Core discipline: reproduce locally before changing anything, fix the root cause (never mask it with sleeps, retries, or bigger timeouts), and prove the fix with an N-run validation loop sized to the observed failure rate. Stabilizing is not the only valid outcome — the skill also gates whether the test should exist, so deleting a test that catches nothing real, or re-leveling one that flakes because of the level it runs at, are first-class endings.
帮我安装这个 skill:https://raw.githubusercontent.com/PostHog/posthog/master/.agents/skills/fixing-flaky-tests/SKILL.mdcurl -fsSL "https://raw.githubusercontent.com/PostHog/posthog/master/.agents/skills/fixing-flaky-tests/SKILL.md"# Fixing flaky tests Three non-negotiables, in order: 1. **Reproduce before you fix.** A fix for a failure you never observed is a guess. Only fall back to analytical fixes when the escalation ladder below is exhausted. 2. **Fix the root cause.** Sleeps, raised timeouts, retries, and weakened assertions hide flakes; they do not fix them. 3. **Validate with an N-run loop.** One green run proves nothing about an intermittent failure. Size N to the observed failure rate. Stabilizing the test is not the only valid ending. Once you know why it flakes, step 5 asks whether it should exist. A test that catches nothing real is worth deleting, and one that flakes because of the level it runs