PostHog/posthog✦ 精选0°

fixing-flaky-tests

仓库创建 2020年1月24日最近提交 3 小时前SkillHot 收录 3 小时前
▸ 精选理由

强调先复现再修复与根因定位,能显著降低 CI 抖动成本。

这个 Skill 做什么

指导复现、定位、修复并验证 CI 中间歇性(flaky)测试的流程与准则。

处理 CI 中间歇性失败(flaky)测试的全流程:先在本地复现,再定位根因并修复,最后用 N 次跑验真伪。绝不靠加 sleeps/重试或放大超时来掩盖问题,适用于 Jest、pytest、Playwright 等测试框架,并会给出验证和回归试验的方法。

▸ 展开 SKILL.md 英文原文

Guides an agent through reproducing, root-causing, fixing, and validating flaky tests in the PostHog monorepo. Use when a test fails intermittently in CI but passes on rerun or locally, when `hogli ci:insights` or the debugging-ci-failures skill classifies a failure as a flaky test, when given a GitHub Actions URL for a flaky job, or when asked to deflake, stabilize, or fix a flaky Jest, pytest, or Playwright test. Core discipline: reproduce locally before changing anything, fix the root cause (never mask it with sleeps, retries, or bigger timeouts), and prove the fix with an N-run validation loop sized to the observed failure rate. Stabilizing is not the only valid outcome — the skill also gates whether the test should exist, so deleting a test that catches nothing real, or re-leveling one that flakes because of the level it runs at, are first-class endings.

开发编程测试不稳定调试CI通用
3.7w
Stars
3.1k
Forks
40
仓库内 Skill
积累中
7 日增星
安装 / 使用
给你的 Agent 一句话(通用)
帮我安装这个 skill:https://raw.githubusercontent.com/PostHog/posthog/master/.agents/skills/fixing-flaky-tests/SKILL.md
或 curl 直取 SKILL.md
curl -fsSL "https://raw.githubusercontent.com/PostHog/posthog/master/.agents/skills/fixing-flaky-tests/SKILL.md"
SKILL.MD 节选查看完整文件 ↗
# Fixing flaky tests

Three non-negotiables, in order:

1. **Reproduce before you fix.** A fix for a failure you never observed is a guess.
   Only fall back to analytical fixes when the escalation ladder below is exhausted.
2. **Fix the root cause.** Sleeps, raised timeouts, retries, and weakened assertions hide flakes; they do not fix them.
3. **Validate with an N-run loop.** One green run proves nothing about an intermittent failure.
   Size N to the observed failure rate.

Stabilizing the test is not the only valid ending.
Once you know why it flakes, step 5 asks whether it should exist.
A test that catches nothing real is worth deleting, and one that flakes because of the level it runs 
via SKILL·HOT · 数据来自 GitHub 公开信息 · 原文版权归作者所有