constitutional-ai

仓库创建 2025年11月3日最近提交 1 个月前SkillHot 收录 21 天前

▸ 精选理由

无需大量人工标注即可改进安全性，适合对齐研究与工程落地。

▸ 风险提示

自动化反馈可能放大偏差，仍需人工审查与监控。

这个 Skill 做什么

通过“宪章”让模型自我批评与用 AI 反馈训练以降低有害输出。

通过一套“宪章”让模型自我批评并改写回答，先做有监督的自我修正，再用 RLAIF（RL from AI Feedback）用模型的反馈做强化训练来降低有害输出。适合做安全对齐、减少有害或偏差回复的场景，能在不大量人工标注的情况下改进行为。特点是把监督和强化都建立在模型自评上，已用于 Claude 等系统的安全层。

▸ 展开 SKILL.md 英文原文

Anthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful outputs without human labels. Powers Claude's safety system.

研究检索对齐方法自我批评RLAIF通用

1.1w

Stars

817

Forks

仓库内 Skill

+704

7 日增星

安装 / 使用

给你的 Agent 一句话（通用）

帮我安装这个 skill：https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/main/07-safety-alignment/constitutional-ai/SKILL.md

或 curl 直取 SKILL.md

curl -fsSL "https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/main/07-safety-alignment/constitutional-ai/SKILL.md"

SKILL.MD 节选查看完整文件 ↗

# Constitutional AI - Harmlessness from AI Feedback

## Quick start

Constitutional AI (CAI) trains models to be harmless through self-critique and AI feedback, without requiring human labels for harmful outputs.

**Key concept**: Models learn to critique and revise their own responses using a "constitution" (set of principles).

**Two phases**:
1. **Supervised Learning (SL)**: Self-critique + revision
2. **Reinforcement Learning (RL)**: RLAIF (RL from AI Feedback)

**Constitution example**:
```
Principles:
1. Choose the response that is most helpful, honest, and harmless
2. Avoid responses that are toxic, racist, or sexist
3. Prefer responses that explain objections rather than refuse
4. Ch

via SKILL·HOT · 数据来自 GitHub 公开信息 · 原文版权归作者所有