peft-fine-tuning
仓库创建 2025年11月3日最近提交 1 个月前SkillHot 收录 20 天前
▸ 精选理由
能在有限显存下训练并显著降低训练开销，适合实务微调。
▸ 风险提示
需 PyTorch/HuggingFace 环境与显卡，且会下载大模型权重。
这个 Skill 做什么
用 PEFT/LoRA 等方法对大模型进行参数高效微调的实践与示例。
帮你在显存有限的机器上，把大模型用更少参数微调到能做特定任务，常用 LoRA、QLoRA 等 PEFT 方法，训练量级通常不到 1% 的参数就能收敛。适合在消费级 GPU（如 RTX 4090）或希望快速迭代多个任务适配器时用。特点是极省显存、迭代快，并能和 HuggingFace/transformers 无缝配合，便于多版本部署。
▸ 展开 SKILL.md 英文原文
Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.
1.1w
Stars
817
Forks
40
仓库内 Skill
+704
7 日增星
安装 / 使用
给你的 Agent 一句话（通用）
帮我安装这个 skill：https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/main/03-fine-tuning/peft/SKILL.md或 curl 直取 SKILL.md
curl -fsSL "https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/main/03-fine-tuning/peft/SKILL.md"SKILL.MD 节选查看完整文件 ↗
# PEFT (Parameter-Efficient Fine-Tuning) Fine-tune LLMs by training <1% of parameters using LoRA, QLoRA, and 25+ adapter methods. ## When to use PEFT **Use PEFT/LoRA when:** - Fine-tuning 7B-70B models on consumer GPUs (RTX 4090, A100) - Need to train <1% parameters (6MB adapters vs 14GB full model) - Want fast iteration with multiple task-specific adapters - Deploying multiple fine-tuned variants from one base model **Use QLoRA (PEFT + quantization) when:** - Fine-tuning 70B models on single 24GB GPU - Memory is the primary constraint - Can accept ~5% quality trade-off vs full fine-tuning **Use full fine-tuning instead when:** - Training small models (<1B parameters) - Need maximum qua
via SKILL·HOT · 数据来自 GitHub 公开信息 · 原文版权归作者所有