deepspeed

DeepSpeed 的分布式与内存优化实践指南与开发支持。

帮你把大模型分布式训练跑得更稳、更省钱：会讲如何用 DeepSpeed 的 ZeRO 分段、pipeline parallelism、混合精度（FP16/BF16/FP8）等手段节省显存和加速收敛。适合在模型太大单卡放不下、想用 1-bit Adam 或 sparse attention 提升效率、或调试 DeepSpeed 配置时调用。特别之处是侧重实战优化和排错，既讲原理也给可落地的配置和调试步骤。

# Deepspeed Skill

Comprehensive assistance with deepspeed development, generated from official documentation.

## When to Use This Skill

This skill should be triggered when:
- Working with deepspeed
- Asking about deepspeed features or APIs
- Implementing deepspeed solutions
- Debugging deepspeed code
- Learning deepspeed best practices

## Quick Reference

### Common Patterns

**Pattern 1:** DeepNVMe Contents Requirements Creating DeepNVMe Handles Using DeepNVMe Handles Blocking File Write Non-Blocking File Write Parallel File Write Pinned Tensors Putting it together Acknowledgements Appendix Advanced Handle Creation Performance Tuning DeepNVMe APIs General I/O APIs GDS-specific APIs Hand