ray-data

用于机器学习的分布式数据处理与预处理，可扩展至集群。

做机器学习用的分布式数据处理和预处理，支持 Parquet/CSV/JSON、图片等，多节点上做流式或批量处理并能跑在 CPU/GPU 上。用在大规模数据预处理、批量推理、多模态数据加载或分布式 ETL 场景。特点是可从单机扩到上百节点，并能和 Ray Train、PyTorch、TensorFlow 无缝集成。

# Ray Data - Scalable ML Data Processing

Distributed data processing library for ML and AI workloads.

## When to use Ray Data

**Use Ray Data when:**
- Processing large datasets (>100GB) for ML training
- Need distributed data preprocessing across cluster
- Building batch inference pipelines
- Loading multi-modal data (images, audio, video)
- Scaling data processing from laptop to cluster

**Key features**:
- **Streaming execution**: Process data larger than memory
- **GPU support**: Accelerate transforms with GPUs
- **Framework integration**: PyTorch, TensorFlow, HuggingFace
- **Multi-modal**: Images, Parquet, CSV, JSON, audio, video

**Use alternatives instead**:
- **Pandas**: Small data