ray-data

面向ML的可扩展分布式数据处理框架，支持Parquet、流式与多模态数据。

面向 ML 的可扩展分布式数据处理框架，能读写 Parquet/CSV/JSON、图片音频等多模态数据，支持流式和批处理并在 CPU/GPU 上运行。适合做大规模数据预处理、批量推理或把本地流程扩展到数百节点的 ETL 管道。特点是和 Ray Train、PyTorch、TensorFlow 无缝集成，方便把数据处理与训练并行化。

# Ray Data - Scalable ML Data Processing

Distributed data processing library for ML and AI workloads.

## When to use Ray Data

**Use Ray Data when:**
- Processing large datasets (>100GB) for ML training
- Need distributed data preprocessing across cluster
- Building batch inference pipelines
- Loading multi-modal data (images, audio, video)
- Scaling data processing from laptop to cluster

**Key features**:
- **Streaming execution**: Process data larger than memory
- **GPU support**: Accelerate transforms with GPUs
- **Framework integration**: PyTorch, TensorFlow, HuggingFace
- **Multi-modal**: Images, Parquet, CSV, JSON, audio, video

**Use alternatives instead**:
- **Pandas**: Small data