Wang Xin, technical lead of Ant Group’s Altum large model training data processing system, has confirmed his attendance at the QCon Shanghai session “AI Infra: Compute Efficiency Determines Large-Scale Deployment,” where he will present “Altum: Design and Practice of Ant Group’s Next-Generation Large Model Training Data Processing System.” The talk comes as large model training data scales continue to rise and model iteration accelerates, creating enormous challenges across GPU efficiency, task stability, and data delivery timeliness.
Training data processing often spans multiple layers—compute, engines, platforms, and data—and inefficiency in any link can slow the entire iteration cycle. Ant Group is trying to address these issues systematically with Altum. Its approach is not point optimization, but rather a trade-off among performance, stability, and efficiency.
On performance, Altum has made several key moves. First, heterogeneous compute fusion scheduling is compatible with different operator systems at the upper layer and supports multiple compute types at the lower layer, improving global resource utilization. Second, an end-to-end upgrade based on Paimon data lake storage optimizes the full chain from data collection to model training consumption, improving storage and network I/O performance. Third, operator performance optimization and incremental computing—including global