Moore Threads URPO Framework Cracks LLM Training Challenge: Single-Model Bidirectional Optimization Ushers in New AI Compute Paradigm

At the AAAI 2026 international conference, Moore Threads publicly unveiled its URPO (Unified Reward and Policy Optimization) framework for the first time, sending shockwaves through the industry. Leveraging an innovative “single-model, dual-task” architecture, the technology merges large language models’ instruction comprehension and self-evaluation capabilities into a single training pipeline. Experimental data shows its overall performance exceeds traditional baseline solutions by over 15%.

The core breakthrough lies in restructuring the underlying logic of large-scale model training. Traditional approaches require separate reward models and policy models to collaborate, while URPO employs a proprietary data format converter to transform three types of heterogeneous information—preference data, reasoning data, and open-ended instructions—into a unified signal stream that feeds directly into the GRPO training pipeline. This not only saves 30% of computing power consumption but also eliminates the error accumulation problems caused by multi-module coordination.

On the compute infrastructure front, Moore Threads simultaneously disclosed the deep adaptation results between the framework and its self-developed compute cards. Its unique tensor acceleration units can process model generation and scoring tasks in parallel, boosting URPO’s training efficiency by 2.4 times under the VERL reinforcement learning framework. This marks the first time a Chinese AI chip has achieved hardware-level customized optimization for a training framework, providing a new blueprint for future intelligent compute cluster construction. Performance verification data is particularly striking: Based on the Qwen2-5.7B base model, the URPO model scored 44.84 points on the AlpacaEval instruction leaderboard—a 6.7-point improvement over traditional methods. Even more noteworthy is its self-iteration ability: through built-in reward loop mechanisms, the model surpassed specialized reward models by scoring 85.15 points in the RewardBench evaluation, confirming the feasibility of the “training-equals-evaluation” paradigm.

总结