Infinigence AI, Tsinghua University, and Shanghai Jiao Tong University have open-sourced APXInf, an edge inference engine for embodied AI. Its goal is not merely to make models run on robots, but to make them run fast, stably, and easily integrable under tight constraints of compute, power, and cost.
The headline numbers are straightforward: PI 0.5 FP8 on Jetson Thor cuts end-to-end inference latency from 278 ms to under 26 ms, with an inference rate of 38.46 Hz—meeting the inference frequency and response latency demands of real-time control in multiple robotic scenarios.
The hard part of edge inference is not only peak performance. Robots must continuously perceive, decide, and act. A few hundred milliseconds of difference may be a user-experience issue in chat applications, but on a robot it can mean delayed actions, discontinuous trajectories, or even repeated task failures.
There is also a practical economic calculation: compute cost, power consumption, and utilization of limited hardware resources. Simply moving cloud inference frameworks onto a robot body often makes it difficult to balance small-batch workloads,