The September release lit up the industry. The Information was first to report that OpenAI’s newly released GPT-6 Astra uses recurrent-depth technology: the same set of Transformer layers runs repeatedly, making computation deeper without proportionally increasing parameters. Once the news broke, Looped Transformers became the focus of industry debate overnight.
Controversy followed. Because recurrence occurs in hidden states, the intermediate process may not form a human-readable chain of thought, and AI safety experts worry the model will be harder to monitor. OpenAI Chief Scientist Jakub Pachocki had to respond and said he would write a full explanation. Behind the buzz, academia is more concerned with another question: if the model loops more rounds, will it actually become stronger?
The route dates back to the Universal Transformer in 2018 and has recently evolved into directions such as Looped Transformer, recurrent depth, and latent reasoning. But academia has already stepped into pitfalls: under equal compute, recurrent models that save parameters often cannot beat ordinary Transformers. The bottlenecks are precisely how information inside the loop is managed and how computation is allocated.
Interestingly, Alibaba and partner universities published papers addressing these issues 11 months earlier. MeSH uses a memory buffer with a read/write router to solve recurrent idling and information overload. Experiments show weight sharing can reduce non-embedding parameters by about 33%, improve average zero-shot downstream accuracy by about 1 percentage point, and add almost