Zhipu details GLM-5.3-Flash inference optimization deployed on 100,000 China-made AI accelerators

On Sept. 17, Zhipu AI's chief scientist of GLM, Tang Jie, shared on X an optimization study for the GLM-5.3-Flash inference system. The model runs on a cluster of more than 100,000 China-made AI accelerators. According to Tang, the team took only two weeks from the first run on the new hardware to carrying full production traffic, with end-to-end throughput improved by 3.2 times.

During the optimization, an infrastructure agent powered by GLM-5.3 helped analyze performance bottlenecks, propose fixes and modify code. The team also introduced a "dense feedback" mechanism that provides the model with more engineer-like signals, such as correctness checks, time consumption analysis and verification results. Tang described this as an early form of recursive self-improvement (RSI), though still far from a true RSI loop.

The deployment faced challenges including limited memory and interconnect bandwidth of domestic chips, adaptation of the new model architecture, 1-million-token context support, and multimodal request processing. Solutions included ReplaySSM, INT8/FP8/BF16 mixed-precision caching, and an Encode-Prefill-Decode (EPD) separated architecture. After deployment, GLM-5.3-Flash was anonymously tested as "Ox-Alpha" on OpenCode and OpenRouter, becoming one of the most-used models on both platforms within a week and processing more than 62 trillion tokens in six days.

The agent also found and fixed issues in the KDA context parallel path related to TF32 rounding error accumulation, merged in Flash Linear Attention PR #1180; improved KV Transfer overlapping with DeepEP dispatch, cutting extra overhead from over 30% to below 1%; and optimized a decode kernel, achieving a 1.71x speedup.