The demand for AI token services in China is experiencing explosive growth, driven by the rapid advancement of large model technology. According to a new report from the China Academy of Information and Communications Technology (CAICT), the daily average consumption of tokens in China reached approximately 500 trillion in June 2026. This staggering figure is nearly four times the consumption level seen in December 2025, highlighting a critical phase where technology is rapidly transitioning into widespread application.
As more companies embrace the Model-as-a-Service (MaaS) paradigm, public cloud APIs are becoming the primary channel for delivering these token services. However, users have historically lacked clear benchmarks to assess the performance of these services, leading to difficulties in selection and implementation.
CAICT Report Sheds Light on Token Service Performance
To address this gap, CAICT’s Artificial Intelligence Research Institute, in collaboration with the Ministry of Industry and Information Technology’s Key Laboratory of Artificial Intelligence Large Models and Soft/Hardware Evaluation, and the China Artificial Intelligence Industry Development Alliance, has been diligently monitoring the performance of public cloud large model token services since 2025. This continuous effort culminates in the recently released Large Model Token Service Performance Monitoring Report (2026).
The report, based on monitoring data from January to August 2026, provides a comprehensive analysis across several key dimensions: invocation success rate, throughput speed, first token latency (TTFT), stability, pricing, and context window size. It offers a comparative analysis between the official platforms of original AI model providers and the performance of four open-source models (DeepSeek, Qwen, GLM, Kimi) when deployed on third-party platforms.
Key Findings from the Report
1. Throughput Doubles, Still Trails International Benchmarks
A significant finding is the doubling of domestic token service throughput, measured in tokens per second (TPS). The average TPS across domestic models reached 78.23 in August 2026, a substantial increase from the 2025 average of 38.25. However, despite this impressive domestic growth, international typical models averaged 112.82 TPS in August 2026, also more than doubling from their December 2025 average of 51.35 TPS. This indicates that while China’s token services are rapidly improving, they still lag behind their global counterparts.
2. Latency Increases Amidst Growing Model Complexity
The report also notes an increase in the average time to first token (TTFT) for domestic models, rising to 1.26 seconds in August 2026 from a 2025 average of 0.78 seconds. Interestingly, international models also saw their TTFT nearly double, from 1.04 seconds in April to 2.08 seconds in August. CAICT clarifies that this increase in latency isn’t necessarily a sign of performance degradation. Instead, it’s a concurrent result of several factors, including the growing parameter count of models, expanding context lengths, and escalating invocation volumes.
3. Token Service Pricing Sees a Notable Resurgence
After a period of intense price competition over the past two years, the cost of token services has seen a marked increase in 2026. The normalized average price for domestic models in the latter half of 2025 was 6.8 yuan, but this year it has risen to an average of 13.46 yuan, an almost 100% increase. International models, while still more expensive, saw their average normalized price around 117.8 yuan for typical models released in 2026.
4. Context Windows Expand Towards the Million-Token Mark
A major trend highlighted is the continuous expansion of context window lengths. In the monitored models, 47.8% now support a context window of 1 million tokens. Furthermore, models with context lengths of 200,000 tokens or more constitute 95.7% of the total, a significant leap of over 76 percentage points compared to the end of 2025. Conversely, models with context lengths below 200,000 tokens are steadily decreasing in prevalence.
5. Open-Source Models Show Strong Performance Gains, but Platform Variance Exists
The report also evaluated the latest versions of four open-source models in May 2026, finding consistent improvements in TPS and reductions in TTFT. For instance, the DeepSeek-V4-Pro achieved twice the TPS of the DeepSeek-V3.2 in August. While initial upgrades led to a temporary increase in latency, subsequent inference optimizations brought TTFT back to pre-upgrade levels. A key observation is the significant performance variation of the same open-source model across different third-party platforms. Some platforms offered performance nearly double that of official services, attributed to differences in inference acceleration, optimization strategies, and cost control measures employed by various providers.
This comprehensive report by CAICT aims to provide crucial references for users navigating the complex landscape of AI token services and offer valuable insights for industry players seeking to enhance their service capabilities.
