Global Markets
Home›Global Markets›China›China’s AI inference demand strains supply of high-end…
China’s AI inference demand strains supply of high-end Nvidia chips
In March, China’s average daily token calls topped 140 trillion, rising more than 1,000-fold since the start of 2024, deepening the compute squeeze as AI deployment expands.
Chinese AI companies are adjusting their software to meet surging inference demand, but they still face acute compute constraints due to limited access to high-end chips associated with Nvidia, according to the SCMP Economy. Inference, the phase where a trained model generates responses, can be adapted to domestic hardware for some workloads. However, industry insiders said complex tasks such as coding still require Nvidia chips, meaning companies are effectively constrained as AI shifts from model development to large scale deployment.
SCMP Economy also highlights comments from Guan Jiawei, vice president of inference optimisation start-up Approaching.AI, who said demand has split into tiers, with high quality tokens and stringent performance metrics far outstripping what domestic processors can reliably deliver. Guan added that models used in premium scenarios like coding place heavy demands on chips, and that high quality tokens still depend on Nvidia for viability.
The compute pressure is being worsened as AI becomes more agentic, performing real world tasks rather than only answering questions. SCMP Economy cites National Data Administration data showing China’s average daily token calls exceeded 140 trillion in March, up more than 1,000 fold from the beginning of 2024.