Five real problems of China's computing power industry
Caijing (财经), China’s famous independent business magazine, recently published an article highlighting “five real problems of the computing power industry”. To me, this is likely to be an underestimated piece:
Five real problems of the computing power industry have emerged （中国算力产业出现五个真问题）
Global computational resources are shifting significantly, placing China's computing industry at a crossroads. The industry, in its broadest sense, encompasses cloud computing providers, equipment service firms, and chip suppliers. Currently, the growth rate of intelligent computing power, such as GPUs, far outpaces that of general-purpose computing power, like CPUs, propelling the entire supply chain forward.
At this critical juncture, computational resources in China are becoming increasingly dispersed. Investment in data centres is surging, yet the growth rate of the public cloud market remains limited. Data from international market research firms IDC and Gartner indicate that the growth rate of China's public cloud market has fallen to a five-year low. In contrast, investment in data centres has reached a five-year high.
The procurement and research and development costs associated with AI computing power are high. Theoretically, centralizing computing power in the hands of cloud providers is the most economical, as public clouds can efficiently leverage economies of scale to reduce costs. However, there is a gap between theory and reality. Public clouds are perceived to have limited short-term growth potential. Locally deployed clouds (hybrid, private, and dedicated clouds) are expected to be the main drivers in the next 1-3 years.
The demand for intelligent computing power has led to the development of large models. In this transformative period, U.S. cloud providers have reaped the first wave of growth dividends, while their Chinese counterparts have yet to see a significant increase in growth dividends.
Large models require advanced AI chips. However, in October 2023, the U.S. Department of Commerce's Bureau of Industry and Security cut off China's supply of high-performance AI chips. The need for domestic alternatives has become more urgent. Chips are considered crucial for breaking through industry barriers. Huawei's Ascend AI chips are one alternative to address the bottleneck issue. However, the current technical level of Ascend is only sufficient to solve the "functional" problem, without addressing usability and cost-effectiveness.
Computing power is regarded as a key factor in national competition. Chinese policy sectors are increasingly eager to implement industrial policies in the computing power market to enhance the competitiveness of China's computing industry. Jensen Huang, the founder of Nvidia, stated, "Computing power is power," advocating for each country to establish its own sovereign AI infrastructure. However, U.S. Commerce Secretary Gina Raimondo publicly stated in January this year that the U.S. is actively preventing China from obtaining the computing power needed to train large models.
A series of key issues are emerging in China's computing industry, including but not limited to:
Why has computing power become more dispersed?
How does the slow penetration of public clouds constrain China's new computational cycle?
When will the industrial dividends of AI computing power serving large models arrive?
How far can the domestic replacement in the computational field go, and what impact will it have on the new cycle of computational layout?
Where will China's computing power industry policy head?
1. Why has computing power become more dispersed?
The advent of large model industries has catalyzed the market for intelligent computing power. In theory, the most economical arrangement is to centralize computational resources in the hands of cloud providers due to the higher acquisition costs, more complex cluster management, and greater research and development investments required for AI computing power. This would enable public clouds to exploit economies of scale to minimize costs.
However, a divergence is observed between China and the global markets: computing power is becoming more concentrated internationally, while in China, it is becoming increasingly dispersed.
Analysis by "Caijing Eleven" comparing the growth rates of public cloud IaaS and data centre investments in China and internationally over the past five years reveals starkly contrasting trends. According to IDC and Gartner data, investment in data centres in China has continuously risen since 2019, while the growth rate for public cloud IaaS has declined. Conversely, the international market has consistently seen a higher growth rate in public cloud IaaS than in data centre investments.
This data implies that in China, investments in data centres are heating up, whereas the public cloud market is not experiencing a similar uptick. In the international arena, new computing power is still gravitating towards a few cloud providers.
In 2023, the investment growth rate for data centres in China began to surpass that of public cloud IaaS. Gartner data shows that the expenditure growth rate for data centre systems in China stood at 20.7% for the year, while the global market saw only a 7.1% increase. IDC data indicates that in the first half of 2023, the growth rate for public cloud IaaS in China was merely 13.2%, compared to 16.9% in the international market.
Three primary factors have exacerbated the dispersion of computing power in China in 2023:
The U.S. ceased supplying advanced AI chips to China, reducing the new AI computing power available to leading tech companies.
Some domestic enterprises continued transporting the restricted AI chips from abroad through various unconventional channels, despite high costs and small quantities.
Numerous local cities aimed to establish AI chip-based intelligent computing centres to attract investment and promote local industry upgrading, thereby driving up investment in infrastructure.
In the international market, major tech firms like Microsoft, Google, Oracle, Facebook, and Tesla were the primary purchasers of Nvidia's advanced AI chips. Before October 2023, Alibaba, Tencent, ByteDance, and Baidu were also major Chinese buyers of Nvidia's advanced AI chips.
Following the U.S. Department of Commerce's Bureau of Industry and Security's update on export control rules for advanced computing chips and semiconductor manufacturing equipment in October 2023, which prohibited U.S. companies like Nvidia from selling advanced AI chips to China, Chinese tech companies were unable to procure Nvidia's A100/A800, H100/H800 chips through regular channels. These chips were considered optimal for training and inference of large AI models.
Nevertheless, these chips continued to flow into the Chinese market through irregular channels. According to several server distributors interviewed by "Caijing Eleven," the prices for these chips often reached 4-5 times the official Nvidia pricing. Leading Chinese tech companies, wary of being added to the U.S. Commerce Department's Entity List, typically refrained from purchasing these chips through irregular channels. Thus, the skyrocketing prices of Nvidia chips became an opportunity for some firms to make speculative investments.
Enterprises that had never been involved in the computing power business began to enter the market with substantial investments, including companies from industries as diverse as monosodium glutamate production, dye manufacturing, textiles, and chemicals. These companies, characterized by declining revenues and low gross margins in their core businesses, found that the primitive business of server leasing offered higher gross margins than their main operations. This venture into computing power even led to a surge in their stock prices. However, this cross-industry foray was speculative in nature, leading to multiple inquiries from regulatory authorities.
Several local cities also made significant investments in intelligent computing centres. For example, in 2024, Harbin's Pingfang District approved a 460 million yuan investment for the Ha Investment Intelligent Computing Center. In 2023, Changchun announced a 480 million yuan investment plan for the Changchun New Area Intelligent Computing Center, Qingdao announced a 1.77 billion yuan investment plan for the Metaverse Intelligence Computing Center, and Nanchang announced a 450 million yuan investment plan for the Jiangxi Artificial Intelligence Computing Center.
Some local intelligent computing centre officials interviewed by "Caijing Eleven" stated that the rationale behind some cities' investments in intelligent computing centres was to expand infrastructure, attract investment, and recruit talent to drive local industrial upgrading. Many local governments also offered computing power vouchers as market subsidies to enterprises. However, the potential risks for some data centres included insufficient utilization rates and the inability to recoup investment costs.
A greater hazard for local intelligent computing centres was the potential need for ongoing market subsidies to sustain themselves, which could impose a financial burden on local governments and crowd out market space for other normal enterprises.
Many policy experts interviewed by "Caijing Eleven" expressed that local governments should not directly engage in building intelligent computing centres, as this could lead to redundant construction and wastage of resources. The joint guidance issued by the National Development and Reform Commission and other ministries at the end of December 2023, aimed at implementing the "East Data West Computing" project and accelerating the construction of a national integrated computing power network, also included restrictive provisions, stating that no large or super-large data centres should be built outside the "eight major hubs and ten clusters."
2. How does the slow penetration of public clouds constrain China's new computational cycle?
The dispersion of computational resources is detrimental to the industry's healthy development, particularly in the context of the large model arms race. Logically, China's public cloud should expand in scale, thereby diluting procurement and research and development costs to establish a positive cycle. This logic is evident in the operations of the three major American cloud service providers, yet the domestic market has taken the opposite direction.
In 2023, a clear division emerged in China's cloud market: Alibaba Cloud and Tencent Cloud focused on public cloud services, while Huawei Cloud and operator-based clouds concentrated on local deployments such as hybrid, private, and dedicated clouds.
For many years, China's cloud market aspired to emulate the public cloud model of the American market, where services are predominantly provided through public clouds known for their large scale and high efficiency. The architecture of public clouds inherently allows for global market expansion, continuously reducing computational research and development costs.
Both providers (including Alibaba Cloud, Huawei Cloud, Tencent Cloud, etc.) and consumers (state-owned conglomerates and comprehensive enterprises) shared the same view with "Caijing Eleven": public cloud computational efficiency is the highest and the cost the lowest. From the perspective of global cloud computing technology development, the public cloud is the most ideal direction. However, they also concurred that, in reality, the penetration rate of public clouds in China is unlikely to increase significantly in the short term.
In recent years, the concept of public clouds has continually faced challenges in the Chinese market. The internet industry, which has been the most enthusiastic adopter of public clouds, has remained sluggish in recent years, leading to insufficient demand for public cloud services. The government and enterprise sector, which accounts for a significant portion of IT expenditure, prefers private clouds, hybrid clouds, and dedicated clouds (a model that sits between private and public clouds, often deployed locally with independent resource pools and managed under a hosting model rather than the pure leasing model of public clouds).
Several reasons underpin the government and enterprise sector's aversion to public clouds: firstly, the data in public clouds is not deployed locally, complicating accountability and clarity in the event of incidents. Local deployments, controlled by the organizations themselves, better meet regulatory and security requirements. Secondly, considering the preservation and appreciation of state-owned assets, these institutions favour private clouds over public clouds, as public clouds are considered capital expenditures, while private clouds are treated as fixed assets in financial statements. Even though the amortization cost of private clouds might be higher, they still appear as tangible assets on the balance sheets.
A strategic analyst from a Chinese cloud provider candidly told "Caijing Eleven" that the internet industry has been the primary customer base for public clouds, but the Chinese internet sector has been in a prolonged slump. The provider assesses that the government and enterprise sector will be the main battleground in the next three years, with hybrid clouds, private clouds, and dedicated clouds becoming the main engines of China's cloud market.
In August 2022, Gartner data showed that China's hybrid cloud adoption rate reached 42% in 2021. Gartner predicted that by 2024, the penetration rate of hybrid clouds in China would reach 70%, significantly higher than the global average of 50%. Reality has confirmed this, with the growth rate of China's public cloud market continuously declining. Local deployments (hybrid, private, and dedicated clouds) have grown steadily.
In 2023, the growth rate of China's public cloud market fell to an all-time low. IDC data revealed that in the first half of 2023, China's public cloud market size (IaaS/PaaS/SaaS) was $19.01 billion, with a year-on-year growth of 14.7%. Among these, the IaaS market size was $11.29 billion, with a year-on-year growth of 13.2%; the PaaS market size was $3.29 billion, with a year-on-year growth of 26.3%. The overall growth rate of China's public cloud market was lower than the international market. In the same period, the global public cloud market size was $315.5 billion, with a year-on-year growth of 19.1%.
The market share of providers primarily focused on public cloud services, such as Alibaba Cloud and Tencent Cloud, has continuously declined. In 2023, Alibaba Cloud's market share in the public cloud IaaS sector fell below the 30% threshold for the first time. Providers with hybrid cloud/private cloud services, like Huawei Cloud and China Telecom Cloud, have seen their market shares increase.
IDC has only released data for the dedicated cloud market, not for hybrid or private clouds. The dedicated cloud market continues to grow steadily. In the first half of 2023, China's dedicated cloud market size was $15.43 billion, with a year-on-year growth of 26.6%. The top five market shares were held by China Telecom (18.7%), China Mobile (12.0%), Inspur (11.6%), China Unicom (10.3%), and JD Cloud (10.2%).
An overabundance of private clouds not only leads to the fragmentation of China's computing power industry but also the fragmentation of the software services sector. The underlying industrial ecosystem is labour-intensive, with diminishing marginal benefits as personnel numbers increase, making it challenging to invest significant research and development costs for innovation. The consensus among cloud providers' technical experts is that to establish scale and cost advantages in the large model arms race, China's computing power industry should adhere to the public cloud's technological direction. Public clouds are conducive to the growth of innovative enterprises, leading to a thriving software ecosystem.
3. When will the industrial dividends of AI computing power serving large models arrive?
Large models have introduced new growth opportunities for the computing power industry. Chinese cloud providers, in particular, have been hopeful that large models will reinvigorate the sluggish cloud market.
However, financial data indicates that American cloud providers have reaped the initial wave of growth dividends, while Chinese counterparts have yet to see significant benefits. Since the latter half of 2023, the US cloud market has begun to recover, thanks to large models.
In 2023, Microsoft's Intelligent Cloud reported revenues of $96.21 billion, marking a 17.6% year-on-year increase; Amazon AWS generated $90.75 billion in revenue, up by 13.4%; and Google Cloud brought in $33.09 billion, a 25.9% increase from the previous year. Microsoft, in particular, stood out with its revenue growth for the Intelligent Cloud accelerating in the third and fourth quarters of 2023, driven by large model technologies. Both Amazon AWS and Google Cloud also saw slight increases in their revenue growth rates.
In contrast, the growth of Chinese cloud providers in 2023 was not significantly linked to large models. Alibaba Cloud's annual revenue growth was below 4%. China Telecom Cloud and China Mobile Cloud experienced over 50% revenue growth in the first half of the year. "Caijing Eleven" gathered that Huawei Cloud's revenue growth for 2023 was 36%. China Telecom Cloud and China Mobile Cloud primarily benefited from government and enterprise comprehensive projects, while Huawei Cloud gained from its hybrid cloud and PaaS products like databases.
Senior executives from cloud companies have candidly stated to "Caijing Eleven" that some clients merely purchase hardware such as chips and servers from cloud providers, with limited progress in training and utilizing models, which is essential to leverage the value of large models fully. They anticipate that as business negotiations and implementations advance, a wave of commercial use cases for large models will emerge domestically by the first quarter of 2024, potentially boosting cloud providers' performance.
Microsoft was among the first to capitalize on the large model boon, serving as a benchmark for Chinese cloud providers. Currently, Microsoft has established a growth trajectory combining "cloud, software, and AI". Its cloud business has seen revenue growth rates between 20%-30%, with a gross margin of approximately 60%; its software business has experienced growth rates between 40%-60%, boasting a gross margin of up to 80%; and its AI computing power revenues have surged by over 100%. These three growth vectors synergistically reinforce one another, with the cloud serving as the foundational resource, software enhancing customer loyalty and generating profit, and AI driving resource consumption and software innovation.
In a small-scale media interaction in January, a senior executive from Huawei Cloud mentioned that their strategic planning and market research are closely aligned with Microsoft's approach, highlighting the similarities between Huawei and Microsoft. Microsoft has a long-standing presence in the government and large enterprise markets and offers a range of renowned enterprise software products including Azure cloud services, Power Platform, Dynamics, Office for Enterprise, and the GitHub open-source community.
The executive further explained that Huawei Cloud possesses its own set of "cutting-edge products", including Ascend AI computing power, large models, databases, software development pipelines, big data, cloud office solutions, and cloud security. In the Chinese market, merely selling cloud resources cannot sway customers. Tailoring these advanced products to the needs of government, financial, manufacturing, and energy sectors is crucial for success.
4. How far can the domestic replacement in the computational field go, and what impact will it have on the new cycle of computational layout?
Another significant factor impacting China's computing power industry is the push for domestic substitution. Though reactive in nature, this initiative represents the only strategy for the Chinese computing power supply chain to seize the initiative.
Amid escalating Sino-American technological tensions and global geopolitical conflicts, China's drive for domestic substitution has accelerated to counteract the potential 'black swan' events such as supply disruptions. This entails replacing chips, servers, operating systems, databases, and storage solutions with products from the domestic industry chain. Advanced AI chips are paramount in this substitution effort, as they directly limit China's computational capabilities and influence cost structures.
In October 2023, the US Department of Commerce's Bureau of Industry and Security ceased supplying China with advanced AI chips, as stated by Commerce Secretary Gina Raimondo in a public address in January, highlighting the US's efforts to prevent China from acquiring the computing power necessary for training large models.
Consequently, Chinese technology companies have struggled to obtain the advanced AI chips required for training large models, leading to significantly higher computational costs. Procuring chips through unconventional channels could risk being added to the Entity List, with costs typically four to five times the official Nvidia prices. The high cost of computing power complicates the industrial application and market penetration of large models.
The US's restrictions on high-end AI chip exports have compelled domestic manufacturers to expedite the adoption of domestic chips, with Huawei emerging as a pivotal entity in this transition.
According to "Caijing Eleven," following the inability to procure Nvidia's A100/A800 and H100/H800 chips through regular channels, the most feasible domestic alternative has been Huawei's Ascend AI chips. In 2023, Huawei's production capacity for Ascend AI chips was estimated to be between 300,000 and 400,000 units. Major technology firms, including Tencent, Baidu, ByteDance, and Ant Financial, have been procuring Huawei's Ascend chips. A high-ranking executive from Huawei Cloud revealed that some internet cloud providers are even utilizing Huawei Cloud's computing power directly to train large AI models.
In November of the previous year, a digitalization director from a large state-owned enterprise group mentioned to "Caijing Eleven" that the company was purchasing Ascend chips in large quantities. Due to high demand, the prices of Ascend chips in November 2023 were more than double those of the past, yet they could secure the products at a preferential rate as key customers.
The primary challenge with domestic AI chips like Ascend lies in the significant gap in the software ecosystem compared to Nvidia, rendering Ascend chips merely functional but not yet optimal. A technician from a technology company noted that Huawei has even dispatched onsite engineers to various enterprises to address compatibility issues with Ascend chips.
A Huawei representative stated that for Ascend chips to transition from being merely functional to highly efficient, it would be ideal to open them up to all technology companies, including Alibaba, Tencent, ByteDance, and Baidu. However, competitive dynamics between Huawei Cloud and other firms have led to a general wariness towards Huawei within the industry. Despite the necessity, there is a reluctance to completely rely on Huawei for chip supply. Building trust with other companies is crucial to bridging the gap in the chip software ecosystem.
The consensus within China's computing power industry is that US sanctions will cause short-term pain, but there is no alternative but to develop and scale up the procurement of domestic AI chips. The strength of these domestic AI chips will directly determine the technological ceiling and cost of China's AI computing power, as well as influence the implementation progress of large models.
5. Where will China's computing industry policy head?
Computing power is increasingly being recognized as a critical factor in national competition. Jensen Huang, the founder of Nvidia, has even asserted that "computing power is power." There is a growing desire among Chinese policy circles to implement industrial policies within the computing power market.
The dispersion and inefficient utilization of China's computational resources are pressing issues for both corporations and the government. Policymakers aim to establish a nationwide "computing power network" to reduce costs, enhance efficiency, and expand usage, with clearly defined roles for each participant. For instance, telecom operators are tasked with constructing network channels and reducing network costs, while computing power service providers supply the resources, and neutral organizations manage the scheduling and transmission of computing power.
One concept still under discussion is whether the computing power market should undergo a separation of generation and distribution, akin to the electricity market. Cloud providers, when confronted with this notion, question its viability, noting that cloud computing inherently integrates computing power with scheduling maturely and efficiently. This proposed separation does not align with market demand realities, especially considering that the US market already has effective solutions, with its three major cloud providers fostering intense competition and maximizing computational efficiency.
A consultant from a state-owned enterprise, during discussions, opined that the electricity market's ability to achieve separation is primarily due to government directives and execution by state-owned entities. The computing power market operates differently, as private enterprises hold the majority of computational resources and data. Thus, only market-driven logic can prevail. For businesses to engage in separation, it must be profitable, rendering the concept feasible in practice.
An expert with experience in formulating information industry policies stated to "Caijing Eleven" in November 2023 that the root cause of China's inefficient computing power usage is the severe fragmentation of the market. Addressing this issue requires adherence to two core principles: computational infrastructure should be market-led rather than government-driven, and computing power is becoming a focal point in the Sino-American strategic competition. The decisive factors will be scale, technology, ecosystem, and chips. At this juncture, the focus should be on expanding platform companies and loosening restrictions on computing power enterprises rather than imposing additional constraints.
The issue of enhancing computational efficiency in China's market is contentious. Alibaba, Huawei, telecom operators, and other computational service providers have varying stances, all intermingled with commercial interests.
Cloud computing firms advocate for sticking to the public cloud path, contending through market competition and technical innovation to establish a market landscape similar to that of the big three US cloud providers. They argue that computing power, due to its technical complexities, should not be standardized like electricity, as this would undermine their technological edge.
Huawei, on the other hand, seeks to align with policy directions, selling more chips and networking equipment while maintaining its cloud business advantages. Huawei adeptly navigates this competitive landscape, pivoting as needed.
Telecom operators, leveraging their network and data centre advantages and state-owned background, aim to capture a larger market share. However, they are reluctant to arbitrarily reduce network costs or invest heavily in network infrastructure for speculative demand.
Computing power transcends mere business and technology issues. Building a competitive edge across the industry is a collective goal, one that will progress through ongoing strategic negotiations among various stakeholders.