The Scramble for AI Computing Power

“What is your moat?” That’s Silicon Valley-speak for “what defends you from the competition.” As investors hunt for the next big AI company, it’s also one question that the hundreds of start-ups launched in the wake of ChatGPT increasingly can’t avoid.

How do you profit off intelligence once it’s been commoditized? Will the AI transition let a thousand flowers bloom, or will the returns largely flow to a few tech behemoths and their infrastructure providers? If there is anything we’ve learned from the social media era, it is that the rules governing AI today have the potential to shape the distribution of economic and cultural power for decades to come. We better get it right.

The way value gets captured in the post-AI economy has implications for domestic competition as well as America’s technological com­petition with China. Just as AI could lead to monopolization domestically, the first country to develop AI systems advanced enough to auto­mate most existing forms of human labor could unlock productivity growth so explosive as to secure indefinite economic and technological supremacy. Alternatively, AI’s deflationary effects could paradoxically undermine U.S. economic leverage by eroding key areas of comparative advantage—higher education, cultural exports, financial services, and R&D—while pushing value into a handful of scarce inputs over which we have limited control.

Artificial intelligence comes in many flavors, but what sets modern systems apart is their dependence on large amounts of computing power. Take large language models. By predicting text sequences from large corpora of training data, systems like ChatGPT not only discover the rules of natural language, but also learn common sense reasoning and other forms of abstract thought. There’s only one catch: the computing cost required to train a model grows exponentially with its raw capability.¹

ChatGPT was created by OpenAI, one of only a handful of compa­nies with the technical talent and data centers (courtesy of Microsoft) needed to train frontier models—best-in-class language, image, and audio models that developers can then build apps on through an applica­tion programming interface (API). Yet if you want to disparage a start-up founder, just call their new application a “wrapper on GPT-4.” Developers can only get so rich building appendages on a technology that someone else controls. Like a remora fish attached to the underbelly of a basking shark, where goes the API, so goes your company. You have no moat. You are, in a word, replaceable.

AI’s stark implications for market power were brought home last year when a pitch deck from OpenAI’s chief competitor, Anthropic, found its way online.² The presentation revealed the company’s billion-dollar, eighteen-month plan to train a frontier AI model ten times more powerful than GPT-4—the digital brain behind OpenAI’s ChatGPT. What caused heads to turn in Silicon Valley, however, was how An­thropic laid out the stakes: “These models could begin to automate large portions of the economy,” the deck reads. “We believe that companies that train the best 2025/26 models will be too far ahead for anyone to catch up in subsequent cycles.”

It’s always worth taking claims geared toward prospective investors with a hefty grain of salt. The company with the best AI model in a few short years will gobble up whole sectors of the economy and leave their competitors in the dust? Talk about “big, if true.”

But suppose it is true. The best multimodal models can already do everything from pass the bar exam at the 90th percentile to autonomously plan and book your next vacation. By some estimates, over half of the code programmers produce is now AI generated. And while the current generation of models suffers from certain limitations—the propensity to hallucinate facts, the lack of a long-term memory—researchers are working furiously to iron out the remaining kinks.

In the very short run, AI will largely augment the work we already do. Average programmers with a coding copilot can become 10x software engineers; doctors with a medical chatbot can get an instant second opinion; and lawyers can use customized models to draft docu­ments and summarize evidence, letting them take on more clients. Overtime, however, AI is trending toward agent-like systems that surpass human experts at a wide variety of tasks, if not entire categories of work. And while Anthropic’s timeline may be ambitious, it is consistent with independent forecasts that project the arrival of AIs competitive with most college-educated labor around 2026.³ What hap­pens next is anyone’s guess.

In March 2023, researchers at OpenAI released estimates of the likely labor market impact from the current generation of GPT models.⁴ Their findings indicate “approximately 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of GPTs, while around 19% of workers may see at least 50% of their tasks impacted.”

If these estimates pan out, AI will be a massive boon for both pro­ductivity and some measures of income equality, as the jobs facing near-term automation span “all wage levels, with higher-income jobs poten­tially facing greater exposure.” Nonetheless, if proprietary models continue to crush open-source alternatives in their power and reliability, these same estimates raise the spectre of a significant cross-section of GDP suddenly flowing through models created by a single, dominant company. This is why OpenAI made the unusual decision to cap their investors’ profits at 100x, while Anthropic plans to shift control of its “public benefit corporation” to a board of trustees shielded from the profit motive.⁵ A world where the first company to build truly super­human AI realizes unbounded market power is, by these companies’ own admission, a real possibility.

The economy is made of people, at least for now. But if AI progress continues at its current pace, “what is your moat” could soon become a question humans start asking themselves.

The Geopolitics of Chips

In The Wealth of Nations, Adam Smith noted an apparent paradox: water is essential to life but nearly free to consume, while diamonds are mostly useless but exorbitantly priced. The resolution to the paradox is to realize that water is plentiful while diamonds are rare, and market prices simply reflect that relative scarcity. (The wrinkle is that diamonds are kept artificially scarce because a single company, De Beers, has his­torically controlled over 80 percent of the world supply, but leave that aside.)

Futurists have long dreamt of AI ushering in a “post-scarcity” world, but such a thing does not exist. Even in a world where most labor is automated, value will continue to flow to what remains scarce: the capital. For AI, that means the owners of large data centers and leading chip makers.

Demand for semiconductors already vastly outstrips supply, par­ticularly for the specialized hardware needed to efficiently train and run the most advanced models. The top chip designer, Nvidia, controls 80–95 percent of the market for the most advanced AI chip designs and has thus seen its stock price rise over 400 percent in just the past five years. With an interconnect bandwidth of nine hundred gigabytes per second (the rate individual chips share information with their supercomputing neighbors), Nvidia’s flagship H100 tensor core GPU is a technological marvel—surpassed only by the company’s newest chip family, Blackwell, which can pack a petaflop of computing power into a single GPU. Nvidia’s GPUs are also the result of one of the most complex and closely guarded design and manufacturing processes in human history—in other words, a moat.

Nvidia just designs the chips and the software to run them. The actual fabrication occurs at TSMC—a factory whose literal moat, the Taiwan Strait, provides only 110 miles of separation from mainland China. With an AI transformation on the horizon, access to advanced chips has thus taken on the crushing gravity of geopolitics.

In a bipartisan show of techno-nationalism, Congress allocated $54 billion to the rebuilding of America’s domestic chip-making capacity in the chips and Science Act of 2022. Multiple U.S.-based semiconductor projects are now underway or under consideration that represent capital expenditures of over $260 billion through 2030. Nevertheless, federal grants have been slow to move given bureaucratic inertia and the litany of mandates imposed on awardees. Delays have thus ensued, from Intel’s $20 billion chip factory in Ohio to the first of Samsung’s eleven planned fabs in Texas. TSMC’s $40 billion fab in Arizona was even forced to spend months wrangling with the local construction union after bringing in five hundred Taiwanese workers with the highly specialized skills needed to wire up semiconductor “cleanrooms”—skills local construction workers simply lack. While some have blamed the setbacks on the Act’s DEI provisions⁶ (from minority set-asides to workforce training programs for “justice-involved individuals”) they more broadly reflect what legal scholar Nicholas Bagley has dubbed the “procedure fetish” afflicting the U.S. government at every level.⁷

As if to buy time, the U.S. government, in concert with Japan and the Netherlands, followed up the chips Act by imposing sweeping export controls on the sale of advanced AI chips and semiconductor manufacturing equipment to China. The message is clear: if AI is the ultimate winner-take-all technology, anything that stymies China’s access to the most advanced chips—and bolsters our own—is imperative to U.S. national security.

One gets the sense that this is only the start. As the main currency in a post-AI economy, the future will be determined by those with access to large computing clusters and the energy needed to power them. Those clusters will ideally be located in the West, but with the monopoly risk looming in the background, it may not suffice to cede control to purely private hands. The power unleashed by future AI models will challenge our basic governance structures to their core, busting through decadent procedures and driving demands for new controls over the distribution of compute—if not outright public ownership.

Nationalization is certainly one answer to AI’s monopoly problem. On our current trajectory, it may even be a likely one. Yet Nvidia, for its part, has no interest in becoming a national champion, as China represents an enormous market for its GPUs. Shortly after export controls were introduced on the high-bandwidth GPUs used for train­ing large AI models, Nvidia unveiled new chip designs—the A800 and H800—tailored for China, with specs tweaked to fall just under the line. A year later, the Bureau of Industry and Security (the home of the U.S. Export Enforcement Office within the Department of Commerce) was forced to update the controls to retroactively account for Nvidia’s workaround. The latest controls are incredibly strict, including a new “performance density threshold” that is essentially impossible to game.

The challenge that remains is in combating chip smugglers and other strategies to circumvent the controls outright.⁸ For its part, the BIS operates with woefully outdated information systems, policing its entity list and export licenses from the barrel of a giant Excel spreadsheet. Given a stagnant budget and myriad other responsibilities, such as implementing Russian sanctions, it is no wonder that BIS has become overwhelmed. The agency only recently hired its second Mandarin speaker, and for a period had just one full-time staff member who knew how to operate the Federal Register system—the system used for all regulatory updates.⁹

And while the BIS could use more funding, what it really lacks is technology. In an ideal world, BIS analysts would have access to ma­chine learning tools to process license applications, track export flows throughout the supply chain, and identify Chinese intermediaries and other anomalies in real time. Such systems will become essential in the years ahead as the number of chips produced above the export-con­trolled performance threshold skyrockets. Fortunately, fully funding the BIS’s modernization plan would only require additional congressional appropriations of about $45 million a year over three years¹⁰—a drop in the bucket when we’re spending a thousand times that amount on the chip fabs themselves.

More immediately, there are also glaring gaps in the BIS’s statutory authority. While training advanced AI models requires access to large computing clusters, there is little under current law to prevent Chinese entities from training their models in the data centers of friendly third countries—what’s known as the “remote access loophole.” Consider that in the three months leading up to the introduction of the chip export controls in October 2022, Nvidia made just over half a billion in sales to companies in Singapore. A year later, Nvidia’s third quarter sales to Singapore jumped over fivefold to $2.7 billion; $4.5 billion over the preceding nine months.¹¹ In essence, companies within Singapore’s booming cloud sector appear to be importing H100s and other chips banned for export to China, only to grant Chinese firms de facto access through the cloud. This is separate from the chips that find their way into Chinese tech companies directly.¹² Fortunately, closing the remote access loophole is relatively straightforward: Congress could pass the cloud ai Act, a bipartisan bill introduced in the House that would simply extend the existing chip export controls to cloud services that make use of those same chips.¹³

Longer term, the only failsafe way to prevent chip smuggling is to implement restrictions on the chips themselves. An on-chip mechanism could, for example, be used to limit a chip’s functionality without a valid access key and IP address, allowing the original seller to track the chain of custody when a chip is re-exported. A recent report from the Center for a New American Security, “Secure, Governable Chips,” makes the case that adopting such on-chip mechanisms will be critical for U.S. national security, while walking through the technical challenge of en­suring they aren’t easily removed or tampered with.¹⁴

Beyond allowing export controls to scale, on-chip mechanisms could also enable a degree of “compute governance” to be baked into AI hardware—not as a covert backdoor, but as a transparent and robust way to preempt rogue actors from, say, running AI-powered botnets or generating DNA sequences for novel pathogens. Secure and local moni­toring of the types of computations performed on a chip presents some technical challenges, but is not without precedent. Nvidia, for instance, once used a firmware update to detect the characteristic hashing algo­rithm used by the Ethereum blockchain to nerf the utility of certain graphics cards for crypto mining.¹⁵ While Sam Altman has called for the creation of an “International Atomic Energy Agency” for global AI oversight, that’s easier said than done. Fortunately, unlike uranium, the hardware inputs into advanced AI models are fully programmable.

The main risk from strict export controls is that they could merely accelerate the development of China’s indigenous chip sector. Building a domestic semiconductor industry has been a goal of the Chinese govern­ment since at least 2000, when its state-backed foundry, the Semiconductor Manufacturing International Corporation (SMIC), first broke ground. SMIC has even had some recent success in repurposing older equipment to fabricate cutting-edge chips for Huawei phones. Nevertheless, most industry analysts agree that China is still many years behind the likes of Nvidia and TSMC.¹⁶ In the meantime, export con­trols are clearly taking their toll. SMIC has had to postpone production at its newest facility by at least six months due to “difficulties in securing key equipment,”¹⁷ while the controls have forced painful layoffs at one of China’s biggest chip makers, Yangtze Memory Technologies Co.¹⁸ The stringency of the controls will also be felt increasingly over time as chip performance continues on its exponential trajectory. In turn, if AGI-like systems are truly possible this decade, even a modest setback to China’s compute capabilities could end up being decisive.

If there is any one major blind spot in the U.S. compute strategy, it is an excessive focus on the highest-value-added technologies. Since 2020, SMIC has announced four new facilities for fabricating the sorts of humdrum chips that go into cars, televisions, and tanks.¹⁹ And despite being banned or restricted in the United States, Canada, and most of Europe, Huawei products have only deepened their penetration into telecom infrastructure worldwide thanks to China’s digital Belt and Road Initiative.²⁰ China’s strong market position in legacy chips and telecommunications equipment reflects the same playbook they’ve exe­cuted for solar panels and electric vehicles: drive down prices in mature technology categories and then flood cost-sensitive emerging markets. Thus, even if the United States wins the race to AGI, the ubiquitous nature of Chinese networking infrastructure throughout Africa and Asia could give China indirect control over how—or whether—frontier AI models get deployed.

Governance Innovation

The United States is currently primed to be the leader in both AI and global AI governance thanks to its enormous compute advantage over every plausible nation-state competitor.²¹ U.S.-based firms own 70 per­cent of the global cloud computing market, with the bulk of that market share split between just three companies: Amazon, Microsoft, and Google. Between the AI governance applied by these compute intermediaries and our latent capacity to control foreign access to AI hardware, the standards adopted in the United States thus have the potential to be exported worldwide.²²

If the United States fumbles this lead, what will be the reasons? At first order, we would only have ourselves to blame: we could fail to invest in the basic administrative capacities needed to enforce export controls at scale; our fetish for regulatory procedure could postpone the chips Act and related public investments into oblivion; and our first-world complacency and internecine political squabbles could distract us from the urgency of the moment.

At second order, our existing advantages could be disrupted by the sheer pace of technological change. Take Loudoun County, Virginia, known as “Data Center Alley” for being home to the largest concentration of data centers in the world. The fact that a significant fraction of global internet traffic passes through Virginian data centers represents a substantial asset for U.S. intelligence agencies and for the U.S. role in internet governance more broadly. At the same time, it’s an advantage that could easily slip away as AI upends older computing paradigms. Given the power-hungry nature of GPU workloads, new computing clusters are increasingly colocating on company premises or migrating to wherever energy is cheapest, including offshore. Given the current rate of compute buildout, compute clusters optimized for AI could conceivably surpass the aggregate compute hosted on the public cloud in a matter of years. And indeed, in March of this year, Loudoun County’s Board of Supervisors denied a new datacenter application for the first time in its history, citing (among other concerns) its projected energy usage of six hundred megawatts—enough to power roughly a quarter million homes.²³

The projected energy demands from AI are so substantial that compute providers such as Microsoft and Amazon have plans to vertically integrate their own power generation.²⁴ This would not be so necessary were it easier to build new transmission lines and get power directly from the grid. Inadequate grid infrastructure has thus accelerated the trend of moving computing clusters on-premises, and thus the risk that AI governance balkanizes down the line.²⁵

More speculatively, technical breakthroughs in the coming years could render many of the assumptions underlying the compute econ­omy as we now know it obsolete. The popularity of the transformer architecture for AI models, for instance, owes as much to its intrinsic properties as to its scalability on the GPU hardware that happened to be available at the time of its discovery.²⁶ In the future, “hardware-agnos­tic” algorithms and model distillation techniques could emerge that make Nvidia and TSMC’s lead over Chinese chip makers far less rele­vant. Conversely, the energy demands from GPUs could cause exotic alternatives to the semiconductor to emerge,²⁷ such as analog or energy-based chips that run neural networks at ultra-low power, inducing a platform shift.²⁸

Last but not least, U.S. dominance in compute could simply fail to compensate for China’s numerous advantages in data collection and political coordination. This is most obvious in the area of computer vision, where China has long been competitive with the United States given the prerogatives of its surveillance state, but it arguably applies to biology and manufacturing as well. BGI Group, for example, has used its prenatal screening service to amass an enormous database of Chinese genomes, which will no doubt prove useful for AI-enabled drug dis­covery and personalized medicines. Similarly, the skilled workers in Chinese factories represent a latent source of training data for imbuing general purpose robots with the tacit knowledge needed to automate complex manufacturing processes, suggesting China’s industrial prowess positions them to translate our innovation in bits into their innovation in atoms. All the while, the capacity of the Chinese state to turn their society on a dime will likely enable a much faster diffusion of any AI application—both utopian and dystopian—that requires complementary investments and high degrees of system-level coordination.

If the United States has any risk of losing its AI lead to China, in other words, it will be because we failed to innovate our governance fast enough, not our technology. To defend our moat while we still have it, U.S. policymakers must learn to expect the unexpected. As the late Intel founder, Andy Grove, put it in Only the Paranoid Survive, “The person who is the star of a previous era is often the last one to adapt to change, the last one to yield to the logic of a strategic inflection point, and tends to fall harder than most.”

This article originally appeared in American Affairs Volume VIII, Number 2 (Summer 2024): 86–96.

Notes