Dylan Patel

Dylan Patel says China has gigawatt-scale AI campuses comparable to Abilene, and that their speed, scale, low cost and vertical integration are astonishing.

China has campuses on a similar scale, though few people outside the country have had a look inside. In September, we visited one, and we found the speed, scale, ultra low cost, and deep vertical integration astonishing.

Dylan Patel thinks vertical integration does not yet win hyperscaler orders in China, but could matter more as power gets scarcer and power architectures change.

It is too early to say whether vertical integration will become a meaningful advantage in China’s datacenter market. For now, it gives integrated players more control over cost and schedule, but hyperscalers still choose vendors mainly on resources and track record. That could change as power becomes harder to secure and the power system inside the datacenter is redesigned.

Dylan Patel argues energy access is becoming part of what datacenter developers sell, and power architecture is shifting to high-voltage DC with batteries replacing UPS.

Across different markets, access to energy is becoming part of what a datacenter developer sells. Second, the power architecture from the grid connection to the rack is being redesigned. In the US, the industry is moving toward higher-voltage DC power, while battery backup units (BBUs) and BESS are starting to take over the role of UPS.

Dylan Patel notes China's datacenter market is fiercely competitive, with rents halved, capex down 25%, and Inner Mongolia power among the world's cheapest.

China’s datacenter market is highly competitive. Rents have roughly halved over the past five years, to about $40/kW/month excluding power. Capex has fallen about 25% over the same period, to less than $3mm per MW. Power in Inner Mongolia costs about $0.055/kWh all-in, among the lowest in the world.

Dylan Patel says Google's TPUv7 now matches or beats NVIDIA on perf per dollar in agentic inference, which has pushed NVIDIA into an urgent engineering response.

TPUs delivering better performance per dollar or competitive performance per dollar is a new experience that NVIDIA senior leadership is unwilling to accept. As soon as TPU posted its first AgentX result on Qwen3.5 397B FP8 showing that even TPUv7 FP8 could beat GB300 NVL72 FP4, NVIDIA tasked a strike force of human engineers and agent swarms to work 997, late into the night until 2am local time and on weekends, across USA and mainland China.

Dylan Patel warns that bureaucracy at NVIDIA is slowing its response to the open-source community and is among the biggest risks to its software moat.

There needs to be an cultural shift in massively reducing bureaucracy at NVIDIA. This is one of the biggest risks to NVIDIA Moat that Jensen has to tackle. In our experience and many customers experience, Nvidia is now slower moving than AMD, Google, and Amazon to external suggestion, git commits, and feedback.

Dylan Patel notes that without native FP4, TPUv7 still trails FP4 GPUs on tokens per dollar until TPUv8 arrives in Q1 2027.

FP4 GPUs still deliver more tokens per dollar at most speeds, as TPUv7 has no native FP4 support, and we will need to wait until Q1 2027 for TPUv8 with FP4 support, as tracked in our Accelerator & HBM Model. At 100 tok/s/user, GB300 and GB200 FP4 deliver 3.6x and 3.5x the tokens per dollar of TPUv7.

Dylan Patel says labs with large TPU fleets co-design model attention shapes for TPUs, so a GPU-optimized model like Muse Spark Open understates TPU potential.

Note that AI labs with lots of TPU capacity, such as Anthropic and Google Deepmind and soon OpenAI, typically co-design their models such that the attention shapes would fit nicely on TPUs to avoid this issue.

Dylan Patel predicts NVIDIA may regain a slight lead at FP4 but cannot retake state-of-the-art at FP8 for some models versus TPUv7.

We do think that with enough agent swarms, NVIDIA could regain a slight performance lead on GB300 NVL72 FP4 vs TPUv7 FP8, but on NVIDIA FP8 versus TPUv7 FP8, we believe that it is not possible for NVIDIA to regain SOTA for certain models.

Dylan Patel says SemiAnalysis expects $11 trillion of global AI capex by the end of 2029, with about $7 trillion of it financed by debt, raising the question of who pays.

The other question is sort of who pays for it. So one of the things that we’ve said is by end of 2029 we expect $11 trillion of capex globally on AI.

Dylan Patel argues SpaceX may now be the world's fastest datacenter builder, outpacing even Chinese peers, citing Colossus 2's near-2GW addition in under a year.

We now believe that SpaceX may be the fastest datacenter builder in the world, even more so than Chinese peers. Watch how Colossus 2 adds nearly two Stargate Abilenes (2GW) in just under one year!

Patel says SpaceX's ~13GW 2027 target is unprecedented in scale but feasible, with over 9GW of capacity already identified.

SpaceX has even revised its estimate up to ~13GW in 2027. This would be the largest amount of datacenter capacity ever built by any one company in that timeframe, roughly equivalent to all of Google’s datacenter capacity from 1999-2026 in just 1.5 years. We think it’s possible and have already identified >9GW of capacity.

Patel notes SpaceX has halved typical greenfield datacenter build times to about four to five months, a record at scale.

To our knowledge, this would be the fastest greenfield datacenter ever built at scale. While most are fighting for 8-12-month build times, SpaceX has cut them in half.

Patel attributes SpaceX's speed to elite hardware engineering, lean labor of ~3k workers per GW, and Tesla-made modular MEP equipment.

One thing often overlooked is that the Elon companies are some of the best hardware companies in the world. We mentioned in our previous newsletter that their labor can run at 3k workers per GW thanks to their top-tier engineers. However, we also believe that Tesla is manufacturing modular MEP equipment, allowing SpaceX to deploy so rapidly.

Patel believes Colossus 2 will pair a ~6GWh BESS with gas turbines to cover a roughly 300MW power shortfall after turbine restrictions.

With the BESS nearing 6GWh, we think the site will orchestrate gas and BESS to increase baseload, while the turbines cycle through charging them. If the datacenters are designed right (and our IT MW assumptions are correct), this could work.

Patel has medium-high confidence that SpaceX's TeraFab chip fab project will also include a datacenter, possibly for third-party rental.

Lastly, we have medium-high confidence that TeraFab will have a datacenter too. That’s right, the 10GW, 100M sq.ft chip fab venture from SpaceX will likely have a datacenter.

Dylan Patel says the 2027 datacenter buildout is largely already under construction and the midterms won't derail it, supporting SemiAnalysis's +50 GW global forecast.

The 2027 datacenter boom is already being built and next week’s midterms won’t stop it now. Roughly three-quarters of the 33.8 GW due across our top 20 states is already under construction, which is why we remain comfortable with the global +50 GW year we laid out in our Oct. 16 article.

Patel warns that a prolonged TCEQ permitting freeze in Texas, which accounts for nearly 30% of 2028 US additions, is the biggest risk to the buildout forecast.

The biggest risk to our buildout forecast right now is a prolonged permitting freeze at the TCEQ. Texas alone adds close to 30% of US GW in 2028. A few months of lost time in that market can move the national number.

Patel is less worried about conditions-only candidates because developers can meet the demands and keep building, unlike outright pauses.

We are less concerned with the candidates who support conditions only. Most of their demands change the price and terms of development. A developer can fund the grid upgrade, change the cooling design or negotiate the community agreement and then move forward.

Patel argues moratoriums can block any project still needing approval and shifting development to other states is hard because power, permits and schedules are still required.

A moratorium can block any project that still needs approval, and it won’t be easy to shift development to another state: anywhere else would still require power, permits and construction schedules that can meet the same delivery timeline.

Dylan Patel says chip substrates are the biggest bottleneck for AI infrastructure buildout.

SemiAnalysis CEO Dylan Patel pointed out that chip substrates are the biggest bottleneck for AI infrastructure.

Original (zh)

SemiAnalysis CEO Dylan Patel指出，芯片基板是人工智能（AI）基建最大瓶颈

Dylan Patel argues that, besides memory, substrates and PCBs are the biggest constraint on the semiconductor side, and in his view the biggest bottleneck of all, ahead of memory, optics, power, land and tools.

Yes, I think on the semiconductor side, the biggest constraint, of course, besides memory, is substrates, PCB…

Dylan Patel argues substrates are likely the largest bottleneck in the semiconductor supply chain, ahead of power, land, fabs and equipment, with memory second.

Dylan Patel believes substrates may be the biggest bottleneck in the entire semiconductor supply chain, even larger than power, land, factory buildings and semiconductor manufacturing equipment, with memory close behind.

Original (zh)

基板可能是当前半导体产业链所有瓶颈中最大的一个，甚至超过电力、土地、厂房、半导体制造设备等因素。 在他看来，内存紧随其后，而CPO（共封装光学）和NPO（近封装光学）等新型互连技术的大规模部署，同样面临严峻的供应链挑战。

Patel says substrate and PCB materials were underinvested and cannot expand fast enough, hitting AI accelerators and server CPUs over the next two years.

Patel says that besides memory, one of the biggest constraints is substrates, PCB materials and packaging substrate materials, which were underinvested and whose capacity expansion lags demand; Japanese material suppliers are conservative about large expansions.

Original (zh)

除了内存之外，半导体产业链面临的最大约束之一，就是基板、PCB材料以及封装基板材料。这些环节此前投资不足，产能扩张相对滞后，难以迅速匹配当前的需求增长。

Dylan Patel argues substrates are the largest bottleneck in the semiconductor supply chain, exceeding power, land, fabs and equipment, with memory second.

SemiAnalysis founder Dylan Patel recently discussed the AI infrastructure supply chain and offered a pointed judgment: substrates may be the biggest bottleneck in the entire semiconductor supply chain, even larger than power, land, fabs and semiconductor manufacturing equipment. In his view, memory comes right behind.

Original (zh)

SemiAnalysis创始人Dylan Patel近日谈及AI基础设施供应链时，提出了一个颇为鲜明的判断：基板可能是当前半导体产业链所有瓶颈中最大的一个，甚至超过电力、土地、厂房、半导体制造设备等因素。 在他看来，内存紧随其后

Patel says CPO and NPO face severe capacity ramp challenges, and the supply chain may not meet demand even if only one large firm deploys at scale in 2028.

Patel believes CPO and NPO face enormous capacity ramp challenges. Even if all companies wanted to adopt CPO in 2028, or only one large company decided to deploy at scale that year, there is no guarantee the supply chain could meet its demand.

Original (zh)

Patel认为，CPO和NPO面临着巨大的产能爬坡挑战。即便所有企业都希望在2028年采用CPO，或者只有一家大型企业决定在当年大规模部署，也无法保证供应链能够满足其需求。

Patel says substrate and PCB materials were under-invested, with lagging capacity and conservative Japanese suppliers, and will constrain AI accelerators and server CPUs over the next two years.

Patel pointed out that beyond memory, one of the biggest constraints facing the semiconductor chain is substrates, PCB materials and packaging substrate materials. These segments were previously under-invested and capacity expansion lagged. Many Japanese firms are conservative and not keen on large-scale expansion.

Original (zh)

Dylan Patel指出，除了内存之外，半导体产业链面临的最大约束之一，就是基板、PCB材料以及封装基板材料。这些环节此前投资不足，产能扩张相对滞后，难以迅速匹配当前的需求增长。

Dylan Patel says the hard bottleneck for AI infrastructure has shifted from power to chip packaging substrates, which will cap accelerator shipments over the next two years.

At the CASPA annual conference on October 23, SemiAnalysis founder Dylan Patel pointed out that the hard bottleneck in AI infrastructure buildout has shifted from power to chip packaging substrates. The substrate, PCB material and upstream material chain has long suffered from severe underinvestment, and with Japanese core suppliers conservative about expansion, global AI accelerator shipments over the next two years will be capped directly by substrate capacity.

Original (zh)

SemiAnalysis创始人Dylan Patel在10月23日的CASPA年度大会上指出，当前AI基础设施建设的硬性瓶颈已从电力资源转移至芯片封装基板。

Dylan Patel says chip substrates are the most severe bottleneck in the AI infrastructure supply chain, worse than power, land or fab equipment.

Patel pointed out that the chip substrate is currently the most severe bottleneck in the AI infrastructure supply chain, with an impact even greater than land, power and semiconductor manufacturing equipment.

Original (zh)

Patel 指出， 芯片基板是目前AI基础设施供应链中最严重的瓶颈，影响程度甚至超过土地、电力及半导体制造设备等环节 。

Patel says chip substrates are the biggest bottleneck in AI infrastructure, more severe than land, power or equipment, due to long underinvestment and cautious Japanese suppliers.

Chip substrates are currently the most severe bottleneck in the AI infrastructure supply chain, with an impact even greater than land, power and semiconductor manufacturing equipment. Because substrates and related materials have been underinvested for a long time and Japanese suppliers are cautious about expanding capacity, capacity expansion for AI accelerators and CPUs is constrained.

Original (zh)

Patel 指出，芯片基板是目前AI基础设施供应链中最严重的瓶颈，影响程度甚至超过土地、电力及半导体制造设备等环节。 由于芯片基板及相关材料过去长期投资不足，加上日本供应商扩产态度谨慎，AI加速器与CPU的产能扩张受到限制。

Patel's firm estimates nearly 50 GW of data centers under construction globally in 2027, and notes power gaps can be eased with on-site turbines and fuel cells while chip supply cannot scale fast.

Power shortages still have other solutions. Operators can deploy on-site gas turbines and fuel cells to partly ease power constraints, but expanding the semiconductor supply chain takes longer and supply cannot be increased quickly.

Original (zh)

相较之下，电力短缺仍有其他解决途径。SemiAnalysis预估，2027年全球在建数据中心规模将接近50吉瓦（GW），其中美国约占35吉瓦。业者可通过部署现场燃气涡轮机、燃料电池等设备，部分缓解电力供应限制；但半导体供应链扩产需要更长时间，难以迅速增加供给。

Patel estimates the cost of 1 GW of AI compute infrastructure rose from about $40B in 2024 to over $60B now.

Patel estimates that in 2024 the total cost of building 1 GW of compute infrastructure was about $40 billion, and it now exceeds $60 billion, an increase of more than 50% in two years.

Original (zh)

Patel估算，2024年建设1吉瓦算力基础设施的总成本约为400亿美元，如今已超过600亿美元，两年内涨幅逾50%。

Patel says chip substrates are the most severe bottleneck in the AI infrastructure supply chain, worse than land, power, or semiconductor equipment.

Patel 指出，晶片基板是目前 AI 基礎設施供應鏈中最嚴重的瓶頸，影響程度甚至超過土地、電力及半導體製造設備等環節。

Patel says Nvidia's share of global HBM and CoWoS resources is declining in 2026 and will fall further in 2027 as Google, Amazon and AMD gain share.

Patel 指出，2023至2024年間，輝達占據全球相當高比例的 HBM 及台積電(2330-TW) CoWoS 先進封裝資源，但這一情況已開始改變。

Patel estimates that excluding chip designers' margins, 1GW of compute could theoretically cost about $20B less, motivating hyperscalers to build in-house chips.

Patel 估算，若完全扣除輝達、博通(AVGO-US) 等晶片設計商的利潤，建置1吉瓦算力基礎設施的成本理論上可降至約200億美元

Patel says packaging substrates are the biggest AI supply chain bottleneck, likely to constrain AI accelerator and CPU output for the next two years, worse than land, power or equipment.

Apart from memory, packaging substrates, PCB materials and other substrate materials are the biggest constraints facing the semiconductor supply chain. Over the next two years, AI accelerators, CPUs and all kinds of electronic components may be affected by the substrate shortage, a bottleneck even more severe than land, power, data center space and semiconductor equipment.

Original (zh)

他表示，未來兩年，AI 加速器、CPU 及各類電子元件都可能受到基板供應不足影響，這項瓶頸甚至比土地、電力、資料中心機房與半導體設備更為嚴重。

Patel ranks optical interconnect after memory as a bottleneck, saying CPO laser and fiber-coupling supply may fall short even in 2028.

Even by 2028, if the large players all want to adopt CPO, the supply of key components such as lasers and fiber coupling may still fail to meet the needs of any single large player.

Original (zh)

即使到了 2028 年，若大型業者都希望導入 CPO，雷射與光纖耦合等關鍵元件的供應，仍可能無法滿足任何一家大型業者的需求，成為 AI 基礎建設持續擴張的另一項限制。

Patel argues power is overemphasized as a bottleneck since on-site turbines, engines and fuel cells offer alternatives and power can scale faster than semiconductors.

Power supply capacity can be raised in a relatively short time, and overall build complexity is far lower than the semiconductor supply chain, so although power draws great market attention, it is not necessarily the hardest constraint to overcome right now.

Original (zh)

他指出，電力供應能力可以在相對較短的時間內提升，整體建置複雜度也遠低於半導體供應鏈，因此電力問題雖然受到市場高度關注，卻未必是目前最難突破的限制。

Patel says chip substrates, not power, are the biggest bottleneck in AI infrastructure, since Japanese suppliers are expanding cautiously.

Patel said the biggest supply chain bottleneck facing AI infrastructure buildout is not power but chip substrates. Because Japanese suppliers are cautious about expanding capacity, supply of key materials is hard to increase quickly.

Original (zh)

Patel 指出，當前 AI 基礎設施建設面臨的最大供應鏈瓶頸並非電力，而是晶片基板。由於日本相關供應商擴產態度謹慎，關鍵材料供應難以迅速增加。

Patel estimates the cost of 1 GW of AI compute has risen from about $40B to over $60B in two years.

Patel estimated that the total cost of building 1 GW of compute infrastructure was about $40 billion in 2024 and now exceeds $60 billion, an increase of more than 50% in two years.

Original (zh)

Patel 估算，2024年建置1吉瓦算力基礎設施的總成本約為400億美元，如今已超過600億美元，兩年內漲幅逾50%。

Patel says Nvidia's share of CoWoS and HBM resources is falling while Google, Amazon and AMD gain share.

Patel said that in 2023-2024 Nvidia held a very high share of global HBM and TSMC CoWoS advanced packaging resources, but this has begun to change. In 2026 Nvidia's share of global CoWoS wafers and HBM has fallen and is expected to decline further in 2027, while the shares of Google, Amazon and AMD gradually rise.

Original (zh)

Patel 指出，2023至2024年間，輝達占據全球相當高比例的 HBM 及台積電(2330-TW) CoWoS 先進封裝資源，但這一情況已開始改變。 2026年，輝達使用的全球 CoWoS 晶圓與 HBM 資源占比已經下降，預計2027年還會進一步下滑，與此同時，Google、亞馬遜及超微半導體(AMD-US) 等競爭對手的占比則逐步提高。

Patel says removing chip designers' margins could cut 1 GW cost to about $20B, motivating hyperscalers' custom silicon.

Patel estimated that if the profits of chip designers such as Nvidia and Broadcom were fully deducted, the cost of building 1 GW of compute infrastructure could theoretically fall to about $20 billion, highlighting the incentive for big tech to build their own chips.

Original (zh)

Patel 估算，若完全扣除輝達、博通(AVGO-US) 等晶片設計商的利潤，建置1吉瓦算力基礎設施的成本理論上可降至約200億美元，凸顯大型科技企業透過自研晶片降低外部採購成本、減少對供應商依賴的動機。

Patel says chip substrates, not power, are the biggest bottleneck for AI infrastructure, because Japanese suppliers are expanding cautiously.

Patel pointed out that the biggest supply chain bottleneck in current AI infrastructure buildout is not power but chip substrates. Japanese suppliers are cautious about expanding capacity, so supply of key materials cannot rise quickly.

Original (zh)

Patel指出，当前AI基础设施建设最大的供应链瓶颈并非电力，而是芯片基板。日本相关供应商扩产谨慎，导致关键材料供应难以迅速增加。

Patel expects Nvidia's share of CoWoS and HBM resources to keep declining in 2027 as Google, Amazon and AMD gain share.

In 2026, Nvidia's share of global CoWoS wafer and HBM consumption has declined, and Patel expects it to fall further in 2027.

Original (zh)

2026年，英伟达消耗的全球CoWoS晶圆和HBM资源占比已经下降，Patel预计2027年还将进一步下降。

Patel estimates the cost of 1 GW of AI compute has risen from about $40B to over $60B in two years.

Patel estimates that the total cost of building 1 GW of compute infrastructure was about $40 billion in 2024 and now exceeds $60 billion, an increase of more than 50% in two years.

Original (zh)

Patel估算，2024年建设1吉瓦算力基础设施的总成本约为400亿美元，如今已超过600亿美元，两年涨幅超过50%。

Patel notes that removing chip designers' margins would cut 1 GW cost to about $20B, showing the incentive for hyperscalers to build custom chips.

Patel estimates that if the profits of chip designers such as Nvidia and Broadcom were removed entirely, the cost of building 1 GW of compute infrastructure could theoretically fall to about $20 billion.

Original (zh)

Patel估算，如果完全剔除英伟达、博通等芯片设计商的利润，建设1吉瓦算力基础设施的成本在理论上可降至约200亿美元。

Patel says Trainium is proven in inference, with Anthropic running production inference and RL rollouts on it, but has historically been a poor fit for research training.

Trainium has already proven itself in the realm of inference. Anthropic is running production inference and RL rollouts on Trainium2 and Trainium3. Historically, Trainium has been great for inference but not the best fit for research training.

Patel says Trainium3 is far from CUDA's maturity but is heading in the right direction, progressing quickly thanks to agentic coding.

It’s nowhere close to the maturity of the CUDA ecosystem, but the direction is right. Their team is making rapid progress with the help of agentic coding.

Patel believes Amazon's shift to native PyTorch via TorchNeuron will meaningfully broaden Trainium3 adoption.

Amazon has made a major course correction to its software strategy, and we believe it will meaningfully broaden Trainium3 adoption. TorchNeuron represents a native PyTorch experience instead of duct-taping Trainium onto PyTorch through the PyTorch/XLA project.

Patel says Trainium's Neuron Explorer profiler is state of the art and better than Nvidia's and AMD's profilers.

Trainium’s profiler is by far leading the industry and is SOTA status.