@adlrocha - Why AI Is Making Your RAM More Expensive (and what can be done about it)

The supply chain stress, the physics behind it, and five architectural bets on what changes next

A few weeks ago I ended the space data centre piece by asking the question of whether, instead of shipping current GPUs into orbit, we should be rethinking and reviewing the chip manufacturing and architectures entirely.

The answer is almost certainly yes, but while I was working out how to write that post, a more immediate (and somewhat related) problem got in the way: RAM is getting expensive, and it’s going to keep getting worse.

I was listening to Dylan Patel on Dwarkesh Patel’s podcast a few days ago, and (as always) I loved how he broke down the problems that are leading to this mismatch between the demand of memory and the amount being produced.

HBM, i.e. high-bandwidth memory, is a type of memory where several DRAM dies are stacked vertically one on top of another in a chip for space efficiency, in order to reduce power consumption, and most importantly to increase memory bandwidth.

Every modern accelerator uses this approach to memory. The problem is that stacking DRAM into HBM configurations reduces the raw bit density by 3–4x. You’re trading density for bandwidth. When you look at standard planar DRAM (like DDR5 in a desktop), the silicon is packed end-to-end with memory cells. But when you move to the 3D-stacked architecture of HBM, you introduce structural overhead that drastically reduces how tightly you can pack those memory bits.

According to Dylan, to meet demand across the current AI buildout, you’d need roughly 170,000 DRAM wafer starts per month per gigawatt of AI capacity. A demand that current memory manufacturers wouldn’t be able to absorb with their current capacity.

Then there’s the EUV bottleneck upstream of all of it. ASML makes roughly 70 EUV lithography machines per year, ramping towards 100 by the end of the decade. Each one costs around $300–400 million, and they take years to produce, with a reticle stage that operates at 9Gs of acceleration with sub-nanometre alignment tolerances and over 10,000 specialised suppliers (with the corresponding manufacturing challenges and quality controls required).

You cannot simply build more of them on short notice, and you need about 3.5 machines per gigawatt of AI chip capacity. The whole industry is, as Patel describes, stuck in a coordination failure: Nvidia doesn’t expand beyond what TSMC commits to, TSMC doesn’t expand beyond Nvidia’s requests, ASML stays conservative. Everyone assumes the demand projections are exaggerated, and every year, they aren’t with all the AI craze going on with some of these companies not being what Patel calls “AGI pilled” (i.e. that they think the demand for intelligence and thus the underlying hardware will sky-rocket in the coming years).

So turns out energy may not be the only short-term bottleneck that scaling AI may face.

Sidenote: If you like chip manufacturing and economics as much as me, I highly recommend listening to the podcast to understand how complex ASML machines and their supply chain are: from the set of glasses and mirrors used to focus the laser, to the number of passes over the dies, and accuracy required when manipulating them (consider their nm scale), and how they have to be assembled, tested, disassembled, shipped to the factory, and then assembled again before they can start their operation.

Unfortunately, the downstream effect isn’t confined to datacentres. Consumer electronics manufacturers are competing for the same DRAM supply. And this is the key reason for the supply chain stress. AI has completely disrupted the traditional demand for semiconductors, introducing higher margins for fabs (thus making them more appealing) than the semiconductors for phones, cars and consumer electronics that the fabs were dedicated to manufacture before AI.

You’ve probably seen all the memes and real world stories of people trading and hedging their RAM, but we are getting to a point where RAM cards are costing as much as the laptop in which it was assembled, or as much as actual GPUs cards.

Lawrence Lundy-Bryan’s has been lately focusing a lot of his research (and his own fund’s focus) in the field of semiconductors and alternative technological plays to the current “status-quo” of the field. I’ve been following Lawrence’s work at State of the Future for some years now (since the release of the original website). Lawrence interviews the founders of companies working on deep tech, and he tries to paint a realistic picture beyond the hype of when a technology can be expected to be mature enough to be adopted, the different players in the industry and their approaches, and investment opportunities.

I’ve learnt a lot from his pieces throughout the years, and as I was researching the problem of RAM prices I realised that many of the technologies that he has been discussing lately could help tackle this problem. In this post I will do a quick rundown of (what I feel) are the most relevant ones (including the appropriate link to Lawrence’s pieces so you can dig deeper into them). Let’s go!

---

The data movement problem

Let’s start from common knowledge. The memory crunch isn’t just a supply-chain problem. It’s a physics problem, and this physics problem has a name that you’ve probably come across already: the memory wall.

Modern GPUs spend the majority of their cycles idle, waiting for weight data to arrive from HBM (you know that I like to recommend in every post books that I’ve read and that have helped me understanding the problem being discussed, in this case I highly recommend this one to deeply understand the memory wall by learning how massive parallel algorithms for GPUs are implemented).

GPU compute units are fast while data movement in the cheap is slow and expensive. Every token generated by a large language model requires shuffling billions of parameters back and forth between memory and compute, and that shuttle is where most of the energy and most of the latency goes.

Manu Nair, founder of Synthara, makes this point directly in his interview with Lawrence. His framing: “Stop moving data.” Three different architectural approaches are trying to do exactly that:

• Move compute closer to memory, like the case of Groq‘s LPUs which bypass HBM entirely, using 230MB of on-die SRAM with 80 TB/s bandwidth. That’s roughly 100x the bandwidth you get from a GPU’s HBM stack.

• Embed compute within memory, á la Cerebras‘ WSE-3, which puts 44GB of SRAM directly on-wafer at 21 PB/s bandwidth. Lawrence describes that as 7,000x what you get from a single GPU’s HBM stack. OpenAI has reportedly deployed 750MW worth of Cerebras capacity, meaning datacentre power draw, not compute units.

• Redesign algorithms to minimise fetching, the case for DeepSeek’s cost reductions which didn’t come from better chips. They came from a mixture-of-experts architecture that reuses fetched weights more intensively, reducing the total data shuttle budget.

• Eliminate the memory wall entirely, the new trendy approach used by Taalas which takes the most radical position: hard-wire the model directly into silicon so there is no separation between model and hardware. Their tagline is “The Model is The Computer.” Their first chip, HC1, runs Llama 3.1 8B at 17,000 tokens per second per user on a TSMC 6nm die. The weights are encoded in the hardware structure itself, there is no memory bus to cross because there is no separate memory. The trade-off is flexibility: the model is fixed at tape-out. Fine-tuning is supported, but you’re not swapping models at runtime. I have to admit that it’s crazy to test their demo

Actually, you may be aware of the news, but in December 2025, Nvidia acquired Groq’s technology and core team for ~$20 billion, its largest deal on record, folding the LPU technology into the Vera Rubin platform as the Groq 3 LPX inference accelerator. Jensen has said low-latency premium inference should represent about 25% of AI cluster compute, a segment Groq was built to dominate. Nvidia buying Groq is probably the clearest possible signal that the GPU-only inference model has limits, and Nvidia itself didn’t want to bet against. Cerebras and Tenstorrent remain independent, for now, but let’s see how much it takes for them to eventually consolidate into bigger entities (by maybe being bough by one of the frontier labs or the big tech giants?).

Manu’s bet at Synthara is that IP licensing (i.e. design the near-memory compute block, license it to existing chip manufacturers the way ARM does) is more viable than trying to win the merchant silicon market outright. This way one doesn’t need to displace Nvidia, you just need your logic to end up inside Samsung or SK Hynix‘s next memory product.

The memory wall isn’t going away by building more HBM. The physics of shuttling data across a wide bus at high speed is expensive regardless of process node. Something has to change at the architectural level as companies like Groq, Cerebras and Taalas have already shown to some extent. I’m already working on a dedicated post to dig deeper into their architectures and other alternatives to general-purpose GPUs and accelerators, so consider this section a teaser.

---

Gallium Nitride and the photonics bet

One floor up from the memory architecture problem sits the interconnect problem: how do chips talk to each other? Right now the answer is copper (and sometimes gold) traces, but copper has limits, both in bandwidth density and in energy per bit.

The photonics answer that I’ve been hearing since I was in college (we even did some experiments in the lab as part of a course) is using light rather than electrons to move data. It has been circling the industry for years, mostly anchored to silicon. James Lee, founder of Wave Photonics, has a blunter take than most in his interview with Lawrence: silicon is “a really poor photonic material.” It lacks a true electro-optic effect, which means it can’t efficiently encode data onto light. It only operates above 1 micron wavelength, cutting it off from the visible spectrum entirely. The entire silicon photonics ecosystem was built around telecom wavelengths (1310nm and 1550nm) because that’s where glass fibre loses the least signal, but that constraint is weakening.

Two things are changing it. First, short-reach data centre links involving signals travelling metres (not kilometres like in the case of FTTH, Fiber to the Home, another of the hot topics when I was in college) make fibre loss largely irrelevant, opening up the wavelength options. Second, hollow-core fibre transmits through air rather than glass, removing the absorption limits that locked the industry into those telecom bands.

Enter gallium nitride. GaN is the second-highest-volume semiconductor in the world after silicon. You may actually be using it daily unknowingly because it’s part of LEDs, lasers, and RF devices. Unlike silicon, it has a native gain (i.e. you can integrate lasers directly on the chip) and a real electro-optic effect for efficient modulation. It’s transparent from near-UV to far infrared. And it’s radiation-tolerant, which not only matters for space and nuclear applications, but also signals its robustness against certain physical effects.

Lawrence has a genuinely useful comparison table in the GaN piece showing where each photonic material sits across the key dimensions. I’m attaching it below because it’s the clearest single-image summary of the competitive landscape I’ve seen.

The trade-offs between indium phosphide (excellent light generation, tiny supply chain), silicon nitride (ultra-low losses, can’t generate light), thin-film lithium niobate (fast modulation, nearly impossible to process at scale), and GaN map out the whole problem in one view. Wave Photonics’ approach is to automate photonic process design kit development, cutting the time to build a new PDK from six months to three weeks, and use that to pull GaN into markets where silicon photonics has never been able to go.

Gallium Nitride + Photonics w/ James Lee of Wave Photonics

It’s early. GaN photonics is a genuine long shot that will require substantial investment before it shows up in a data centre. But the physics argument is sound, and GaN’s supply chain position means it doesn’t need to build from zero. Developments on GANs could reduce the pressure on the traditional silicon semiconductors supply chain, by diverting some of the demand coming from AI to that of GaN. I highly recommend reading Lawrence’s post about it.

---

Glass, packaging, and the interconnect gap

According to a 2024 IEEE Micro paper on the AI memory wall, computing power has grown ~60,000x over the past two decades, while interconnect bandwidth has grown only ~30x. That gap is why the industry is increasingly focusing on packaging and not fab nodes as the next competitive frontier. I don’t know if you’ve been following the latest reports from Dylan Patel’s SemiAnalysis or Austyn Lyons (another of my go-to recommended semiconductors newsletters), but they’ve been writing about advanced packaging for a while now.

Andrea Rocchetto, founder of Ephos, makes the case for glass. The current alternatives for chiplet interconnect substrates are organic (warps under heat, compromising trace alignment) and silicon interposers (expensive and size-limited). Glass is dimensionally stable at panel scale, Intel has demonstrated substrates of 515mm × 510mm, and it’s manufactured inside the material, which means less clean-room infrastructure than traditional lithography processes (one of the big bottlenecks of the current supply chain as introduced above). It couples naturally with optical fibre, with less than 2% coupling loss.

Rocchetto’s claim is broader than just packaging: “The big chip manufacturing companies of the future are going to be packaging companies.” The performance gains from sub-2nm logic nodes are diminishing (the physics at these levels are brutal). The gains from integrating chiplets efficiently, getting memory closer to compute, getting logic chiplets talking to each other at high bandwidth and low energy, are not. Both Nvidia and Broadcom have moved publicly into co-packaged optics, Broadcom at Hot Chips 2024, Nvidia at GTC 2025 with Spectrum-X Photonics. The direction seems to be clear even if the material winner isn’t.

Glass photonics also benefits from something interesting: the quantum photonics community (Xanadu, PsiQuantum, Quandela) needs the same ultra-low-loss components and minimal coupling losses as classical interconnect. That shared demand base doesn’t make glass photonics a certainty, but it gives Ephos a market to build process maturity in before the larger classical chiplet interconnect opportunity crystallises.

Albeit optimistic, glass packaging could relax the manufacturing requirements of traditional semiconductors lightening the dependency of its supply chain to ASML machines.

---

Carbon nanotubes

Lawrence’s piece on CNTs in the datacentre is one of the more practically grounded things I’ve read on alternative materials. The framing: this isn’t a revolution, it’s a steady 5–10 year substitution across specific applications.

Power density is the main reason that a lot of work and investment is being poured in this material. A Nvidia GB200 NVL72 runs at 120 kW of liquid-cooled rack density. A decade ago, a typical rack drew 10 kW. Some AI clusters spend 40% of total power on cooling. The thermal management problem is, right now, the binding constraint in many deployments. Not compute, not memory, but heat.

CNTs have thermal conductivity superior to copper or diamond along the tube length, 300x the tensile strength of steel by weight, and lighter than aluminium. Today they’re production-ready as thermal interface materials — replacing the thermal paste between a chip and its heat spreader. LG Chem is already using CNTs as battery additives. The economics are straightforward: CNT thermal pads cost around $8 per socket versus $2 for thermal paste, but they last the server lifetime rather than degrading every 18–24 months. For a 100,000-server deployment with dual sockets, that’s $400K in direct savings before you account for the performance gains from better thermal contact.

The longer roadmap, on-chip Cu+CNT interconnects, advanced chiplet packaging, CNT photonics. According to the references post this is more than 5 years out and requires getting commercial CNT fibre conductivity from the current 13–18% of copper up to 70–80%. Development programmes are showing around 61% in lab conditions. It’s an engineering problem, not a fundamental physical limit, which is a more tractable category.

Carbon Nanotubes in the Datacentre

---

Fungible compute and a competition for different resources

The approaches above are all trying to solve the high-end problem: how do you solve the bottlenecks that AI workloads are facing? But there’s a different angle in Pragmatic Semiconductor’s work that Lawrence covers, one that asks whether we’re thinking about the demand side wrong, and that could slightly relax the current stress in the supply chain.

Pragmatic makes flexible integrated circuits using indium gallium zinc oxide deposited on polymer substrates, not the traditional silicon. Their entire 20×30 metre facility produces billions of chips per year. The process takes days to weeks. They’re moving towards modular fabs that can be deployed at customer sites.

The concept that Lawrence coins as “fungible compute” in the article is the following: if chips become cheap enough, computation becomes nearly disposable, embedded in everything without the strategic overhead of “do we really need a chip in this?”. Smart packaging, temperature sensing at item level, agricultural monitoring at plant level, continuous glucose monitors. Imagine how approachable this would make the Farming Lego that I presented in this post.

The market bifurcation he identifies is sharp: bleeding-edge nodes (around the 2nm) for AI datacentres, and cheap ubiquitous computing for billions of objects that have never previously been computational. And I personally love this idea.

This may matter for the memory crunch and the current supply chain stress in a non-obvious way. As described above, part of the pressure on DRAM supply comes from the manufacturing of semiconductors for AI competing with that of consumer electronics.

If we manage to reduce the demand from consumer electronics with alternative (but suitable) semiconductors, the supply chain will be less stressed. Even more, if compute is cheap enough to live at the edge, on the smart label, on the sensor, at the point of measurement, the data movement budget shrinks. Less data needs to cross that expensive bus between memory and processor, because less data leaves the edge in the first place and more computations can be made where the data is sourced. This is a weaker argument from the supply chain perspective, but it may open the door to really interesting use cases where more and more computation is made on the edge.

It doesn’t solve the HBM problem for frontier model training. But it chips away at one of the structural assumptions underneath it.

---

How any of this actually fixes the crunch

If you were looking for the short-term solution to pricey RAMs, I am sorry to disappoint you. I don’t think any of the technologies described above can immediately solve the problem. However, it makes the future of semiconductors (and the real bottlenecks that we are going to face) a bit brighter.

The EUV bottleneck and the HBM density constraint are multi-year problems. ASML isn’t going to produce 200 EUV tools a year by 2028. HBM stacking isn’t going to suddenly recover the 3–4x bit density penalty. And the coordination failure Dylan Patel describes, where everyone downstream assumes demand projections are too high, so nobody invests ahead of need, is a structural problem in an industry with 5-year lead times on fab capacity.

Neither near-memory compute, model-as-hardware, photonic interconnects, CNTs, nor flexible silicon and fungible compute will arrive together or fast enough to solve the memory crunch. Each of them are at different stages of maturity, and may share some of the existing manufacturing bottlenecks (as I guess groq and Cerebras may also depend on TSMC and ASML directly or indirectly to manufacture their chips). However, they may be developed in parallel to TSMC and ASML ramping up of production (which would be good news!).

All the research in the alternatives sections above draws from Lawrence Lundy-Bryan’s interviews at State of the Future. He does the work of sitting down with the actual founders, and pulling out the real physics and trade-offs. Worth following if this space interests you.

I wished I could have gone deeper into each of these technologies. I wanted to briefly introduce all of them while keeping the post bearable. I don’t know if I achieved the goal, but in any case, I am already planning a few monographic posts to explore in more depth some of these technologies, the open problems, opportunities and potential impact with the notes I took working on this one.

Any other technology that I don’t have in the radar and that I should read about? Hit me a message or a comment. Until next week!