Seems like a deliberate strategy to me. They've been progressively improving the performance of on-device AI for a while now. Model-training could turn out to just be a bad business, why take a chance on it when Apple's core competency is hardware? Wrote a similar piece last November here https://substack.com/@joshfrance/p-179685404
Agreed it seems deliberate on unified memory. Unlikely on the LLM open-weight trajectory, though.
Unified memory came at a time when it was already very clear "plain ML" with convolutional networks was slowed down by memory movements (Apple is concerned by the so-called inference, not really training, but the movement problem was on both). Unified memory was in the air 10 years ago already, as an option to avoid moving things around from memory bank to another bank. Other options like HBM and faster DRAM also appeared, but imply more costs, and more importantly more physical space---pretty much what Apple keeps iterating over to improve (thiner laptops and phones).
On top of unified memory, their architectural choice for comparatievely fast memory bandwidth seems consistent with their history (back to the 80-90s attitude even), and I was suprised they also score very well on metrics like TOPS/W. But they did something else too, with software, perhaps in the Metal layer, where the runtime seems capable of allocating smartly payloads to either CPU, GPU or their ML accelerator (aka NPU, modulo quantisation). AMD has similar software now on its Ryzen AI chips (same with CPU/GPU/NPU), but it seems more limited (despite coming apparently years later) where what cannot fit the NPU is automatically run on CPU.
It's an accident on Apple's part, but that falls in line with a lot of accidents that have benefited Apple over the years. I worked there in the 80's and 90's. That period were Jobs was out and a succession of CEO's bumbled their way through slowly bringing the company near bankruptcy until the accident of Job's return, with a UNIX based OS, reappeared and, literally, saved the company from extinction.
I won't go through all the other accidents Apple stumbled upon, but, I'm pretty sure there's some sort of invisible force in the universe looking out for that company, and tht's coming from a bonified atheist. Nothing else explains how a company that's done so many stupid things can still be so successful.
Interesting points. My feeling is they have a lucky position. Because when they built apple intelligence, they went the route of the "private" cloud option. But there is a nascent market: AI inference hubs - somewhere between a mac mini and studio optimized for AI inference, sitting in your home or office and connected to your devices. Everything private. I do not understand why a hardware company like Apple isn't going for this.
> These were of course non-binding. Micron, reading the demand signal, shut down its 29-year-old Crucial consumer memory brand to redirect all capacity toward AI customers. Then Stargate Texas was cancelled, OpenAI and Oracle couldn’t agree terms, and the demand that had justified Micron’s entire strategic pivot simply vanished. Micron’s stock crashed.
Exactly. The moat wasn’t the keynote claim. It was the side-effects: habit, switching costs, trust, and all the small frictions that made leaving feel expensive. If you only optimise for the stated goal, you miss the by-products that become the real asset.
I think this is an interesting idea. But one thing I am not sure is that if the scale law of llm is still effective. If the law is effective now, local model will not chase chatgpts. Hope to hear more good insights.
I think that’s the real question. If scale laws still dominate, local models probably won’t win on raw capability. But they can still win on packaging: latency, privacy, offline use, integration, and trust. The side-effect can become the moat even if the frontier stays ahead on the core metric.
Great complement to this post and this one from a few weeks ago (https://adlrocha.substack.com/p/adlrocha-why-ai-is-making-your-ram). It looks like Apple is also becoming a victim of RAMpocalypse: https://www.macrumors.com/2026/04/11/some-mac-mini-mac-studio-currently-unavailable/
Seems like a deliberate strategy to me. They've been progressively improving the performance of on-device AI for a while now. Model-training could turn out to just be a bad business, why take a chance on it when Apple's core competency is hardware? Wrote a similar piece last November here https://substack.com/@joshfrance/p-179685404
Agreed it seems deliberate on unified memory. Unlikely on the LLM open-weight trajectory, though.
Unified memory came at a time when it was already very clear "plain ML" with convolutional networks was slowed down by memory movements (Apple is concerned by the so-called inference, not really training, but the movement problem was on both). Unified memory was in the air 10 years ago already, as an option to avoid moving things around from memory bank to another bank. Other options like HBM and faster DRAM also appeared, but imply more costs, and more importantly more physical space---pretty much what Apple keeps iterating over to improve (thiner laptops and phones).
On top of unified memory, their architectural choice for comparatievely fast memory bandwidth seems consistent with their history (back to the 80-90s attitude even), and I was suprised they also score very well on metrics like TOPS/W. But they did something else too, with software, perhaps in the Metal layer, where the runtime seems capable of allocating smartly payloads to either CPU, GPU or their ML accelerator (aka NPU, modulo quantisation). AMD has similar software now on its Ryzen AI chips (same with CPU/GPU/NPU), but it seems more limited (despite coming apparently years later) where what cannot fit the NPU is automatically run on CPU.
All in all, this seems too much for just luck.
love this, shared with friends
It's an accident on Apple's part, but that falls in line with a lot of accidents that have benefited Apple over the years. I worked there in the 80's and 90's. That period were Jobs was out and a succession of CEO's bumbled their way through slowly bringing the company near bankruptcy until the accident of Job's return, with a UNIX based OS, reappeared and, literally, saved the company from extinction.
I won't go through all the other accidents Apple stumbled upon, but, I'm pretty sure there's some sort of invisible force in the universe looking out for that company, and tht's coming from a bonified atheist. Nothing else explains how a company that's done so many stupid things can still be so successful.
Now that the dust has settled. See HN discussion about this post here: https://news.ycombinator.com/item?id=47747017
Interesting points. My feeling is they have a lucky position. Because when they built apple intelligence, they went the route of the "private" cloud option. But there is a nascent market: AI inference hubs - somewhere between a mac mini and studio optimized for AI inference, sitting in your home or office and connected to your devices. Everything private. I do not understand why a hardware company like Apple isn't going for this.
> These were of course non-binding. Micron, reading the demand signal, shut down its 29-year-old Crucial consumer memory brand to redirect all capacity toward AI customers. Then Stargate Texas was cancelled, OpenAI and Oracle couldn’t agree terms, and the demand that had justified Micron’s entire strategic pivot simply vanished. Micron’s stock crashed.
Micron's stock is up 38% since that, and almost at it's all time high https://finance.yahoo.com/quote/MU/
Exactly. The moat wasn’t the keynote claim. It was the side-effects: habit, switching costs, trust, and all the small frictions that made leaving feel expensive. If you only optimise for the stated goal, you miss the by-products that become the real asset.
I think this is an interesting idea. But one thing I am not sure is that if the scale law of llm is still effective. If the law is effective now, local model will not chase chatgpts. Hope to hear more good insights.
I think that’s the real question. If scale laws still dominate, local models probably won’t win on raw capability. But they can still win on packaging: latency, privacy, offline use, integration, and trust. The side-effect can become the moat even if the frontier stays ahead on the core metric.
Completely agree.
I wrote something similar and along the same lines: https://productandinvestingnerd.substack.com/p/apple-is-as-apple-does