Francesco Di Costanzo
Back to shorts

(10) The Memory Boom Could Re-Centralize AI

/5 min read

Server First

The 2026 memory boom could re-centralize AI at the margin by widening the divide between users who can absorb rising hardware costs and those who rent inference. Local models are improving, but the memory needed to run them sits inside a supply chain now organised around data centres.

Local inference does not usually consume the HBM stacks attached to top-end accelerators. It relies on client DRAM, LPDDR or graphics memory. The markets are still connected through cleanrooms, process capacity and supplier capital budgets. In March, TrendForce reported that suppliers were reallocating capacity towards HBM and server applications. SK hynix described the same mechanism from the producer side: focused HBM investment diverts resources from general-purpose memory and tightens its supply-demand balance.

Hyperscalers can secure supply through long-term agreements while consumer-device makers face tighter allocation elsewhere. Memory prices then become a market readout of a server-first production strategy.

The Consumer Spillover

TrendForce forecast in February that conventional DRAM contract prices would rise 90–95% quarter on quarter in the first quarter of 2026. It expected PC DRAM prices to more than double and LPDDR4X and LPDDR5X prices to rise by about 90%. Its March update projected another 58–63% increase for conventional DRAM in the second quarter.

Those figures were forecasts rather than audited transaction results, but subsequent supplier disclosures confirmed the direction of pressure. Samsung reported record Memory Business revenue and operating profit for the second quarter. It said limited capacity had pushed the division to focus primarily on server products and expected the market to remain undersupplied in the second half, despite softer demand from mobile devices and PCs. Samsung's mobile and networks businesses reported an operating loss, with higher component costs among the pressures on earnings.

Micron likewise attributed record fiscal second-quarter results partly to tight industry supply. Its earnings deck expected DRAM and NAND conditions to remain tight beyond 2026, citing cleanroom constraints, a higher HBM trade ratio and long construction lead times.

This evidence does not show consumers abandoning local AI. It shows the entry price moving against it. For an intermittent user, higher hardware expenditure can lengthen the period before owning a machine beats renting inference. The exact break-even still depends on the model, usage, electricity, depreciation and cloud price.

Compression Pushes Back

Software efficiency could move faster than chip prices. Apple demonstrated Llama 3.1 8B on an M1 Max using Int4 quantisation, reducing the model from 16 GB to 4.2 GB and doubling measured throughput to about 33 tokens per second at a 2,048-token context. The authors did not fully evaluate model quality, so the result proves a memory reduction on one setup rather than a free conversion.

Apple's own on-device model makes the trade-off clearer. It compresses roughly three billion parameters to two bits per weight, yet Apple says it is designed for application tasks rather than general world knowledge. Google reported that TurboQuant cut key-value cache memory by at least six times on tested long-context retrieval tasks. The key-value cache is only one part of a model's memory footprint.

The strongest counterargument is therefore substantial. Quantisation, smaller specialised models and cache compression can offset expensive chips, while privacy, offline operation and predictable high usage still favour local deployment. These gains also reward firms that can co-design the model, runtime, operating system and silicon. Scarcity may replace some cloud dependence with platform dependence rather than produce broad independence.

A Higher Entry Price

The likely outcome is a split market. Owners of recent memory-rich hardware will run increasingly capable compressed models. Users with older devices, irregular demand or a need for the largest models will continue to rent access. Re-centralisation would happen through marginal purchasing decisions, not through the disappearance of local AI.

Buyers should size a machine against a named model, quantisation level and context length instead of an “AI PC” label. Developers should publish those memory requirements and offer a smaller local fallback. Investors assessing local AI should watch client DRAM prices and installed memory configurations alongside HBM demand. Those measures will show whether software efficiency is widening access faster than the memory cycle is raising its price.

Sources

  1. TrendForce, "Memory Price Outlook for 1Q26 Sharply Upgraded; QoQ Increases of All Product Categories to Hit Record Highs, Says TrendForce" https://www.trendforce.com/presscenter/news/20260202-12911.html

  2. TrendForce, "AI Server Demand to Drive Memory Contract Price Increases in 2Q26 as CSPs Secure Supply via Long-Term Agreements" https://www.trendforce.com/presscenter/news/20260331-12995.html

  3. Samsung Electronics, "Samsung Electronics Announces Second Quarter 2026 Results" https://news.samsungsemiconductor.com/global/samsung-electronics-announces-second-quarter-2026-results/

  4. Micron Technology, "Micron Technology, Inc. Reports Results for the Second Quarter of Fiscal 2026" https://investors.micron.com/news-releases/news-release-details/micron-technology-inc-reports-results-second-quarter-fiscal-2026

  5. Micron Technology, "Q2 2026 Earnings Deck" https://investors.micron.com/static-files/9c0becf5-df56-4eec-bd67-453dda68b273

  6. SK hynix, "2026 Market Outlook – “Focus on the HBM-Led Memory Supercycle”" https://news.skhynix.com/en/2026-market-outlook-focus-on-the-hbm-led-memory-supercycle/

  7. Apple Machine Learning Research, "On Device Llama 3.1 with Core ML" https://machinelearning.apple.com/research/core-ml-on-device-llama

  8. Apple Machine Learning Research, "Updates to Apple’s On-Device and Server Foundation Language Models" https://machinelearning.apple.com/research/apple-foundation-models-2025-updates

  9. Google Research, "TurboQuant: Redefining AI Efficiency with Extreme Compression" https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/