- Micron Unveils Dense 256GB LPDDR5x Module Aimed Squarely at AI Servers
- Eight SOCAMM2 modules can increase server memory capacity to 2TB
- AI inference workloads increasingly shift performance bottlenecks toward system memory capacity
Modern large language models (LLMs) and inference pipelines increasingly demand huge pools of memory, forcing hardware vendors to rethink server memory architecture.
Micron has now introduced a 256 GB SOCAMM2 memory module aimed at data center systems where capacity, bandwidth and power efficiency influence overall performance.
The module is based on 64 32GB monolithic LPDDR5x chips, forming a dense LPDRAM package that addresses the growing memory footprint required by contemporary AI workloads.
Article continues below.
Expanding memory capacity for AI server platforms
The module also increases the maximum memory available per processor configuration. When eight of these SOCAMM2 modules are installed in an eight-channel server CPU, the total capacity can reach 2TB of LPDRAM.
That figure surpasses the previous generation of 192GB modules by about a third, allowing systems to accommodate larger context windows and more demanding inference tasks.
Micron describes the SOCAMM2 design as more efficient than conventional server memory modules.
“Micron’s 256GB SOCAMM2 offering enables the most energy-efficient CPU-attached memory solution for both AI and HPC,” said Raj Narasimhan, senior vice president and general manager of Micron’s cloud memory business unit.
“Our continued leadership in low-power memory solutions for data center applications has uniquely positioned us to be the first to offer a monolithic 32Gb LPDRAM die, helping to drive industry adoption of more energy-efficient, high-capacity system architectures.”
The company says the new LPDRAM module consumes about a third of the power of comparable RDIMMs and takes up only a third of the physical space.
Its reduced power demand and smaller module size enable greater rack density within large data center deployments, while lower memory power reduces thermal load and infrastructure costs.
The SOCAMM2 architecture also follows a modular design intended to simplify maintenance and future expansion.
The format supports liquid-cooled server systems and can accommodate additional capacity as memory requirements grow along with model complexity and data set scale.
Micron claims that the 256GB SOCAMM2 module can influence certain inference operations under unified memory architectures.
The company reports a more than 2.3x improvement in time to first token during long context inference when the module is used for key-value cache flushing.
In standalone CPU workloads, the LPDRAM configuration delivers three times better performance per watt compared to conventional server memory modules.
The company’s LPDRAM portfolio spans components from 8GB to 64GB and SOCAMM2 modules from 48GB to 256GB, and samples of the 256GB SOCAMM2 module are already being shipped to customers.
Follow TechRadar on Google News and add us as a preferred source to receive news, reviews and opinions from our experts in your feeds. Be sure to click the Follow button!
And of course you can also follow TechRadar on TikTok for news, reviews, unboxings in video form and receive regular updates from us on WhatsApp also.
