- Future memory chips of AI could demand more power than combined complete industrial areas
- 6TB of memory in a GPU sounds incredible until you see the power drawing
- HBM8 batteries are impressive in theory, but scary in practice for any energy conscious company
The relentless impulse to expand the IA processing power is marking the beginning of a new era for memory technology, but has a cost that raises practical and environmental concerns, experts warned.
The research of the Advanced Institute of Science and Technology of Korea (Kaist) and the interconnection of Terabyte and the Package Laboratory (Tera) suggest that by 2035, the GPU accelerators of the IA GPU equipped with 6 TB of HBM could become reality.
These developments, although technically impressive, also highlight the pronounced energy demands and the increase in complexity involved in pushing the limits of AI infrastructure.
The increase in GPU memory capacity of IA brings great energy consumption
The road map reveals that the evolution of HBM4 to HBM8 will offer great gains in bandwidth, memory stacking and cooling techniques.
From 2026 with HBM4, the AMD Instinct Mi400 platforms will incorporate up to 432 GB of memory, with bandwidths that reach almost 20TB/s.
This type of memory uses direct liquid packaging methods and custom packaging to handle energy densities around 75 to 80W per battery.
HBM5, projected by 2029, doubles the entrance/exit lanes and moves towards immersion cooling, with up to 80 GB per battery that consumes 100W.
However, the power requirements will continue to rise with HBM6, anticipated in 2032, which pushes the bandwidth at 8TB/Sy stacking capacity to 120 GB, each of the 120W.
These figures are quickly added when considering the complete GPU packages that are expected to consume up to 5.920 W per chip, assuming 16 hbm6 batteries in a system.
By the time HBM7 and HBM8 arrive, the numbers extend towards a previously unimaginable territory.
HBM7, which is expected around 2035, triples the bandwidth at 24TB/Sy allows up to 192 GB per battery. The architecture admits 32 memory batteries, pushing the total memory capacity beyond 6TB, but the energy demand reaches 15,360W per package.
The estimated energy consumption of 15,360 W marks a dramatic increase, which represents an increase of seven times in just nine years.
This means that one million of these in a data center would consume 15.36GW, a figure that is approximately equivalent to all wind generation capacity in the United Kingdom in 2024.
HBM8, projected by 2038, further expands the bandwidth and bandwidth with 64TB/s per battery and up to 240 GB capacity, using 16,384 I/s and speeds of 32 Gbps.
It also presents coaxial TSV, integrated cooling and double -sided interposers.
The growing demands of AI and inference to the large language model (LLM) have led researchers to introduce concepts such as HBF (high bandwidth flash) and HBM centered computing.
These designs propose the integration of Nand Flash and LPDDR in the HBM battery, trusting new methods and cooling interconnections, but its viability and efficiency of the real world have not yet been demonstrated.
