Micron Launches Industry’s First 8-High 24 GB HBM3 Gen2 Memory with 1.2 TB/s Bandwidth

[Tsing]

Micron Technology has announced that it has begun sampling the industry's first 8-high 24 GB HBM3 Gen2 memory, featuring bandwidth greater than 1.2 TB/s and a pin speed of over 9.2 Gb/s. According to Micron, this would equate to up to a 50% improvement over currently shipping HBM3 solutions. The new memory, which is designed to advance generative AI innovation, also boasts a 2.5x performance-per-watt improvement over previous generations.

See full article...

 

[Grimlakin]

That's pretty nice but what do you want to bet it comes with a 3x cost to buy improvement too!

24gb memory chips.. so 3 of those on a module would be 72gb of memory on a single module.

Would look a little funky on memory configurations on a server but on a AI processing card in a chassis it makes sense.

Load up a 4 set of those on a card giving it 288gb of high speed memory per controller, across say 6 of those cards in a chassis we're looking at near 1.75TB of useable AI memory for processing and consumption of data...

That makes my local AI that I'm going to increase to 64gb seem anemic.

I wonder when we will retire our... Ohhhh 32 cores... 64 threads... 384gb memory... hummmm that might just work...

 

[DrezKill]

I'd still like to get cards like the Radeon Fury X, Radeon Vega 64, and Radeon VII as collector's pieces, as they are the only consumer graphics cards with HBM memory. Only if I was super rich though.

 

[Uvilla]

With only one of those, a large apu with 16gb main system memory and 8 gb designated for video. Yet, "not possible" somehow.