NVIDIA, IBM, and Cornell University Developing Big Accelerator Memory for GPUs

[Peter_Brosdahl]

Researchers from NVIDIA, IBM, and Cornell University have been working on a potential solution for fast, fine-grain access to large amounts of data storage for GPU-accelerated applications called Big Accelerator Memory (BaM).

Go to post

 

[LazyGamer]

Seems like a stop-gap solution, when the real solution should be to decrease latency and increase bandwidth between the GPU and mass storage.

Granted it may also be necessary for a generation or two - asking host bus (chipset / northbridge / whatever) controllers to supply that much bandwidth while cutting the CPU out of the equation to reduce latency is perhaps a big step that likely wouldn't provide much gain elsewhere.

 

[Brian_B]

Seems like a stop-gap solution, when the real solution should be to decrease latency and increase bandwidth between the GPU and mass storage.

Granted it may also be necessary for a generation or two - asking host bus (chipset / northbridge / whatever) controllers to supply that much bandwidth while cutting the CPU out of the equation to reduce latency is perhaps a big step that likely wouldn't provide much gain elsewhere.

I’m wondering how far we are from just having the GPU take over the main parts and the CPU just being a co-processor

 

[LazyGamer]

I’m wondering how far we are from just having the GPU take over the main parts and the CPU just being a co-processor

For those companies with integrated IP, i.e. Apple (and other ARM users), AMD, Intel, Nvidia maybe someday, this is already a reality, right?

Imagine a <5nm update to the console APUs using Zen 4 cores as an example closer to what we see on desktops.

And Apple's M1 family as probably the best modern example of a high-performance SoC with both DRAM and NAND on package.

 

[Grimlakin]

I’m wondering how far we are from just having the GPU take over the main parts and the CPU just being a co-processor

When this happens PC's will just become glorified consoles. You have a chassis, perhaps a cooling and power delivery solution. Everything else is on the 'card' and your Motherboard is just a techie version of a high speed port replicator. Upgrades... Sure you can buy a new card. Oh you wanted to upgrade your RAM.. sorry no closed box. Storage... Sure you can ADD a storage card to your motherboard. But it won't be as fast as native. New video processors... nope that's a new card and it comes with new storage, and Ram too.

I don't want that future. I'm not naive enough to think it's not coming. But I think that will be as NUC like device, and bigger badder compute will still be in the realm of DIY.

Soon enough we will have to have licenses to have a DIY system. Just wait.

 

[LazyGamer]

I think it's even more interesting than that - I find our current 'solution', that being all manner of 'chassis' that we put mostly standardized parts into - to be fairly inelegant and necessary largely due to a lack of collective will to standardize.

I'm sitting here next to a Lian Li O11 Dynamic XL, which houses my personal desktop. It's the largest desktop case I've owned in the thirty years I've been building them, and I went with it for one reason only: to have more room to work inside the case.

As I've been able to pick up a 3080 and have started venturing down the Alder Lake path, the larger Lian Li case has proven to be a great companion.

Thing is, if you don't need to work in a system, and you don't need the highest-TDP parts, well, you could get away with an SFF the size of a toaster!

---

Now, to the meat.

With the proliferation of fast network interconnects as well as a wide variety of virtualization advancements, I can see 'desktop computing' going a similar direction that 'mobile computing' has already gone, that is, punted off to some big iron on the other end of a network link. Everything from 400Gbps fiber to variable-frequency wireless links can bring latency down to manageable levels with the 'local machine' being used primarily for the user interface and feedback-intensive work.

And if you consider what is within the realm of possibility today in say a <100W package, something that can be reasonably inconspicuous in terms of visual and aural footprints (small and quiet), I'm thinking that we can get pretty far toward 'computers' really being tuned for whatever I/O they need to service in terms of user interaction. Like phones, you really wouldn't need to upgrade them very often, and they'd really be more like appliances with decade-long or longer service lives being a given, like laundry, refrigeration, televisions before the digital age and now into the 4k age, and so on.

Very likely few will actually 'need' significant local compute resources. Gaming is literally the main consumer workload that has fairly absolute interactive latency limits, so it's likely to remain a bit of a special case.

For any other 'heavy' processing, whatever that might be, it can either be done on a local compute node (in a closet, etc.) or farmed out to a cloud environment of some sort.

 

[Grimlakin]

I think it's even more interesting than that - I find our current 'solution', that being all manner of 'chassis' that we put mostly standardized parts into - to be fairly inelegant and necessary largely due to a lack of collective will to standardize.

I'm sitting here next to a Lian Li O11 Dynamic XL, which houses my personal desktop. It's the largest desktop case I've owned in the thirty years I've been building them, and I went with it for one reason only: to have more room to work inside the case.

As I've been able to pick up a 3080 and have started venturing down the Alder Lake path, the larger Lian Li case has proven to be a great companion.

Thing is, if you don't need to work in a system, and you don't need the highest-TDP parts, well, you could get away with an SFF the size of a toaster!

---

Now, to the meat.

With the proliferation of fast network interconnects as well as a wide variety of virtualization advancements, I can see 'desktop computing' going a similar direction that 'mobile computing' has already gone, that is, punted off to some big iron on the other end of a network link. Everything from 400Gbps fiber to variable-frequency wireless links can bring latency down to manageable levels with the 'local machine' being used primarily for the user interface and feedback-intensive work.

And if you consider what is within the realm of possibility today in say a <100W package, something that can be reasonably inconspicuous in terms of visual and aural footprints (small and quiet), I'm thinking that we can get pretty far toward 'computers' really being tuned for whatever I/O they need to service in terms of user interaction. Like phones, you really wouldn't need to upgrade them very often, and they'd really be more like appliances with decade-long or longer service lives being a given, like laundry, refrigeration, televisions before the digital age and now into the 4k age, and so on.

Very likely few will actually 'need' significant local compute resources. Gaming is literally the main consumer workload that has fairly absolute interactive latency limits, so it's likely to remain a bit of a special case.

For any other 'heavy' processing, whatever that might be, it can either be done on a local compute node (in a closet, etc.) or farmed out to a cloud environment of some sort.

We or even maybe I am the minority here. I don't want to pay a monthly subscription to have a computer to use. I would much rather own it and have a portal to access my computer from or stream steam games or whatever from remotely rather than pay a monthly subscription to whosit online for twenty bucks a month. Oh you want the newest video card features and more performance... that's MORE per month.

No thank you to ALL of that. I would much rather own my PC and game how I want and use it how I see fit without some 3rd party assigning services based on a monthly fee.

I'm not going to break out my hardware spend for my compute costs. I already know I'm paying more for personal hardware than I would on a month to month basis. Part of that is I enjoy the hardware journey it also feeds directly into my career.

I can honestly say my year to year salary growth is more than I spend year to year on computer parts.

 

[LazyGamer]

We or even maybe I am the minority here. I don't want to pay a monthly subscription to have a computer to use. I would much rather own it and have a portal to access my computer from or stream steam games or whatever from remotely rather than pay a monthly subscription to whosit online for twenty bucks a month. Oh you want the newest video card features and more performance... that's MORE per month.

This is a threat, sure; how much of one I think depends on where consumers go.

For example, Adobe has been making money hand over fist by 'renting' their software. It's to the point that actually buying their software is either extremely difficult or outright impossible.

Some have complained. However, for most, it's been an improvement. The cost is low, as low as buying updates would be or lower, and quality and features have steadily improved. Video editing and digital photography, for example, haven't changed that much in the last decade, but the equipment very much has, both in terms of the capture chain and the editing chain. Cameras and lenses need profiling, codecs need to be integrated for stills and video, and so on. Adobe keeps up with that stuff better than many / most.

On the other hand, I'm not big on not owning the hardware, and I think many consumers feel the same way, whereas there's definitely a business case for leasing at the enterprise scale.

 

[Brian_B]

I was really just thinking the GPU would be the main attraction on the motherboard and the CPU would go on a daughter card (as opposed to the current thinking which has that flipped), but the SOC examples are probably much more pertinent.