AMD Launches Instinct MI350P: CDNA 4 AI Muscle in a Standard PCIe Slot

D

David_Schroth

Administrator
Staff member
Joined
Apr 23, 2019
Messages
1,704
Reaction score
2,201
Getting CDNA 4 AI compute into a standard rack-mount server has been harder than it should be — until today. AMD has officially launched the Instinct MI350P, a PCIe add-in card based on its CDNA 4 architecture, and it’s the first Instinct GPU in this form factor in roughly four years. The pitch is simple: […]

See full article...
 

AMD unveils Helion racks for AI agents at OCP summit​

CPU-only and mixed CPU/GPU configurations due from partners in November

Dan Robinson IT INFRASTRUCTURE REPORTER

Helion offers an OCP-compliant blueprint for agentic AI infrastructure built around AMD's CPUs and accelerators.

AMD launched Helios earlier this year for frontier-scale AI training and large-scale inference. Helion's enterprise configuration instead combines agent execution and inference within a single rack.

Customers can opt for a CPU-only hyperscale configuration or an enterprise configuration with a mix of CPU and GPU blades.

Depending on configuration, AMD advertises up to 32,768 of AMD's "Zen 6" CPU cores, delivered via compute nodes fitted with 256-core versions of its Venice Epyc processor, up to 16 of its Instinct MI350P GPU accelerators, and up to 102 Tbps of networking bandwidth.

The CPU-only hyperscale version offers dedicated capacity for agent execution and connections to separate model inference infrastructure, while the enterprise configuration combines agent execution, GPU inference, and supporting workloads in one rack.

Both configurations offer dense, balanced, or high-performance CPU blades. For dense deployments, EPYC 9006 chips rated at 400 W are used, while performance blades sport a version of the chip with just 96 cores but clocked at 5 GHz. Balanced blades use 256-core processors rated at 600 W.

There are also GPU inference blades, with eight Instinct MI350P GPUs in a 5U server, plus general-purpose blades, intended for database and storage servers or technical compute workloads.

https://www.theregister.com/systems...ion-racks-for-ai-agents-at-ocp-summit/5302652
 
Marees said:

AMD unveils Helion racks for AI agents at OCP summit​

CPU-only and mixed CPU/GPU configurations due from partners in November

Dan Robinson IT INFRASTRUCTURE REPORTER

Helion offers an OCP-compliant blueprint for agentic AI infrastructure built around AMD's CPUs and accelerators.

AMD launched Helios earlier this year for frontier-scale AI training and large-scale inference. Helion's enterprise configuration instead combines agent execution and inference within a single rack.

Customers can opt for a CPU-only hyperscale configuration or an enterprise configuration with a mix of CPU and GPU blades.

Depending on configuration, AMD advertises up to 32,768 of AMD's "Zen 6" CPU cores, delivered via compute nodes fitted with 256-core versions of its Venice Epyc processor, up to 16 of its Instinct MI350P GPU accelerators, and up to 102 Tbps of networking bandwidth.

The CPU-only hyperscale version offers dedicated capacity for agent execution and connections to separate model inference infrastructure, while the enterprise configuration combines agent execution, GPU inference, and supporting workloads in one rack.

Both configurations offer dense, balanced, or high-performance CPU blades. For dense deployments, EPYC 9006 chips rated at 400 W are used, while performance blades sport a version of the chip with just 96 cores but clocked at 5 GHz. Balanced blades use 256-core processors rated at 600 W.

There are also GPU inference blades, with eight Instinct MI350P GPUs in a 5U server, plus general-purpose blades, intended for database and storage servers or technical compute workloads.

https://www.theregister.com/systems...ion-racks-for-ai-agents-at-ocp-summit/5302652
Click to expand...

AMD Instinct™ MI350P, eight GPUs in a 5U server. Two GPU servers can be added to a rack for sixteen GPUs total — two nodes of eight. The two nodes operate independently; there is no scale-up GPU fabric between them, because this configuration is designed for enterprise agentic workloads and acceleration rather than frontier-scale training.

https://www.amd.com/en/products/rackscale-solutions/helion.html

https://newsroom.amd.com/news/open-rack-blueprint-scale-agentic-ai/
 
imo, if this is purely inference then HBM seems like a overkill. might make sense for training use cases 🤔
 
You must log in or register to reply here.
Become a Patron!
Back
Top