SAN JOSE — Supermicro, Inc., a systems provider for AI, Cloud, Storage, and 5G/Edge, is announcing its latest portfolio to accelerate the deployment of generative AI. The Supermicro SuperCluster solutions provide foundational building blocks for the present and the future of large language model (LLM) infrastructure.
The three powerful Supermicro SuperCluster solutions are now available for generative AI workloads. The 4U liquid-cooled systems or 8U air-cooled systems are purpose-built and designed for powerful LLM training performance, as well as large batch size and high-volume LLM inference. A third SuperCluster, with 1U air-cooled Supermicro NVIDIA MGXTM systems, is optimized for cloud-scale inference.
“In the era of AI, the unit of compute is now measured by clusters, not just the number of servers, and with our expanded global manufacturing capacity of 5,000 racks/month, we can deliver complete generative AI clusters to our customers faster than ever before,” said Charles Liang, president and CEO of Supermicro. “A 64-node cluster enables 512 NVIDIA HGX H200 GPUs with 72TB of HBM3e through a couple of our scalable cluster building blocks with 400Gb/s NVIDIA Quantum-2 InfiniBand and Spectrum-X Ethernet networking. Supermicro’s SuperCluster solutions combined with NVIDIA AI Enterprise software are ideal for enterprise and cloud infrastructures to train today’s LLMs with up to trillions of parameters. The interconnected GPUs, CPUs, memory, storage, and networking, when deployed across multiple nodes in racks, construct the foundation of today’s AI. Supermicro’s SuperCluster solutions provide foundational building blocks for rapidly evolving generative AI and LLMs.”
“NVIDIA’s latest GPU, CPU, networking and software technologies enable systems makers to accelerate a range of next-generation AI workloads for global markets,” said Kaustubh Sanghani, vice president of GPU Product Management at NVIDIA. “By leveraging the NVIDIA accelerated computing platform with Blackwell architecture-based products, Supermicro is providing customers with the cutting-edge server systems they need that can easily be deployed in data centers.”
Supermicro 4U NVIDIA HGX H100/H200 8-GPU systems double the density of the 8U air-cooled system by using liquid-cooling, reducing energy consumption and lowering data center TCO. These systems are designed to support the next-generation NVIDIA Blackwell architecture-based GPUs. The Supermicro cooling distribution unit (CDU) and manifold (CDM) are the main arteries for distributing cooled liquid to Supermicro’s custom direct-to-chip (D2C) cold plates, keeping GPUs and CPUs at optimal temperature, resulting in maximum performance. This cooling technology enables up to a 40% reduction in electricity costs for the entire data center and saves data center real estate space. Learn more about Supermicro Liquid Cooling technology: https://www.supermicro.com/en/solutions/liquid-cooling
The NVIDIA HGX H100/H200 8-GPU equipped systems are ideal for training Generative Al. The high-speed interconnected GPUs through NVIDIA® NVLink®, high GPU memory bandwidth, and capacity are key for running LLM models, cost effectively. The Supermicro SuperCluster creates a massive pool of GPU resources acting as a single AI supercomputer.
Whether fitting an enormous foundation model trained on a dataset with trillions of tokens from scratch or building a cloud-scale LLM inference infrastructure, the spine and leaf network topology with non-blocking 400Gb/s fabrics allows it to scale from 32 nodes to thousands of nodes seamlessly. With fully integrated liquid cooling, Supermicro’s proven testing processes thoroughly validate the operational effectiveness and efficiency before shipping.
Supermicro’s NVIDIA MGX system designs featuring the NVIDIA GH200 Grace Hopper Superchips will create a blueprint for future AI clusters that address a crucial bottleneck in Generative Al: the GPU memory bandwidth and capacity to run large language (LLM) models with high inference batch sizes to lower operational costs. The 256-node cluster enables a cloud-scale high-volume inference powerhouse, easily deployable and scalable.
