NVIDIA is rumored to be going with a totally new type of cooler for its next-gen Rubin Ultra AI range, as the company has serious concerns about an increasing power bill and THICC temps for data centers. This is a significant change as we are migrating to an effective microchannel cold plate (MCCP) design.
NVIDIA’s Manhattan Project With Ruben Ultra is A New Normal With Microchannel Cover Plates, Driving Perf/Watt to the Extreme
Now, for those not already aware, the issue of a proper cooling solution for next-gen cards is high on the priority list for companies like NVIDIA (ED- power draw goes through the roof with every new iteration) as they need to keep things cool onboard. The current direction of higher and higher power consumption in AI chips renders previous cooling techniques inadequate.
NVIDIA Contacting Cooler Partners for Direct-To-Chip Microchannel Cooling on Rubin Ultra AI GPUs, Potential Shift From Traditional Liquid Cooling Design Gives NVIDIA A Competitive Edge: @QQ_Timmy Simply Refrains According to the chatter by QQ_Timmy, NVIDIA is known to be reaching out to its cooling partners to supply ‘direct-to-chip’ microchannel cooling through cold plates for their upcoming Rubin Ultra AI based GPUs which might end up becoming a massive departure from how it traditionally approaches liquid cooling and could give NVIDIA a top-tier solution when it comes to absolute performance.
Now, speaking of MCCP as mention here. Microchannel cold plates are somewhat like the direct-die cooling that enthusiasts commonly use with modern CPUs: They’re copper cold plates, but they have microchannels machined inside them that allow coolant to flow through. This configuration is used to enable the plate to locally generate convection, thereby minimizing the thermal resistance from the die to the fluid. It’s similar to NVIDIA’s existing standard planar cooling solutions, except the modifications are applied directly to the liquid-cooled plates sitting on top of the chip for much more effective thermal control. The effectiveness of microchannel cold plates comes from the huge surface area for heat exchange in these tiny channels, a very desirable characteristic for next-generation high-power GPUs.
I could get into all the nitty gritty on how MCCP really works, but it’s more important to understand why this technology — microchannel cold plates above all else — is necessary for NVIDIA in 2018. Team Green is now on a super tight product cadence, and throwing more of Blackwell to Rubin scales up the power exponentially in something like a high-complexity rack-scale system, which is why NVIDIA needs to ‘think outside the box’. Whether or not the company likes it, architectural progress is putting pressure on them to invest in more advanced cooling solutions like microchannel cold plates. These developments were driven by the increasing thermals associated with next-gen AI accelerators.
The use of microchannel cold plates underscores that NVIDIA is prioritizing cooling in order to enable their aggressive performance roadmap. According to Wcctech, the MCCP (microchannel cold plates) have been commissioned by NVIDIA from Asia Vital Components a Taiwanese thermal solution manufacturer to design for Rubin Ultra. But it originally was meant for Rubin; the deadline created a short timetable, and suppliers didn’t have time to shift to an advanced solution such as microchannel cold plates. This use of microchannel cold plates has followed steps being taken across the industry to step away from standard thermal solutions.
Interestingly, Microsoft has made a recent announcement regarding what they termed microfluidic cooling–a technology largely resembling MCCP with the inverted flow of coolant and the instream validation being replaced by ‘in-chip cooling’ where coolant is directly in or behind silicon.
That’s proof that the industry is definitely in need of new cooling solutions and microchannel cold plates could well be at the top of the list for high-end, current-generation designs. The future of AI depends largely on the adoption of these new thermal solutions, and selection of microchannel cold plates by one of industry-leading NVIDIA is a significant industry trend. The higher transistor packing density and the related power load require for microchannel cold plates the high efficiency.
In addition, the design folds in microchannel cold plates so NVIDIA’s Rubin Ultra runs stably at full-throttle performance, which is essential for training ever-larger AI models. Transitioning to microchannel cold plates poses a daunting engineering task, but one that stands to lower power demands significantly.
This dedication to top-tier cooling through microchannel cold plates reflects the tremendous thermal pressures at play in the AI hardware industry. These advanced microchannel cold plates are a significant advancement in design, fabrication and realization. In the end success of Rubin Ultra will tie directly to performance of the microchannel cold plates.
Note: The author’s take on this issue is done through fair analysis of the industry news, research papers around high-performance computing thermal management and supply chain reports covering NVIDIA’s GPU architecture. While I can’t speak to having worked on them firsthand, the author contributed in-kind support based on specialized knowledge of AI hardware and data center cooling technologies and IP competitive analysis within next-gen GPU design, though not specific end-product cold-plate designs for microchannelistration with NVIDIA.
The site serves as a deep resource in emerging technologies such as high-performance and AI computing, so information about advanced cooling solutions – including microchannel cold plates – is up to date and accurate.
