We take apart an NVIDIA HGX H200 8 GPU server from Aivres. The Aiveres KR6288 is the company's Intel Xeon-powered AI server for the NVIDIA Hopper generation, and we have installed over 3.6Tbps of networking for a massive AI server.

STH Main Site Article: 

Substack: https://axautikgroupllc.substack.com/
STH Top 5 Weekly Newsletter: https://eepurl.com/dryM09

----------------------------------------------------------------------
Become a STH YT Member and Support Us
----------------------------------------------------------------------
Join STH YouTube membership to support the channel: https://www.youtube.com/channel/UCv6J_jJa8GJqFwQNgNrMuww/join
Professional Users Substack: https://axautikgroupllc.substack.com/
----------------------------------------------------------------------
Where to Find The Unit We Purchased
Note we may earn a small commission if you use these links to purchase a product through them.
----------------------------------------------------------------------
STH Merch on Spring: https://the-sth-merch-shop.myteespring.co/

----------------------------------------------------------------------
Where to Find STH
----------------------------------------------------------------------
STH Forums: https://forums.servethehome.com
Follow on Twitter: https://twitter.com/ServeTheHome
Follow on LinkedIn: https://www.linkedin.com/company/servethehome-com/
Follow on Facebook: https://www.facebook.com/ServeTheHome/
Follow on Instagram: https://www.instagram.com/servethehome/

----------------------------------------------------------------------
Other STH Content Mentioned in this Video
----------------------------------------------------------------------
- Touring the xAI Colossus: https://youtu.be/Jf8EPSBZU7Y
- NVIDIA HGX H100: https://youtu.be/4Np1HnWiHb4
- NVIDIA HGX A100: https://youtu.be/E6HJi0iqYbw
- NVIDIA V100: https://youtu.be/2FV5zHO1z1M
- NVIDIA P100: https://youtu.be/ipQXdjjAPGg
- NVIDIA L40S: https://youtu.be/yKOg1DjtB50
- NVIDIA Arm with A100: https://youtu.be/t4dPmgZqWdk

----------------------------------------------------------------------
Timestamps
----------------------------------------------------------------------
00:00 Introduction
02:45 NVIDIA HGX H200 Server Aivres KR6288 Hardware Overview
15:00 NVIDIA H200 Performance
16:00 NVIDIA H200 Power Consumption
18:21 Key Lessons Learned

ServeTheHome

The NVIDIA HGX H200 GPU server features 141 GB of memory per GPU and promises over 1.1 TB of HBM 3 memory in total. This assembly is designed for peak AI performance, utilizes NVLink switches for high-speed interconnects, and supports robust serviceability with innovative design choices such as easily accessible GPU and NIC modules. The AIers KR 6288 server incorporates these GPUs, equipped with significant memory and networking capabilities, making it a top-tier choice for AI workloads.

Inside a MEGA AI GPU Server with the NVIDIA HGX H200