Greetings from LGA Terminal C. Amidst the AI buzz, it's easy to forget that as data speeds up like a runaway train, our need for beefier processors to keep up with AI's demands grows exponentially. But who knows? Maybe in the future, AI will be so brilliantly efficient that it could run on a stack of dusty old laptops in your garage. That seems unlikely, though, so the race to build the very best processors is fierce.
Enter NVIDIA, the reigning champ of the processor world, flexing its tech muscles yet again with the announcement of the HGX H200, succeeding its highly sought-after predecessor, the H100. The H200 features a 1.4x increase in memory bandwidth and a 1.8x enhancement in memory capacity, crucial for handling demanding generative AI tasks.
A notable advancement is the incorporation of HBM3e memory, boosting the memory bandwidth to 4.8 terabytes per second and total memory capacity to 141GB. This is not just an upgrade; it's a very big upgrade.
Hold your power cords, though – the H200 won't hit shelves until Q2 2024. NVIDIA has not provided specific details on production volumes, raising questions about its availability given the supply constraints experienced with the H100. Given the gold rush for the H100 and its supply issues, we might as well start camping outside NVIDIA's headquarters now.
Designed to play nice with the H100 systems, the H200 facilitates easy integration for cloud service providers. Expect Amazon, Google, Microsoft, and Oracle to elbow each other out of the way to get their hands on these.
H100s cost about $30,000… if you can find them. The H200 will cost more, but the real game here will be finding one. 鈥�
As always your thoughts and comments are both welcome and encouraged. Just reply to this email. -s
ABOUT SHELLY PALMER
Shelly Palmer is the Professor of Advanced Media in Residence at Syracuse University’s S.I. Newhouse School of Public Communications and CEO of The Palmer Group, a consulting practice that helps Fortune 500 companies with technology, media and marketing. Named he covers tech and business for , is a regular commentator on CNN and writes a popular . He's a , and the creator of the popular, free online course, . Follow or visit .
