AI chip maker Nvidia announced details about their new GPU – the H200 – at the Supercomputing 2023 (SC23) conference in Denver, Colorado.
The H200 is an upgrade from the company’s extremely popular H100 GPU, which is utilized by most of the major data centers being used to power generative AI models like ChatGPT.
The huge growth of generative AI has driven the company into the stratosphere, with a market cap topping $1.2 trillion dollars and making the company the 6th most valuable company in the world.
The company’s stock has risen 1000% in the last five years.
According to the company blog, the H200 will have “141GB of memory at 4.8 terabytes per second, nearly double the capacity and 2.4x more bandwidth compared with its predecessor, the NVIDIA H100.”
Using Meta’s large language model Llama 2 as an example, the H200 will double the inference speed and will pave the wave for the next big developments in generative AI.
The new chip is expected in late 2024, but the price will be hefty. The earlier H100 is reported to sell for $25,000 to $40,000.
Nvidia will still produce the H100 and is ramping up production from 500,000 this year to an expected two million in 2024.
The chips are in such high demand that companies are using them as collateral for loans, often to buy more chips to power the new gold rush of AI.
It is unclear how many H200s the company will be able to make, but big tech companies like Microsoft, Amazon, and Google are lining up to get their hands on them.