Elon Musk Unveils Memphis Supercluster: World’s Most Powerful AI Training Cluster

MetaversePlanet July 23, 2024Last Updated: July 23, 2024

0 2 minutes read

Elon Musk announced that the world’s most powerful artificial intelligence training cluster, called the Memphis Supercluster, has been established and has begun training for his artificial intelligence company xAI.

Elon Musk, a leader of multiple companies, oversees the car company Tesla, the space technology company SpaceX, the satellite internet company Starlink, the social media platform X (formerly Twitter), the brain chip company Neuralink, and the artificial intelligence company xAI. Recently, the tech baron announced that the Memphis Supercluster, the world’s most powerful AI training cluster for xAI, has been established and has started its training.

It will train the most powerful AI

For those who don’t know, xAI is a very young artificial intelligence company founded in 2023. The company is developing a large language model called Grok, making it available to paid subscribers via X with a chatbot of the same name. Musk established xAI as a natural competitor to OpenAI, aiming to unleash the most advanced artificial intelligence.

In line with this goal, the Memphis Supercluster is being commissioned. The AI training cluster consists of 100,000 liquid-cooled Nvidia H100 GPUs connected by a single RDMA (remote direct memory access) fabric. These GPUs are highly valuable and in high demand.

Nvidia is set to launch H200 chips this year, which are more powerful than the H100s. Additionally, the next generation of much more advanced Blackwell-based B100 and B200 GPUs will go into mass production next year.

The Memphis Supercluster, described as the world’s most powerful artificial intelligence training cluster, will be used to train the world’s most powerful AI, according to Musk. Grok 1.5 has already been released, and Grok 2 is expected to be released in the coming months. Musk has previously announced that Grok 3 will be released in late 2024. In his latest post, he mentions that this “most powerful artificial intelligence” will complete its training by December.

The Memphis Supercluster is unprecedented in scale. The newly formed cluster comfortably outperforms all systems on the Top500 list. At the top of this list in terms of processing power, Frontier uses 37,888 AMD GPUs. The world’s most powerful supercomputers, such as the Aurora (60,000 Intel GPUs) and Microsoft Eagle (14,400 Nvidia H100 GPUs), also lag behind significantly.