Intel's third-generation AI accelerator, Gaudi 3, succeeds Gaudi 2 and aims to directly compete with NVIDIA's H100, a widely used AI accelerator among major AI startups globally. Anil Nanduri, VP and GM of Intel’s DCAI Category and Head of the AI Acceleration Office, highlighted that Gaudi 3 can handle the same AI model sizes as NVIDIA's top GPUs but offers better value. At the Intel Tech Tour in Taipei, Nanduri emphasized that Intel focuses on performance-per-dollar rather than direct performance comparisons. Intel claims Gaudi 3 provides 1.9 times the performance per dollar for training and 2.3 times for inferencing compared to the H100.
The Gaudi 3 AI accelerator kit, which includes eight accelerators, is priced at $125,000, significantly lower than the NVIDIA H100 kit, which costs over $430,000. Unlike many AI accelerators that use proprietary networks, Gaudi 3 employs RDMA over Converged Ethernet (RoCE) and supports open frameworks like PyTorch, making it an open platform. Intel also aims to encourage the use of more open-source models with Gaudi 3.
Nanduri mentioned that the Indian AI startup Ola Krutrim, which recently established its own AI servers, currently uses Gaudi 2 and plans to upgrade to Gaudi 3 due to its higher value. Gaudi 3 is set to enter production soon, with availability expected in Q3 2024 after being sampled by select customers.
Regarding the debate between general AI accelerators like Gaudi 3 and specialized ones from companies like Google and Microsoft, Nanduri noted that the industry is still far from determining the optimal hardware and software layers for AI accelerators. He suggested that both general and specialized accelerators will likely coexist, balancing generality and specificity.
Comments