New Microsoft Azure VMs target generative AI developers

New virtual machines for Microsoft Azure allow developers to create generative AI apps that can be scaled to work with thousands of Nvidia H100 GPUs.

The ND H100 v5 VM series on Azure, which works in tandem with Quantum-2 InfiniBand networking, boosts the performance of large-scale deployments by companies such as OpenAI, creators of the much talked about ChatGPT, and Nvidia's chips.

The new supercomputing system in the cloud provides the type of infrastructure required to handle the latest large-scale AI training models, according to Matt Vegas, principal product manager for Azure HPC and AI at Microsoft.

“Generative AI applications are rapidly evolving and adding unique value across nearly every industry,” Vegas wrote in a blog post this week. “From the newly released AI-powered Bing and Edge to AI-powered assistance in Microsoft Dynamics 365, AI is becoming a pervasive component of software and how we interact with it. We want to ensure that our AI infrastructure will be there to pave the way.”

Version 5 of the ND H100, now in preview, includes eight H100 Tensor Core GPUs that are interconnected through NV Switch and NVLink 4.0, 400 gigabytes/sec Nvidia Quantum-2 CX7 Infiniband per GPU, NVSwitch and NVLink 4.0 with 3.6 TBps bidirectional bandwidth among the 8 local GPUs, 4^th generation Intel Xeon Scalable processors and PCIE Generation 5 host-to-GPU interconnect with 64 gigabytes/second per GPU.

While the system’s specs may be impressive, the cost may make it reasonable for only the largest of enterprises.

“This is Nvidia’s highest end GPU/AI configuration and will run things like ChatGPT well, or for banks dealing with thousands of customers they watch to make sure they are not committing fraud or are a bad risk,’’ said Jack Gold, president and principal analyst with J. Gold Associates, LLC. “But here’s the gotcha -- the H100 is a very expensive system. If you want it to handle complex environments, it might cost you a million or more.”

Organizations most likely to purchase a fully loaded Nvidia system would be third-party developers and service providers.

Microsoft has not disclosed pricing for its new Azure offering. Other AI cloud services that provide large generative AI models, such as Azure OpenAI Service, offer pay as you go consumption models and users pay per unit for each model.

Nvidia has taken an active role in helping not just Microsoft but all of its hyperscale partners build their data centers for AI, said Ian Buck, vice president of hyperscale and high-performance computing at Nvidia. “What we inevitably end up with is the partner’s data center with Nvidia’s brains.”

But while Nvidia partners with Microsoft, the chip giant also partners with Microsoft's largest competitors, including AWS and Google, in putting together their respective AI supercomputers. Microsoft may be in a better competitive position with its recent AI-focused acquisitions and services.

“Microsoft has arrived at an advantageous position,” said Dan Newman, chief analyst of Futurum Research and CEO of The Futurum Group. “With new AI services, the implementation of ChatGPT, a sizeable investment in OpenAI the company is moving fast and with likely more announcements next week [at Nvidia’s GTC conference].”

A variety of Azure services will be available with the new VMs, including Azure Machine Learning, which makes Microsoft’s AI supercomputer available to users for model training, and the Azure Open AI Service which provides users with the capabilities of large-scale generative AI models, the company said.

Microsoft may have a technology edge now, according to some, but this latest partnership is likely to inspire the delivery of even more capable systems by both well-known and little-known competitors alike as the AI market continues to heat up.

“The competition has been roaring the last year or two but with technology being offered like this, competitive offerings will go up to the next level,” Newman said.

As Editor At Large with TechTarget’s News Group, Ed Scannell is responsible for writing and reporting breaking news, news analysis and features focused on technology issues and trends affecting corporate IT professionals.