Amazon Web Services has started renting out a new class of graphics chip by the hour, and it got there before anyone else. Amazon Web Services says its EC2 G7 instances are generally available, making it the first major cloud provider to offer NVIDIA's RTX PRO 4500 Blackwell Server Edition GPUs to customers who would rather not buy the silicon themselves.
Two bits of vocabulary before the numbers arrive. EC2 is Elastic Compute Cloud, said as the letters and the digit, and it is the part of Amazon's business that rents computers by the minute. A GPU, spelled out letter by letter, is a chip built to do a great deal of simple arithmetic at once rather than a little complicated arithmetic very fast, which is why it draws game worlds and trains models with equal enthusiasm.
The comparison AWS reaches for is its own previous generation, the G6. Against that, the company claims up to 4.6 times the AI inference performance and up to 2.1 times the graphics performance. Inference is the running of a trained model rather than the training of it, the part where the model actually answers something, and it is now the workload most companies are paying for.
Memory is where this generation moves furthest. Each GPU carries 32 GB, which AWS puts at 1.33 times the capacity of the G6 and, more interestingly, 2.45 times the memory bandwidth. Capacity is how much you can keep next to the processor; bandwidth is how quickly it can be handed over. Widening the door turns out to matter more than enlarging the room, because a model that fits in memory but starves waiting for it is still a slow model.
Networking gets the same treatment. G7 instances come with 700 Gbps of throughput on EFA, Amazon's Elastic Fabric Adapter, which the company puts at seven times the G6 figure. There is also up to 7.6 TB of local NVMe solid-state storage attached to the instance, so a large model or dataset can sit on the machine rather than being dragged across the network every time somebody asks it a question.
Video work gets a mention too, and it is not an afterthought. The ninth-generation NVENC encoder and sixth-generation NVDEC decoder support 4:2:2 colour, and AWS puts the result at 1.5 times the concurrent video streams of the previous generation. That is a broadcast and post-production number more than an AI one, which tells you who else these machines are aimed at.
Sizing runs from one GPU to eight. The largest configuration reaches 256 GB of total GPU memory, 192 virtual CPUs, 768 GiB of system memory and that full 700 Gbps, and there are seven sizes in between. AWS says a bare metal version is still to come. The processors alongside are custom sixth-generation Intel Xeon Scalable parts, which is a quiet reminder that even a GPU instance still needs something to feed it.
The bet underneath all this is that inference, rendering and virtual desktops are converging on the same rented hardware, and that customers would rather pay On-Demand, Spot or Savings Plan rates than sign for a rack. Whether the 4.6 times figure survives contact with anyone else's benchmark is a separate question, and AWS ran that one itself.
Leave a Reply