RTX PRO 6000 Blackwell for AIWhat changes between its three versions, which servers take it, when an H200 is the better buy and how to split it into four with MIG.
The RTX PRO 6000 Blackwell is a PCIe card with 96 GB of memory, and that figure explains why it turns up in so many AI server configurations. 96 GB holds a 70-billion-parameter model at 8 bits with room left for a few conversations, something that until now took two cards or a far more expensive data centre GPU.
Many of the servers in our catalogue take it, and before configuring one you need to decide which version to order, how many fit in each machine and how to share it between several teams.
The three versions of the RTX PRO 6000
All three have the same chip, the same 96 GB of error-correcting GDDR7 and the same 24,064 CUDA cores, and all three support MIG. What changes is how much power they can draw and how they are cooled, and that decides where each one can go.
For a rack server you want the Server Edition, because it has no fans and is cooled by the air the chassis itself moves. The Workstation Edition pushes its heat into the case and is designed for a single card in a desktop machine. The Max-Q draws half the power and exhausts it out of the back, so you can put several side by side in a workstation, at the cost of lower performance than the 600 W cards.
Which servers take it and how many fit
Two cards give you 192 GB, enough for a 70-billion-parameter model with plenty of people at once, or for one of more than 100 billion at 8 bits. That is what the Supermicro SYS-212GB-FNR carries, fitting them into 2U with a single Xeon, and also the AS-5126GS-TNRT, which ships with two and leaves room to reach eight when you need them.
From Lenovo, the ThinkSystem SR650a V4 takes four in 2U if they are limited to 450 W, or two at 600 W. For four or eight at full power there is the SYS-422GA-NRT with four, the same chassis with eight and 768 GB of GPU memory between them, and the ThinkSystem SR675 V3, which also takes eight.
With eight cards, the GPUs alone can ask for almost 5 kW, so before choosing the chassis it is worth checking how much power reaches the rack and how much heat the room can remove. The full calculations, from the model to the data centre, are in our guide to choosing an AI server.
RTX PRO 6000 vs H200 and vs RTX 5090
The H200 NVL, also PCIe, has 141 GB of HBM3e memory that reads at 4.8 TB/s, three times faster than the RTX PRO 6000. That speed sets how quickly each word of the answer appears when a lot of people are waiting, and H200 cards can also be joined with NVLink bridges. RTX PRO 6000 cards have no NVLink and talk over PCIe, so they do best when each model fits on one card or a few.
If you are going to serve several models of up to about 70 billion parameters, or one of that size to a department, the RTX PRO 6000 is usually enough. If it is a large model spread across several GPUs and hundreds of people will use it, an H200 or an HGX board with B200 or B300 is worth the extra cost. If you are not sure, we can measure it with your workload before you buy.
The GeForce RTX 5090 comes up a lot in this comparison because it uses the same generation of chip, but it has 32 GB, does not support MIG and its cooling is designed for a desktop tower. It is fine for trying small models at a workstation, and on a shared server it runs out of memory before it runs out of compute.
Splitting the RTX PRO 6000 into four with MIG
MIG (Multi-Instance GPU) divides the card in hardware into smaller GPUs, each with its own memory and cores, so what happens on one does not affect the others. According to NVIDIA’s MIG guide, the RTX PRO 6000 supports up to four 24 GB instances, the RTX PRO 5000 up to two, and the H100, H200 and B200 up to seven. Four 24 GB slices are enough to serve several small models at once, such as an embeddings model, a classifier and a speech-to-text model, without them fighting over memory.
To enable it you need driver R575 (575.51.03) or later and a minimum version of the vBIOS, the card’s boot firmware. Some RTX PRO 6000 cards left the factory with an earlier vBIOS, and then nvidia-smi -i 0 -mig 1 answers with a “Not Supported” that does not explain the cause. You can check it with nvidia-smi --query-gpu=vbios_version --format=csv, and if it falls short, the update comes from the server manufacturer or whoever sold you the card.
On a workstation with a single card, MIG needs the card to run in compute mode, and in that mode it stops driving the monitor. This is not an issue on a server, where the motherboard handles the display.
If the card is going into a Kubernetes cluster, NVIDIA’s GPU Operator configures the slices on each node and offers them as if they were normal GPUs. We explain how they combine with time-slicing and per-team queues in GPU sharing on Kubernetes.
Frequently asked questions
How much memory does the RTX PRO 6000 Blackwell have?
96 GB of error-correcting GDDR7, the same in all three versions, with bandwidth of about 1.6 TB/s.
Which language model fits on an RTX PRO 6000?
A 70-billion-parameter model at 8 bits takes up about 70 GB, so it fits with about 25 GB free for conversations. gpt-oss-120b, which comes at 4 bits, also fits. Models of more than 100 billion at 8 bits need two or more cards.
Does the RTX PRO 6000 have NVLink?
No. The cards communicate over PCIe 5.0, so they work best when each model fits on one card or a few. Spreading a very large model across eight GPUs calls for HGX boards with NVLink.
What is the difference between the Server Edition and the Max-Q?
The Server Edition has no fans, can draw between 400 and 600 W and goes in rack servers. The Max-Q stays at 300 W, has a fan that exhausts air out of the back and is designed for workstations with several cards.
Why can’t I enable MIG on my RTX PRO 6000?
The first thing to check is the vBIOS. NVIDIA requires a minimum version for each edition and some cards shipped with an earlier one. Check it with nvidia-smi --query-gpu=vbios_version --format=csv and make sure the driver is R575 or later too.
How many RTX PRO 6000 cards do you need?
Tell us which model you want to serve and for how many people, and we will propose the server with the cards, the system installed and support included. If you would rather do the sums yourself, the configurator lets you build it step by step.
Lenovo and Supermicro servers with RTX PRO 6000, with Ubuntu or Rocky Linux and supportBlog!
Our mission
SIXE ensures that your infrastructures based on IBM®, Red Hat®, Canonical®, and SUSE® technologies remain stable, secure, and optimized for as long as you need them.
At SIXE, we are committed to sustainability. That’s why we help you design, implement, and maintain durable and efficient technological solutions.
