Description
A course on letting several teams use the same GPUs in a Kubernetes cluster: the production model, data science notebooks and training jobs. You install the NVIDIA GPU Operator, share one card by time-slicing and partition another with MIG, and set up per-team queues with Kueue, all on a lab cluster with GPUs.
Course details
| SIXE code: SXIA12 | Category: Artificial Intelligence / AI infrastructure |
| Delivery: Online and on-site | Duration: 2 days |
Who it is for
System administrators and platform teams who run servers with NVIDIA GPUs and have to share them between several teams or applications.
Prerequisites
Kubernetes administration (deployments, nodes, labels and Helm) and Linux. If the team is new to Kubernetes, the starting point is the Kubernetes with MicroK8s course.
What you will learn
- The GPU Operator: drivers, container toolkit, device plugin and DCGM exporter on every node, and how to upgrade the driver without stopping the whole cluster.
- Ways to share a GPU: time-slicing, MPS and MIG: what each one isolates, which cards support MIG and how its profiles are configured.
- Requesting GPUs with DRA: the dynamic resource allocation built into Kubernetes since 1.34 and NVIDIA’s DRA driver, to request a GPU by its characteristics rather than by number.
- Queues and quotas with Kueue: one queue per team, borrowing what another team is not using and priority so production can reclaim its cards.
- Measuring usage: DCGM metrics in Prometheus and Grafana per team, which is the figure you need to adjust quotas or decide whether another card is needed.
You work on a GPU cluster and finish with one card partitioned with MIG, another time-sliced, queues for three teams with borrowing between them and a per-team usage dashboard.
What you take away
- GPU Operator configurations for MIG and time-slicing.
- Kueue definitions for several teams, ready to adapt.
- A Grafana dashboard with GPU usage per team.
If you would rather we built it for you, that is our Kubernetes GPU sharing service. To serve models on that cluster there is the vLLM and llm-d on Kubernetes course.
Frequently asked questions
How long is the course?
2 days.
Is it online or on-site?
Both: at your offices or remotely, with a live instructor.
What do I need to know beforehand?
Kubernetes administration (deployments, nodes, labels and Helm) and Linux. If the team is new to Kubernetes, the starting point is the Kubernetes with MicroK8s course.
Which GPUs support MIG?
The A100, A30, H100, H200, B200 and B300, plus the RTX PRO 6000 Blackwell. The L4 and L40S do not; on those cards you share by time-slicing or by whole GPUs.
Do we need our own GPUs?
Not for the course: the lab has GPUs. If you already have GPU servers, the labs can run on them.
Can it be tailored to my team?
Yes. We adapt the examples and labs to your team’s tools and processes, and it can be combined with other AI courses.
