Description
A course for systems teams who want a language model running on their own servers and need to know what hardware it takes before buying it. You deploy an LLM behind an access-controlled API, measure how much it can handle and find where the bottleneck is.
Course details
| SIXE code: SXIA03 | Category: Artificial Intelligence / Infrastructure |
| Delivery: Online and on-site | Duration: 3 days |
Who it is for
Linux administrators, systems architects, platform and data centre teams who have to get an LLM running locally or decide which servers to buy for it.
Prerequisites
Linux, containers, HTTP and reading logs. No experience training models or administering Kubernetes is needed.
What you will learn
- Choosing the model: weights, licences, quantisation, context length and how much memory each option needs.
- Serving it through an API: vLLM or another compatible runtime, with authentication, limits and a record of who uses it.
- Measuring: latency, memory and concurrency under a defined load, comparing two configurations.
- Finding the bottleneck: CPU, GPU, storage or network, with metrics you can defend; for example, a GPU sitting idle because data arrives too slowly.
- Running it: updating the model, rolling back to the previous version and preparing the capacity plan for the pilot.
You work on an isolated Linux server with a model whose licence has been checked, and finish with a reproducible service, its measurements and the procedure to update it.
What you take away
- The service configuration, with its access control, ready to reproduce.
- Quality, memory and latency measurements for the configurations tested.
- A map of which data leaves the server and the update and rollback procedure.
For background, we have written about why companies are running their own AI servers and how to run vLLM on IBM Power without a GPU.
Frequently asked questions
How long is the course?
3 days.
Is it online or on-site?
Both: at your offices or remotely, with a live instructor.
What do I need to know beforehand?
Linux, containers, HTTP and reading logs. No experience training models or administering Kubernetes is needed.
Does a local model guarantee that no data leaves the server?
Not on its own. The course reviews downloads, telemetry, integrations and connections to check what leaves and what does not.
Which GPU do I need?
It depends on the quality, concurrency, context and latency you need. The course teaches you to measure it before choosing hardware.
Can it be tailored to my team?
Yes. We adapt the examples and labs to your team’s tools and processes, and it can be combined with other AI courses.
