Storage for AI and HPC: Ceph, Lustre, GPFS or DAOS?
Storage for AI and HPC in 2026: Ceph, Storage Scale, Lustre, DAOS and BeeGFS.
You build a GPU cluster that costs as much as a building, and then you decide where the data lives. That decision — the file system — is what determines whether your GPUs compute or wait. We map the five big options, what each is good for, and how not to get it wrong.
There's a scene that repeats itself whenever we review an AI or HPC platform. The client proudly shows us the shiny new GPU cluster. Impressive TFLOPS figures. And then we ask the boring question: "and where do they read the data from?" Far too often, the answer is an NFS share bolted onto a box in the corner.
Buying eight GPUs that cost as much as a house and feeding them over NFS is the digital equivalent of putting a moped's fuel tank on a Formula 1 car: all the power is there, but it spends the day in the pits. Storage is what decides whether those GPUs work or wait — and getting it right starts with knowing what your options are and what each one is for.
IO500 production list (SC25)
the next 30 systems
FS in supercomputing
at CERN
Four concepts before comparing anything
Distributed storage piles up jargon fast, and marketing labels don't always help. Before we get into it, it's worth pinning down four ideas — they're the ones that actually separate one system from another when it's time to choose.
open()/read()) spreads each file across hundreds of servers that serve it at once: it's what classic HPC software expects. Object storage has no directories, just keys and objects over an S3 API: it's the language of data lakes and modern AI.The best storage is the one that fits your dominant workload. If your day-to-day is simulations writing giant files, you want sequential bandwidth. If it's training models over millions of images, you want fast metadata and IOPS. Mixing the two up is the most expensive design mistake we see.
Who plays in which league?
Here are the five systems that dominate serious storage for AI and HPC. There's no single winner: each has a sweet spot. The bars compare, roughly, two axes that are almost always in tension: the peak performance a system can reach and how easy it is to operate day to day.
Five systems at a glance
The essentials of each one: what type of storage it is, where it shines, the license it ships under, and which version is in production as of mid-2026.
| System | Type | Sweet spot | License | Version (2026) |
|---|---|---|---|---|
| Ceph | Unified | Hybrid AI + HPC platforms, private clouds, S3 data lakes. Object, block and file from a single cluster. | Open source LGPL | 20.2.2 "Tentacle" (Jun 2026) |
| IBM Storage Scale | Parallel POSIX | Enterprise HPC with support and SLAs, AI on NVIDIA DGX BasePOD/SuperPOD, high performance out of the box. | Commercial IBM | 5.2.x |
| Lustre | Parallel POSIX | Classic max-bandwidth supercomputing. The TOP500 standard when you have a team that knows how to run it. | Open source GPLv2 | 2.17 (Dec 2025) |
| DAOS | Object / KV | Extreme performance on pure NVMe. The fastest thing on the IO500, for workloads where latency is everything. | Open source BSD | 2.6 |
| BeeGFS | Parallel POSIX | Mid-to-large HPC clusters and research groups that want good performance without a dedicated storage team. | Community + Enterprise | 8.3.x |
What each does well (and what it doesn't)
Ceph — the one that does almost everything
Ceph is the Swiss army knife of open source distributed storage: it serves object (S3 via RGW), block (RBD) and file (CephFS) from the same cluster, self-heals, and scales by adding nodes. For AI that means RBD for the training boxes, S3 for the data lake and CephFS for parallel access — with native OpenStack and Kubernetes integration. It isn't the peak-performance king in extreme HPC, but nothing matches its versatility, and deployments like CERN's 50+ PB prove it holds up at serious scale. Version 20.2.2 "Tentacle" (June 2026) brings Crimson for erasure-coded pools and RGW encryption improvements. It's also the natural winner of the gap MinIO left when it went into "maintenance mode" in late 2025. We dig into all of this in our guide to open source storage for AI and HPC. At SIXE we work with it daily: Ceph for AI & HPC and official Ceph training.
IBM Storage Scale (GPFS) — performance with an SLA
Once GPFS, then Spectrum Scale, and today IBM Storage Scale: the same workhorse with three names. It's a mature, POSIX parallel file system with enterprise support that, in version 5.2.x, integrates NVIDIA GPUDirect Storage and certification for DGX BasePOD/SuperPOD and Grace Blackwell platforms. It's the choice when you want the performance of a parallel FS but don't want to depend on your team being able to rebuild a Lustre at three in the morning: you pay for the license, you get the SLA. We build HPC infrastructure with Storage Scale + Spectrum LSF/SLURM and run Storage Scale administration training.
Lustre — the TOP500 standard
Lustre is still, by a wide margin, the most widely used file system in supercomputing: it moves a huge share of the world's largest compute centres. It delivers extremely high sequential bandwidth and scales to exabytes. The price is operations: it's not "install and forget", and it takes real expertise to tune and recover. Version 2.17 (December 2025) continues that line, with 2.18 preparing file-level erasure coding (FLR-EC) and client-side compression. If you're coming from Lustre and want commercial support, we've written about migrating from Lustre to IBM Storage Scale.
DAOS — the rocket that came back
DAOS is the answer to "what if we throw POSIX out and design for NVMe from scratch?" The result: it holds positions 1 and 2 on the IO500 production list (Argonne and LRZ, SC25), and 16 of the top 30 in the full ranking run DAOS. It was born tied to Intel's Optane memory; when Intel killed Optane, many wrote DAOS off too. But since version 2.6 (July 2024) it runs on plain NVMe SSDs and is very much alive. It's now governed by the DAOS Foundation under the Linux Foundation (Argonne, HPE, Google Cloud, Intel), after Intel handed its team to HPE in 2024. It's the top-performance option for anyone with pure NVMe and a workload that justifies it — not a general replacement.
BeeGFS — the easy one to live with
BeeGFS is the parallel FS that prioritises being deployable on a Tuesday afternoon without a PhD in storage. It delivers very good performance for mid-sized clusters and is a favourite of many research groups for its simplicity. Watch out for one important change: since BeeGFS 8 the Community Edition is still free but requires generating a community license (beegfs license), and enterprise features sit behind a paid license. The stable release in 2026 is 8.3.x.
Notice the tension: Lustre and DAOS deliver top performance but demand an expert team. Storage Scale buys that performance with a license and support. Ceph and BeeGFS give up a little peak in exchange for an easier life. No option is free: each one trades performance for simplicity, or license cost for support.
Is your workload classic HPC or AI?
Before you look at brands, look at your workload. The question that orders the decision isn't "which is best?" — it's what you're going to ask of the storage. Here are the two profiles and what each one needs.
- Simulation, CFD, molecular dynamics, weather
- Writes huge checkpoints sequentially
- What it wants: sustained GB/s of bandwidth
- Large files, more predictable access
- Fits: Lustre, Storage Scale, BeeGFS
- Training and inference over massive datasets
- Reads millions of small files at random
- What it wants: IOPS, fast metadata and
GPUDirect - Must feed the GPUs without leaving them idle
- Fits: DAOS, Storage Scale, Ceph (S3)
Most modern platforms mix the two, which is why open source storage for AI and HPC tends toward hybrid architectures: a parallel FS for the compute side and S3 object storage for the data lake, often coexisting on Ceph + OpenStack + Kubernetes.
Four steps before signing anything
Define your dominant workload
Before looking at products, measure. Do you write a few giant files or read millions of small ones? Is bandwidth or metadata the bottleneck? That single answer already rules out half the options. And don't just plan for today: size for the workload you'll have once the AI project grows.
Decide: open source with a team, or support with an SLA
Lustre and DAOS are free to license but expensive to operate: you need people who know them. Storage Scale flips that equation: you pay for the license and buy peace of mind. Ceph and BeeGFS sit in between. The honest question is: do you have — or want to have — a storage team?
Look at the ecosystem you already run
Already running OpenStack or Kubernetes? Ceph fits like a glove. Have NVIDIA DGX / BasePOD? Storage Scale is certified. Pure NVMe and thirsty for latency? DAOS. The best storage is usually the one that creates the least friction with what's already in production.
Test at scale before you commit
A four-node benchmark won't predict how forty nodes behave. Test with your real workload, not the datasheet's synthetic one, and watch the metadata under concurrency. Experience here saves months — and a six-figure wrong purchase.
Designing the storage layer of an AI or HPC platform, comparing Ceph, Storage Scale, Lustre, DAOS or BeeGFS for your case, or migrating from one to another without losing data or sleep — at SIXE we've spent more than 15 years at the crossroads of distributed storage, IBM Power and open source. See Ceph support and IBM Storage.
The essentials in 5 points
→ Storage decides whether your GPUs work or wait. It's as strategic as the compute cluster itself.
→ There's no "best": there's the best for your workload. Classic HPC wants bandwidth; AI wants metadata, IOPS and GPUDirect.
→ DAOS leads on performance (1st and 2nd on the IO500), Lustre on prevalence, Storage Scale on enterprise support.
→ Ceph is the versatile choice for hybrid AI + HPC architectures; BeeGFS, the easiest to deploy.
→ Before choosing: define the workload, decide open source vs. SLA, look at your ecosystem, and test at real scale.
Frequently asked questions
What is better for AI, Ceph or Lustre?
It depends on the workload. Lustre delivers more sequential bandwidth and is the standard in classic supercomputing, but it takes real expertise to operate. Ceph is more versatile: it serves object (S3), block and file at once, integrates with OpenStack and Kubernetes, and its object storage fits AI data lakes. For large-scale training with huge checkpoints, choose Lustre or Storage Scale; for hybrid platforms and data lakes, Ceph.
Is DAOS ready for production?
Yes, in its niche. It holds positions 1 and 2 on the IO500 production list (SC25) and is governed by the DAOS Foundation under the Linux Foundation. Since version 2.6 (July 2024) it runs on plain NVMe SSDs, without Optane. It's the top-performance option, but it needs NVMe hardware and a skilled team; it's not a general replacement for Lustre or Ceph.
Is Storage Scale the same as GPFS or Spectrum Scale?
Yes. It's the same IBM product under different names: GPFS, then IBM Spectrum Scale, and today IBM Storage Scale. In 2026 it's on 5.2.x and supports NVIDIA GPUDirect Storage and DGX BasePOD/SuperPOD certification.
Is BeeGFS still free?
The Community Edition is still free, but since BeeGFS 8 you have to generate a community license with beegfs license. Enterprise features sit behind a paid license. The stable release in 2026 is 8.3.x.
Can you use Ceph for serious HPC?
Ceph appears on the IO500 production list and there are massive deployments (CERN runs over 50 PB), but for extreme-performance HPC it usually doesn't replace Lustre, Storage Scale or DAOS. Its strength is hybrid HPC + AI architectures: object, block and file from a single cluster, integrated with OpenStack and Kubernetes.
Do your GPUs compute or wait?
Tell us what workload you're moving, how many GPUs, and what you run today. We'll get back to you within 24 hours with a storage architecture sketch and a realistic idea of effort. If Ceph fits, we'll tell you. If yours is Lustre, Storage Scale or DAOS, we'll tell you that too.
