> ## Documentation Index
> Fetch the complete documentation index at: https://docs.expanse.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# How it works

> What Expanse runs in your network, how it learns from your jobs, and what it tells your users before and after a job.

Expanse has three parts:

* **Collectors** on your cluster's nodes record what each job asks for, what it uses and how it ends.
* **Two services inside your network** store what the collectors see and turn it into predictions and diagnoses.
* **A service run by Expanse** handles sign-in and licensing, and never sees your workload data.

Expanse architecture. Inside your network, collectors on your cluster's nodes send what they observe to the data plane, which keeps it in your databases and object storage and exchanges it with the intelligence plane. Only health, licence and aggregate counts leave your network for the Expanse control plane.

## What runs where

| Part | Where it runs | What it does |
| - | - | - |
| Collectors | On your cluster's nodes | Record what each job asked for, what it actually used, and how it ended. One runs on each node, on Kubernetes and SLURM alike, with one more on the Kubernetes cluster or the SLURM controller. |
| Data plane | Your network | Receives what the collectors send and keeps it in databases and object storage that you own. |
| Intelligence plane | Your network | Trains a model on your organisation's jobs and answers questions about them: what a job will need, and why one failed. |
| Control plane | Expanse | Signs your users in through your own identity provider, registers your clusters and checks the licence. |

Collection doesn't slow down or block jobs. If a collector or the data plane can't be reached, jobs carry on as normal and the collector holds what it saw until it can deliver it.

## Before a job runs

A researcher, or your scheduler on their behalf, asks Expanse about a job before it's submitted. For each resource the job asks for (memory, cores, GPUs and their memory, walltime) Expanse gives a value, a range where the evidence supports one, and how confident it is. Where it doesn't have enough evidence, it tells you.

A researcher who asks gets the change as an edit to their own submission, to look over and apply. A scheduler that asks applies it under your platform team's rules.

## After a job runs

When a job fails or runs badly, Expanse explains what happened from what it observed: an out-of-memory kill and what was holding the memory, a GPU fault, a stalled data loader, a job that reserved GPUs it didn't use. Every claim points at its evidence, and Expanse names anything it couldn't see. It doesn't write fixes, but it gives your engineers a prompt they can paste into their own coding agent.

## How it learns

Each deployment trains its own prediction model on its own jobs, inside your network. Nothing is pretrained on another customer's data, and your data never leaves to train anything elsewhere.

Predictions come from that model and Expanse's own analysis of each job. A language model can add to that, but it's optional, and Expanse works without one.

For the first month Expanse runs in the background. It makes and records a prediction for every job, but nobody sees a recommendation and nothing changes in how jobs are scheduled. That month gives the model its training data and lets you and us check its answers against what your jobs actually did before anyone relies on them.

## Talk to us

> **[Book a call](https://expanse.sh/contact)**
>
> We'll walk through how this would look on your cluster.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.