# On a cloud GPU

> Run decisio on one NVIDIA card, serve the default model on port 8000, pin what it serves, and keep it running under a supervisor.

Source: https://opendecisio.com/docs/deploy/cloud-gpu/

Install it from PyPI and start it: it serves the default model, Gemma 4 31B, on `http://127.0.0.1:8000`, ready once `/health` answers.

 

```
python3.12 -m venv .venv && . .venv/bin/activate
pip install "decisio[serve]==0.18.1"
# serves Gemma 4 31B; the first start downloads about 31 GB
python -m decisio.serve.vllm_engine
# answers once it is ready
curl -s http://127.0.0.1:8000/health
```

 

Shown, not run: it needs Linux with one NVIDIA card, which this check does not have; decisio's README timed the same install, of decisio 0.11.0, on 2026-10-09. Its comments are shown above their commands.

 

[Figure: The server listens on the machine itself; a proxy in front lets your backend in. Your backend calls a reverse proxy that authenticates callers, since decisio has no authentication. The proxy passes requests to the decisio server on 127.0.0.1:8000, which runs under a restart policy and exits with code 70 when its engine dies. Its engine runs on one NVIDIA card, from weights on disk, fetched once.]

 

## 1 Pick your card

  

| Model | Weights in memory | Measured on | Longest text, with no flag |
| --- | --- | --- | --- |
| Gemma 4 31B, the default | 30.6 GiB, FP8; about 31 GB to download | one 96 GB card; also starts and answers on one 80 GB H100 PCIe | the largest that fits, 32,768 to 98,304 tokens |
| Gemma 4 12B | 22.8 GiB, bf16 | one 96 GB card; served 65,536 tokens on the same H100 | 32,768 to 65,536 tokens |
| Qwen3.6-35B-A3B | 33.3 GiB, FP8; about 36 GB to download | one 96 GB card | 32,768 tokens |

 

Start another model with `--base gemma-4-12b` or `--base qwen3.6-35b-a3b`. [Choosing a model](https://opendecisio.com/docs/bases/) compares them; [Requirements and limits](https://opendecisio.com/docs/limits/) has every machine.

  

| Part | What you need |
| --- | --- |
| System | Linux, with one NVIDIA card |
| Driver | one that supports CUDA 13.0 |
| Python | 3.12 |
| Disk | room for the first download, in the Hugging Face cache |

 

## 2 Start the server

 

The first start downloads the weights and loads them onto the card. The 31B was ready after 7 min 54 s with the download, and after 4 min 3 s with the weights already on disk.

 

Measured on decisio 0.11.0 from PyPI, one RTX PRO 6000 Blackwell at 600 W, 2026-10-09.

 

It is ready when `/health` answers 200, with the model and the longest text it serves:

 

```
{
 "ok": true,
 "base": "gemma-4-31b",
 "max_model_len": 98304,
 "max_model_len_source": "auto: the base's ceiling of 98,304 fits on this card",
 "max_model_len_ceiling": 98304,
 "engine_process": "in",
 "vllm": "0.31.0",
 "gpu": "NVIDIA RTX PRO 6000 Blackwell Workstation Edition"
}
```

 

Recorded output.

 

Recorded on Gemma 4 31B from its FP8 checkpoint, decisio 0.17.0, one RTX PRO 6000 at 600 W, 2026-10-19; trimmed to the fields a reader checks.

 

pip resolves its own versions around the engine. For exactly the versions decisio was measured with, install from a clone of the release with its lock file:

 

```
git clone --branch v0.18.1 https://github.com/aminry/decisio
cd decisio
uv sync --extra serve --frozen
uv run python -m decisio.serve.vllm_engine
```

 

Shown, not run: it needs Linux with one NVIDIA card, which this check does not have.

 

## 3 Pin what you serve

 

Name the model repository at a commit, so a later change to it cannot change your answers:

 

```
# its current commit, printed
uv run python -m decisio.serve.vllm_engine --base tachara-ai/decisio-gemma-4-31b
# a pinned commit
uv run python -m decisio.serve.vllm_engine --base tachara-ai/decisio-gemma-4-31b@<commit>
```

 

Needs Linux with one NVIDIA card: taken as it is from decisio's checked docs. Its comments are shown above their commands.

 

- The server checks the repository’s settings against its own and refuses to start on a difference, naming what differs.
- Pin `--max-model-len 32768` unless you need longer texts: `--max-model-len 32768` serves only what was measured, and the same value on every card keeps one certificate valid on all of them, since the length served is part of what it covers.
- After upgrading decisio, certify your [tasks](https://opendecisio.com/docs/glossary/#task) again ([Upgrade](https://opendecisio.com/docs/upgrading/)).

 

## 4 Put it behind a proxy

 

- The server listens on `127.0.0.1:8000`; `--host` and `--port` change that.
- It has no authentication. Put a reverse proxy that authenticates the caller in front of it, and call it from your backend.

 

## 5 Keep it running

 

When its engine dies, the server answers 503 and exits with code 70, so a supervisor can start a new one. With systemd, for a clone in `/opt/decisio` and a user `decisio`:

 /etc/systemd/system/decisio.service

```
[Unit]
Description=decisio server
After=network-online.target
Wants=network-online.target

[Service]
User=decisio
WorkingDirectory=/opt/decisio
Environment=HF_HOME=/var/lib/decisio/hf
ExecStart=/usr/local/bin/uv run --no-sync python -m decisio.serve.vllm_engine --host 127.0.0.1 --port 8000
Restart=on-failure
RestartSec=5
# a first start downloads the checkpoint and loads the engine: do not let systemd give up on it
TimeoutStartSec=infinity

[Install]
WantedBy=multi-user.target
```

 

Shown, not run: decisio wrote it for its docs and has not run it; the options are systemd's documented ones.

 

On Kubernetes, one GPU, the weights on a volume, and `/health` as the startup and liveness probe:

 

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: decisio
spec:
  replicas: 1
  selector:
    matchLabels: {app: decisio}
  template:
    metadata:
      labels: {app: decisio}
    spec:
      restartPolicy: Always            # the default: the exit with code 70 starts a new container
      containers:
        - name: decisio
          image: ghcr.io/aminry/decisio@sha256:5f758e5cfbf42892f42f62e037f26376154c3e77c5caf6afdd6ccc1a5cd9c4ee
          env:
            - {name: DECISIO_BASE, value: gemma-4-31b}
          ports:
            - containerPort: 8000
          resources:
            limits:
              nvidia.com/gpu: 1
          volumeMounts:
            - {name: data, mountPath: /data}
            - {name: shm, mountPath: /dev/shm}
          startupProbe:                # the first start downloads the checkpoint: 45 minutes, as the image's HEALTHCHECK
            httpGet: {path: /health, port: 8000}
            periodSeconds: 15
            failureThreshold: 180
          livenessProbe:               # /health answers 503 once the engine has died, before the process exits
            httpGet: {path: /health, port: 8000}
            periodSeconds: 30
            timeoutSeconds: 10
            failureThreshold: 3
      volumes:
        - name: data
          persistentVolumeClaim: {claimName: decisio-data}
        - name: shm
          emptyDir: {medium: Memory, sizeLimit: 8Gi}
```

 

Shown, not run: decisio wrote it for its docs, with the release image in place of its placeholder, and has not run it; the probes read /health as measured.

 

- Each probe reads `/health`: 200 once the engine is loaded, 503 from the moment it has died.
- Each replica is a separate server with its own registered tasks, which live in memory.

 

## What next

 

- [With Docker](https://opendecisio.com/docs/deploy/docker/): the same server from the release image, with Compose.
- [Requirements and limits](https://opendecisio.com/docs/limits/): every machine, the context length and what was measured on what.
- [Operate](https://opendecisio.com/docs/operating/): health, monitoring and what to watch in production.