On a cloud GPU

Run decisio on one NVIDIA card, serve the default model on port 8000, pin what it serves, and keep it running under a supervisor.

Install it from PyPI and start it: it serves the default model, Gemma 4 31B, on http://127.0.0.1:8000, ready once /health answers.

python3.12 -m venv .venv && . .venv/bin/activate
pip install "decisio[serve]==0.18.1"
# serves Gemma 4 31B; the first start downloads about 31 GB
python -m decisio.serve.vllm_engine
# answers once it is ready
curl -s http://127.0.0.1:8000/health

Shown, not run: it needs Linux with one NVIDIA card, which this check does not have; decisio's README timed the same install, of decisio 0.11.0, on 2026-10-09. Its comments are shown above their commands.

Figure · On one card

The server listens on the machine itself; a proxy in front lets your backend in.

The proxy and the restart policy are yours; the steps below have a systemd unit and a Kubernetes Deployment.

1. Pick your card

Model	Weights in memory	Measured on	Longest text, with no flag
Gemma 4 31B, the default	30.6 GiB, FP8; about 31 GB to download	one 96 GB card; also starts and answers on one 80 GB H100 PCIe	the largest that fits, 32,768 to 98,304 tokens
Gemma 4 12B	22.8 GiB, bf16	one 96 GB card; served 65,536 tokens on the same H100	32,768 to 65,536 tokens
Qwen3.6-35B-A3B	33.3 GiB, FP8; about 36 GB to download	one 96 GB card	32,768 tokens

Start another model with --base gemma-4-12b or --base qwen3.6-35b-a3b. Choosing a model compares them; Requirements and limits has every machine.

Part	What you need
System	Linux, with one NVIDIA card
Driver	one that supports CUDA 13.0
Python	3.12
Disk	room for the first download, in the Hugging Face cache

2. Start the server

The first start downloads the weights and loads them onto the card. The 31B was ready after 7 min 54 s with the download, and after 4 min 3 s with the weights already on disk.

Measured on decisio 0.11.0 from PyPI, one RTX PRO 6000 Blackwell at 600 W, 2026-10-09.

It is ready when /health answers 200, with the model and the longest text it serves:

{
 "ok": true,
 "base": "gemma-4-31b",
 "max_model_len": 98304,
 "max_model_len_source": "auto: the base's ceiling of 98,304 fits on this card",
 "max_model_len_ceiling": 98304,
 "engine_process": "in",
 "vllm": "0.31.0",
 "gpu": "NVIDIA RTX PRO 6000 Blackwell Workstation Edition"
}

Recorded output.

Recorded on Gemma 4 31B from its FP8 checkpoint, decisio 0.17.0, one RTX PRO 6000 at 600 W, 2026-10-19; trimmed to the fields a reader checks.

pip resolves its own versions around the engine. For exactly the versions decisio was measured with, install from a clone of the release with its lock file:

git clone --branch v0.18.1 https://github.com/aminry/decisio
cd decisio
uv sync --extra serve --frozen
uv run python -m decisio.serve.vllm_engine

Shown, not run: it needs Linux with one NVIDIA card, which this check does not have.

3. Pin what you serve

Name the model repository at a commit, so a later change to it cannot change your answers:

# its current commit, printed
uv run python -m decisio.serve.vllm_engine --base tachara-ai/decisio-gemma-4-31b
# a pinned commit
uv run python -m decisio.serve.vllm_engine --base tachara-ai/decisio-gemma-4-31b@<commit>

Needs Linux with one NVIDIA card: taken as it is from decisio's checked docs. Its comments are shown above their commands.

The server checks the repository’s settings against its own and refuses to start on a difference, naming what differs.
Pin --max-model-len 32768 unless you need longer texts: --max-model-len 32768 serves only what was measured, and the same value on every card keeps one certificate valid on all of them, since the length served is part of what it covers.
After upgrading decisio, certify your tasks again (Upgrade).

4. Put it behind a proxy

The server listens on 127.0.0.1:8000; --host and --port change that.
It has no authentication. Put a reverse proxy that authenticates the caller in front of it, and call it from your backend.

5. Keep it running

When its engine dies, the server answers 503 and exits with code 70, so a supervisor can start a new one. With systemd, for a clone in /opt/decisio and a user decisio:

[Unit]
Description=decisio server
After=network-online.target
Wants=network-online.target

[Service]
User=decisio
WorkingDirectory=/opt/decisio
Environment=HF_HOME=/var/lib/decisio/hf
ExecStart=/usr/local/bin/uv run --no-sync python -m decisio.serve.vllm_engine --host 127.0.0.1 --port 8000
Restart=on-failure
RestartSec=5
# a first start downloads the checkpoint and loads the engine: do not let systemd give up on it
TimeoutStartSec=infinity

[Install]
WantedBy=multi-user.target

Shown, not run: decisio wrote it for its docs and has not run it; the options are systemd's documented ones.

On Kubernetes, one GPU, the weights on a volume, and /health as the startup and liveness probe:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: decisio
spec:
  replicas: 1
  selector:
    matchLabels: {app: decisio}
  template:
    metadata:
      labels: {app: decisio}
    spec:
      restartPolicy: Always            # the default: the exit with code 70 starts a new container
      containers:
        - name: decisio
          image: ghcr.io/aminry/decisio@sha256:5f758e5cfbf42892f42f62e037f26376154c3e77c5caf6afdd6ccc1a5cd9c4ee
          env:
            - {name: DECISIO_BASE, value: gemma-4-31b}
          ports:
            - containerPort: 8000
          resources:
            limits:
              nvidia.com/gpu: 1
          volumeMounts:
            - {name: data, mountPath: /data}
            - {name: shm, mountPath: /dev/shm}
          startupProbe:                # the first start downloads the checkpoint: 45 minutes, as the image's HEALTHCHECK
            httpGet: {path: /health, port: 8000}
            periodSeconds: 15
            failureThreshold: 180
          livenessProbe:               # /health answers 503 once the engine has died, before the process exits
            httpGet: {path: /health, port: 8000}
            periodSeconds: 30
            timeoutSeconds: 10
            failureThreshold: 3
      volumes:
        - name: data
          persistentVolumeClaim: {claimName: decisio-data}
        - name: shm
          emptyDir: {medium: Memory, sizeLimit: 8Gi}

Shown, not run: decisio wrote it for its docs, with the release image in place of its placeholder, and has not run it; the probes read /health as measured.

Each probe reads /health: 200 once the engine is loaded, 503 from the moment it has died.
Each replica is a separate server with its own registered tasks, which live in memory.

What next

With Docker: the same server from the release image, with Compose.
Requirements and limits: every machine, the context length and what was measured on what.
Operate: health, monitoring and what to watch in production.