Deploying and operating

A decisio server is a model server, not a hardened public endpoint: run it on a private network, behind your own proxy, under something that restarts it.

Figure · A server's life

A dead engine does not serve again: the server says so and exits, and something restarts it.

1. Startingdownloads the checkpoint on the first start, then warms the engine
2. Healthy/health answers 200; requests are served
3. The engine diesa failed forward pass or an exited engine process, confirmed before it is declared
4. Every request gets 503/health answers 503 with the reason
5. Exit code 70about 2 seconds later, once the requests in flight have their answers
6. Your restart policybrings a new server, which loads the engine again

Run the server under a restart policy: Docker Compose `restart: unless-stopped`, systemd `Restart=on-failure`, or Kubernetes' default `restartPolicy: Always` with a liveness probe on `/health`.

Source: docs/running.md "When the engine dies" at the pinned release (0.8.0 and later).

TL;DR

Wait until a server you started is ready, then check what it serves:

until curl -sf http://127.0.0.1:8000/health > /dev/null; do sleep 5; done
curl -s http://127.0.0.1:8000/health

Run against the CPU stand-in at v0.11.1 on 2026-10-14.

Word	Meaning
restart policy	what brings the server back after it exits: Compose’s restart: unless-stopped, systemd’s Restart=on-failure, or Kubernetes’ restartPolicy: Always
exit code 70	what the server exits with when its engine has died
single-tenant	one server per trust boundary: its cache and registered tasks are shared by every caller

Common mistake

Docker restarts a container when its process exits, not when its HEALTHCHECK reports it unhealthy. It is the server’s exit that brings a new one up, so give the container a restart policy.

1. Run it under a restart policy

An engine that has failed does not serve again, so the server ends itself and leaves the restart to whatever runs it: compose restart: unless-stopped, as compose.yaml has it; systemd Restart=on-failure; Kubernetes’ default restartPolicy: Always, with a liveness probe on /health. The new server loads the engine again, so requests sent until it is healthy are refused. Health and monitoring says exactly what happens from the moment the engine dies.

2. Keep tasks across restarts

Registered tasks live in the server’s memory. Teach it your question shows how to export them and load them again at start.

3. Put a proxy in front

Decisio is a model server, not a hardened public endpoint.

• It is designed to run behind your own proxy on a private network. The server binds to 127.0.0.1 by default; do not expose it directly to the internet.
• There is no authentication on any route. Put authentication, rate limiting and request-size limits in the proxy.
• POST /v1/tasks, POST /v1/tasks/import, DELETE /v1/tasks/{id} and the abstention routes change server state. Restrict them to trusted callers.
• --debug-readout exposes raw model readouts; leave it off in production.
• The image route accepts image payloads from requests; size limits belong in the proxy.
• vLLM's own security guidance applies to the engine underneath: https://docs.vllm.ai/en/stable/usage/security/

A decisio server is single-tenant. Its prefix cache and registered tasks are shared by every caller, so response timing can reveal whether another caller recently sent the same text, and one caller's registered task answers another's identical question. Run one server per trust boundary.

What next

• Health and monitoring: what to watch, and what a dead engine looks like.
• Troubleshooting: symptoms and what the documentation says to do.
• Upgrading: what changes for a running deployment between releases.