Skip to content
Decisio

DocsRun it

2 min read · updated

Docker

Run the server in a container on a GPU host.

The container is the GPU server with its dependencies pinned, started with Docker Compose. It serves the default base, Gemma 4 31B, and needs the NVIDIA Container Toolkit and a GPU host.

Figure · In a container

Build the wheel, bring the container up, wait for /health.

Those starts were with the Qwen base, the container's default at the time; the container now serves the 31B, which needs a 96 GB card.

Source: docs/running.md at the pinned release (runs/2026-10-01_docker-first-gpu-start/).

TL;DR

Section titled “TL;DR”
# the image installs this wheel
uv build --wheel
# needs the NVIDIA Container Toolkit and a GPU host
docker compose up --build
# answers once the first start has fetched the checkpoint
curl http://127.0.0.1:8000/health

Needs a GPU host with the NVIDIA Container Toolkit: taken verbatim from the repository's tested docs, run record. Its comments are shown above their commands.

WordMeaning
wheelthe Python package the image installs, built by uv build --wheel
NVIDIA Container Toolkitwhat lets a container see the card
volumedecisio-data, where the first start downloads the checkpoint, so it is never part of the image

How the container starts

Section titled “How the container starts”
# the image installs this wheel
uv build --wheel
# needs the NVIDIA Container Toolkit and a GPU host
docker compose up --build
# answers once the first start has fetched the checkpoint
curl http://127.0.0.1:8000/health

Needs a GPU host with the NVIDIA Container Toolkit: taken verbatim from the repository's tested docs, run record. Its comments are shown above their commands.

  • The image is vLLM's 0.30.0 release image, pinned by digest, plus the decisio wheel with its serve extra; the build argument APPLY_PATCHES=1 adds the patch series (off by default).
  • The first start downloads the checkpoint into the decisio-data volume, mounted at /data; it is never part of the image.
  • The container runs as a non-root user, and compose publishes the port on 127.0.0.1 only.
  • The entrypoint starts the server with --base "${DECISIO_BASE:-gemma-4-31b}" and --model-class hidden-readout, listening on DECISIO_HOST and DECISIO_PORT; arguments after the image name are passed to the server. DECISIO_BASE names another base (qwen3.6-35b-a3b, gemma-4-12b) or a decisio repository, and DECISIO_MODEL names a checkpoint instead: alone it brings the base its config.json declares, so a container configured with DECISIO_MODEL=Qwen/Qwen3.6-35B-A3B-FP8 keeps serving the Qwen base.
  • It checks for an NVIDIA GPU first and exits with an explanation when the container cannot see one.
  • Release images go to ghcr.io/aminry/decisio, with their digest in the release notes; to run one instead of building, replace build: in compose.yaml with image: ghcr.io/aminry/decisio@sha256:<digest>.
  • Dockerfile and compose.yaml explain the rest.

First GPU start (2026-10-01, one RTX PRO 6000 Blackwell, the image built on the machine from the Dockerfile): healthy in 651 s including the checkpoint download and in 206 s from the cached volume, and the README's example request and the conformance gates C2 to C4 pass against the container (runs/2026-10-01_docker-first-gpu-start/). The image published with v0.1.0 repeated the start from its digest: healthy in 223 s with the checkpoint cached, example and conformance passing (pushed_image_0.1.0/ in the same record). Those starts were with the Qwen base, the container's default then. The no-flag vLLM start selected Gemma 4 31B on 2026-10-08 on one RTX PRO 6000 Blackwell at 600 W and AMD EPYC 9654, using decisio 0.9.0 at the PR head; /health recorded base: gemma-4-31b, and the README request produced one distinct response in 30 repeats (RLCD experiments/2026-10-08_lab2_gate_session/results/box/session_a/). The same start through a built 0.10.0 container remains to be recorded.

What next

Section titled “What next”