Skip to content
Decisio

DocsRun it

1 min read · updated

CPU stand-in for development

Serve every route from a small model on the CPU, to build against the API anywhere.

The stand-in serves the same routes from a small model, so you can develop and test a client without a card. It is not the measured system, and its answers are not for measurement.

Figure · The stand-in

Every route, from a small model on the CPU, so you can build against the API anywhere.

CI runs the stand-in on a pinned revision of the model, and the commands on these pages that it can serve are run against it.

Source: docs/running.md "CPU stand-in for development" at the pinned release.

TL;DR

Section titled “TL;DR”
uv sync --extra dev --frozen
uv run pytest
uv run python -m decisio.serve.vllm_engine --backend hf --model Qwen/Qwen3-0.6B-Base

Run against the CPU stand-in at v0.11.1 on 2026-10-14: the install and the server start; the unit tests passed in decisio's own CI on this commit.

The commands on these pages that the stand-in can serve are run against it.

WordMeaning
stand-in--backend hf: every route, from Qwen/Qwen3-0.6B-Base on the CPU
unit testsuv run pytest, which runs on any machine; the GPU tests need a card

What it is, and is not

Section titled “What it is, and is not”
uv sync --extra dev --frozen
uv run pytest
uv run python -m decisio.serve.vllm_engine --backend hf --model Qwen/Qwen3-0.6B-Base

Run against the CPU stand-in at v0.11.1 on 2026-10-14: the install and the server start; the unit tests passed in decisio's own CI on this commit.

  • uv run pytest runs the unit tests on any machine; the GPU tests (pytest -m gpu) need a card, and run by hand until a nightly GPU runner is registered.
  • --backend hf serves every route from a small Hugging Face model on the CPU, for developing and testing the routes; it is not the measured system and its answers are not for measurement.
  • CI runs the stand-in on Qwen/Qwen3-0.6B-Base at a pinned revision (STAND_IN_REVISION in .github/workflows/ci.yml).
  • examples/tasks/ walks through task registration against it.

What next

Section titled “What next”