# Choosing a base > The three models served behind the same routes, their provenance, and when to choose each. Source: https://opendecisio.com/docs/bases/ Three base models are served behind the same routes, wire format and features, one per server, chosen with `--base`. With no flag the server serves Gemma 4 31B. [Figure: One server, one flag to pick the model: the default is the most accurate, the others are faster or smaller. Gemma 4 31B: 30.6 GiB (FP8) in memory, 80.7 ms for a new 300-token state, MMLU-Pro 0.694, 213 correct on JevBench's 231 published questions; Qwen3.6-35B-A3B: 3.3 GiB in memory, 49.9 ms for a new 300-token state, MMLU-Pro 0.613, 200 correct on JevBench's 231 published questions; Gemma 4 12B: 22.8 GiB in memory, 53.6 ms for a new 300-token state, MMLU-Pro 0.549, 200 correct on JevBench's 231 published questions.] ## TL;DR ``` # gemma-4-31b, the default uv run python -m decisio.serve.vllm_engine uv run python -m decisio.serve.vllm_engine --base qwen3.6-35b-a3b uv run python -m decisio.serve.vllm_engine --base gemma-4-12b ``` Needs an NVIDIA card: taken verbatim from the repository's tested docs, [run record](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-04_gemma-base). Its comments are shown above their commands. | Word | Meaning | | --- | --- | | base | the model a server serves, with its checkpoint at a pinned revision and every setting measured as one configuration | | FP8 | 8-bit floating point; the 31B’s weights are stored in it, quantized once with the arithmetic vLLM uses on load | | provenance | where a base’s weights come from: each is an official checkpoint, frozen, with no training or fine-tuning by us | Common mistake The 31B’s numbers were measured in its own session, on another card of the same type, so they are not paired with the other two bases’. Compare the three on the paired rows, and read the card before quoting a difference ([the evaluation card](https://opendecisio.com/docs/evaluation-card/)). ## The choice Three base models are served behind the same routes, wire format and features, one per server, chosen with `--base`; with no flag the server serves Gemma 4 31B: ``` # gemma-4-31b, the default uv run python -m decisio.serve.vllm_engine uv run python -m decisio.serve.vllm_engine --base qwen3.6-35b-a3b uv run python -m decisio.serve.vllm_engine --base gemma-4-12b ``` Needs an NVIDIA card: taken verbatim from the repository's tested docs, [run record](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-04_gemma-base). Its comments are shown above their commands. `--base` brings its checkpoint at a pinned revision and every setting below; a flag given explicitly overrides the base's value. Each base's settings were measured as one configuration. A checkpoint named with `--model` and no `--base` brings the base its `config.json` declares, as before. | Base | Checkpoint | Provenance | | --- | --- | --- | | `gemma-4-31b` (default) | `tachara-ai/decisio-gemma-4-31b`: Google's [`google/gemma-4-31B-it`](https://huggingface.co/google/gemma-4-31B-it) at `842da379`, stored as FP8, 30.6 GiB in memory; `--model google/gemma-4-31B-it` serves Google's weights at the same revision instead, quantized to FP8 when they load (vLLM 0.30.0) | Official checkpoint from Google, at a pinned revision, quantized to FP8 once with the arithmetic vLLM uses on load (`python -m decisio.hub_fp8`); no training or fine-tuning by us; the repository's card states the check that its tensors equal the ones made on load | | `qwen3.6-35b-a3b` | [`Qwen/Qwen3.6-35B-A3B-FP8`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8) at `95a723d0`, 33.3 GiB in memory | Official checkpoint from Alibaba's Qwen team, at a pinned revision; no adapter or fine-tuning by us | | `gemma-4-12b` | [`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it) at `707f0a3b`, bf16, 22.8 GiB in memory | Official checkpoint from Google, at a pinned revision; no adapter or fine-tuning by us | A base that is a third-party fine-tune names its publisher in the provenance column and states what it was trained on. | Setting | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` (default) | | --- | --- | --- | --- | | Temperatures | 1.370 for choice questions, 1.506 for yes/no and score | 3.592 for every question type | 4.672 for choice questions, 5.252 for yes/no and score | | Prompt | no system turn, read after an "Answer:" prefill, one token per option letter | a system turn, read at the chat template's own answer position, every single-token form of each letter summed | as `gemma-4-12b` | | Yes/no | a two-option letter choice with its sides named | a two-option letter choice, each side shown as its description | as `gemma-4-12b` | | Padding | the state padded to the 1,056-token block | none | none | | Several questions in one request | the same probabilities as each question sent alone, bit for bit | the same choice as each question sent alone, probabilities within 0.035 | scored in one batch after the state is read (`--multi-question warm`, its default since 0.8.0): the same choice as one engine call per question on every item measured, probabilities within 0.0073 | | Precision | FP8, as the checkpoint stores it | bf16 | FP8, as its repository stores it (on load from Google's bf16 weights with `--model google/gemma-4-31B-it`; at bf16 they leave too little of a 96 GB card for a 32,768-token context) | | A second, different question on a document already read (since 0.8.1; one RTX PRO 6000 at 585 W, AMD Ryzen Threadripper 9960X) | read from the cache, as always: 23.5 ms after an 85 ms first read (3,000 tokens) | read from the cache since 0.8.1: 35.8 ms after a 260 ms first read; since 0.9.0 the document's boundary is registered after the answer, so the first read costs nothing more (+16 to +28 ms at 300 to 3,000 tokens before) | read from the cache since 0.8.1: 43.3 ms after a 439 ms first read; the boundary is registered before the answer, which costs the first read +21 to +34 ms and keeps throughput under load | In plain words: - **Gemma 4 31B, the default, for accuracy.** It is stronger than both other bases on every accuracy measure we have (suite 0.799 against 0.770 and 0.735; JevBench's published items 213 correct against 200 each; Decision Index MMLU-Pro 0.694 against 0.613 and 0.549). It needs a 96 GB card on vLLM (its weights take 31 GB); on a 64 GB Mac it runs from its 6-bit MLX conversion for states up to 16,383 tokens ([Mac with MLX](https://github.com/aminry/decisio/blob/v0.11.1/README.md#mac-with-mlx)). - **Qwen3.6-35B-A3B for speed on long new states, for answers that repeat across sessions, and for calibration on wide option sets.** - A question on a new state reads 3.2 and 5.1 times faster than on the 31B at 1,000 and 3,000 tokens (50.0 and 85.2 ms against 160.1 and 438.1; 1.6 times at 300 tokens, 49.9 against 80.7), measured in one session on one card at 585 W ([`runs/2026-10-06_latency-585w/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-06_latency-585w)). - Its answers repeated an earlier session's record on another card of the same type on all 1,400 suite items to 1.1e-16 ([`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) 6.2), where the 31B's near-tied answers moved between sessions (7.5). - On the 77-option BANKING77 items of the suite its calibration error is 0.055 against the 31B's 0.078; the temperatures were fitted on that suite, so both are in-sample ([`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) 3, 4 and 7.2). - **Gemma 4 12B for machines without a 96 GB card.** It runs on a 32 GB Mac from its MLX conversion ([Mac with MLX](https://github.com/aminry/decisio/blob/v0.11.1/README.md#mac-with-mlx)), and its Ollama listing takes 8.6 GB loaded ([Ollama](https://github.com/aminry/decisio/blob/v0.11.1/README.md#ollama)). On vLLM its weights are the smallest of the three (22.8 GiB), but it did not start on a card limited to 32 GB, and cards between 32 and 96 GB were not measured ([`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) 6.6). - On JevBench's published items the 12B and Qwen are level (200 of 231 each). What drives the choice, measured on one RTX PRO 6000 Blackwell in one session, each base with its own defaults, paired over the same items (95% bootstrap intervals; [`runs/2026-10-04_gemma-base/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-04_gemma-base)): | Measure | Gemma 4 12B | Qwen3.6-35B-A3B | Gemma minus Qwen | | --- | --- | --- | --- | | JevBench v1.5 open-set reading, I_open (equal types) | 64.4 | 49.4 | +15.0 [+7.0, +23.6] | | JevBench v1.5, yes/no / score | 49.0 / 63.7 | 13.7 / 57.2 | | | Yes/no answers between 0.20 and 0.80 (74 items) | 15% | 39% | | | JevBench, published items, correct | | | 0.0 points [-4.3, +4.3] | | Decision Index accuracy, CLINC150+OOS | | | +4.5 [+3.6, +5.5] | | Decision Index accuracy, BANKING77 | | | -1.4 [-2.7, -0.1] | | Decision Index accuracy, GPQA Diamond | | | -13.6 [-21.7, -5.6] | | Decision Index accuracy, MMLU-Pro | | | -6.4 [-7.2, -5.5] | | 1,400-item suite, accuracy | 0.735 | 0.770 | -3.5 [-5.5, -1.6] | | One question on a new 3,000-token state, server time (0.8.1's served defaults, the engine in the server's process; one RTX PRO 6000 at 585 W, AMD Ryzen Threadripper 9960X; one session, [`runs/2026-10-06_latency-585w/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-06_latency-585w)) | 258.6 ms | 85.2 ms | | The full paired table, the Gemma base's repeatability and its limits are in [`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) section 6. Gemma 4 31B was measured in its own session, on another card of the same type ([`runs/2026-10-04_gemma-4-31b/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-04_gemma-4-31b)), so its numbers are not paired with the table above: suite accuracy 0.799 (Qwen 0.770, Gemma 4 12B 0.735), 213 of JevBench's 231 published questions correct (200 each), Decision Index MMLU-Pro 0.694 and GPQA Diamond 0.520 (Qwen 0.613 and 0.510), one question on a new 3,000-token state 438 ms (Qwen 85 ms; 0.8.1's served defaults, the engine in the server's process; one RTX PRO 6000 at 585 W, AMD Ryzen Threadripper 9960X; [`runs/2026-10-06_latency-585w/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-06_latency-585w)). It went in under the maintainer's decision, past a pre-registered rule it missed by 0.2 to 1.1 points on three of four benchmarks; [`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) section 7 states the rule, the numbers and the reason. On the four demos, run on each base in one session ([docs/demos](https://opendecisio.com/docs/demos/)), Gemma 4 31B played Pong best (85% agreement with a perfect paddle, against 60% and 62%), made the fewest driving motion errors and was most accurate on triage (97.0% against 93.8% for Gemma 4 12B and 91.0% for Qwen), but lost one of nine real-time drives. Gemma 4 12B was the most accurate browser agent (269 of 272 labelled steps, against 248 and 243) and took the shortest path in every run. Qwen3.6-35B-A3B answered fastest in Pong, driving and triage (Pong 44 ms per decision, against 48 and 68 ms; level with Gemma 4 12B in the browser agent), but made the most browser-agent errors. ## What next - [Choose your machine](https://opendecisio.com/docs/quickstart/): what each base needs to run. - [Flags and options](https://opendecisio.com/docs/flags/): `--base` and everything it sets. - [The evaluation card](https://opendecisio.com/docs/evaluation-card/): what was measured, on what, for each base.