# Flags and options

> The behaviour flags, then every flag with its default per base.

Source: https://opendecisio.com/docs/flags/

The served defaults need no flags: every default is what was measured, base by base. A flag is for the few times you want something else, and an explicit flag overrides the base’s value.

 

[Figure: 48 flags in 8 groups; the defaults are what was measured, so most need none. The server's flags by group: Model and engine, 14; The prompt, 8; Temperatures and outputs, 7; Several questions and padding, 6; Tasks and abstention, 7; Images, 3; The Mac (--backend mlx), 2; Verification, 1.]

 

## TL;DR

 

The base is the one flag most deployments set; with none, the server serves Gemma 4 31B:

 

```
# gemma-4-31b, the default
uv run python -m decisio.serve.vllm_engine
uv run python -m decisio.serve.vllm_engine --base qwen3.6-35b-a3b
uv run python -m decisio.serve.vllm_engine --base gemma-4-12b
```

 

Needs an NVIDIA card: taken verbatim from the repository's tested docs, [run record](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-04_gemma-base). Its comments are shown above their commands.

  

| Word | Meaning |
| --- | --- |
| base | the model the server serves, chosen with `--base`, which brings its checkpoint and every setting marked “the base’s” |
| served default | the value a flag has when you give none, measured as one configuration per base |
| backend | `vllm` (the default), `mlx` for Apple silicon, `hf` for the CPU stand-in |

 

Common mistake

`--multi-question warm`, `--pad-policy row` and `--register-boundary after` trade something for speed (an answer that depends on the batch, a little repeatability, throughput). Each has its cost in the reference below; turn one on knowing it, and measure on your own questions.

 

## The behaviour flags

 

The served defaults need no flags; these change behaviour ([`docs/cli.md`](https://opendecisio.com/docs/flags/) has every flag and its default per base):

 

- `--pad-policy row`: pads a single-question request so its whole row ends on the block boundary, faster on a new state at a small cost in repeatability.
- `--multi-question warm`: scores a request's questions in one batch for bulk scoring, faster, but each answer then depends on the batch. It is the Gemma 4 31B base's default: with the engine in the server's process a warm request repeats exactly within one session on one machine (not across sessions, [`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) section 7.5), and on the items measured it chose as `sequential` did every time ([`runs/2026-10-05_engine-death-gates/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-05_engine-death-gates)).
- `--noul-commit`: reports a yes/no probability inside JevBench v1.5's no-answer band at the band's edge; the answer never changes, its calibration does.
- `--prompt-tail compact`: the earlier layout, for tasks registered under it.
- `--image-model`: a second engine on the same card for requests that carry images.
- `--backend`: `vllm` (the default), `mlx` for Apple silicon, `hf` for the CPU stand-in.

 

## Every flag

 

```
uv run python -m decisio.serve.vllm_engine [flags]
```

 

A shape to fill in, not a command to run.

 

`--base` chooses the base model and brings its checkpoint at a pinned revision and its value for every setting marked "the base's" below; a flag given explicitly overrides it. With a `--model` and no `--base`, the base is detected from the checkpoint's `config.json`. With neither, the vLLM server serves `gemma-4-31b`, the default base since 0.10.0 (the CPU stand-in and the MLX backend always name a checkpoint, and refuse to start without one). `--base` also takes a decisio model repository, `owner/name[@revision]` or a directory holding its `decision_config.json`: the repository's weights are the checkpoint, its file names the base, and the server refuses to start if the file differs from the installed profile ([`docs/running.md`](https://github.com/aminry/decisio/blob/v0.11.1/docs/running.md), "Model repositories"). The served defaults need no other flag: every default below is what was measured, base by base ([`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) sections 1, 6.1 and 7.1). `gemma-4-31b` is served from its FP8 repository, `tachara-ai/decisio-gemma-4-31b`, and `--model google/gemma-4-31B-it` serves Google's weights instead, quantized to FP8 when they load (vLLM 0.30.0's FP8 on load), a setting of its profile; `--engine '{"quantization": null}'` loads them at bf16, which does not leave room for a 32,768-token context on a 96 GB card ([`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) section 7). `--help` prints the same list with each flag's description.

 

### Every flag

 

Where the columns differ, the value is the base's.

 

#### Model and engine

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--base` |  |  | the default | the base, or a decisio model repository (`owner/name[@revision]`, or a directory); without it, detected from `--model`, or the default if no `--model` either |
| `--model` | [`Qwen/Qwen3.6-35B-A3B-FP8`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8) | [`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it) | [`google/gemma-4-31B-it`](https://huggingface.co/google/gemma-4-31B-it) | the checkpoint, a local directory or a Hugging Face repository id |
| `--revision` | `95a723d08a9490559dae23d0cff1d9466213d989` | `707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7` | `842da3794eaa0b77d5f08bae87a17459d91ff475` | the checkpoint's revision on the Hub; the pin applies whenever `--model` is the base's own repository |
| `--backend` | `vllm` | `vllm` | `vllm` | `vllm`; `mlx` for Apple silicon (the three bases, [`docs/running.md`](https://github.com/aminry/decisio/blob/v0.11.1/docs/running.md)); `hf` for the CPU stand-in, not for measurement |
| `--model-class` | `hidden-readout` | `hidden-readout` | `hidden-readout` | `hidden-readout`: decisio's text class that also returns the hidden state at the answer position, for the intent head; `text-only`: the same without it (the default with `--head-engine`); `view`: a directory built by `decisio.serve.make_text_only`, loaded as it is, needing no plugin |
| `--mode` | `separate` | `separate` | `separate` | `separate`: one prompt per question, the state shared through the prefix cache; `packed`: questions packed into one pooling request (the compact layout only) |
| `--pack` | 16 | 16 | 16 | questions per pack in packed mode |
| `--engine` | `{"compilation_config": {"max_cudagraph_capture_size": 4096}}` | the same | the same | extra `LLM(...)` keyword arguments, as JSON |
| `--adapter` | none | none | none | `name=path` of a vLLM-format LoRA adapter; repeatable |
| `--gpu-memory-utilization` | 0.9 | 0.9 | 0.9 | the text engine's share of the card |
| `--allow-deep-gemm` | off | off | off | start even when `VLLM_USE_DEEP_GEMM` is set to something other than 0 |
| `--served-name` | `decisio-qwen3.6-35b-a3b-letters` | `decisio-gemma-4-12b-it-letters` | `decisio-gemma-4-31b-it-letters` | the name `GET /v1/models` lists |
| `--host`, `--port` | `127.0.0.1`, 8000 | the same | the same | the listen address; put a reverse proxy in front to expose it |

 

#### The prompt

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--prompt-tail` | `spaced` | `spaced` | `spaced` | `spaced`: a blank line before and after the lettered options, then one line asking for the chosen option's letter alone; `compact`: the earlier layout, for tasks registered under it |
| `--answer-slot` | `prefill` | `template` | `template` | where the label is read: after "Answer:" prefilled in the assistant turn, or at the chat template's own first assistant position |
| `--label-variants` | `single` | `summed` | `summed` | the tokens read per label: the one form the slot reads, or every single-token form summed |
| `--system-prompt` | off | on | on | a system turn before each question |
| `--noul-rendering` | `letters-keys` | `letters` | `letters` | how a yes/no question is asked: `letters-keys`, a two-option letter choice, the false side first, the sides named; `letters`, the same with each side shown as its description; `words`, the earlier rendering, read from the yes and no tokens |
| `--describe-options` | on | on | on | show an option that has a description as its description alone; `--no-describe-options` shows `key: description` |
| `--hide-index-keys` | on | on | on | never show enumerated keys (`option_0`, `option_1`, ...) |
| `--desnake-labels` | on | on | on | show bare snake_case labels as words |

 

#### Temperatures and outputs

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--temperature` | 1.506 | 3.592 | 5.252 | the temperature on the text route's plain readout, `softmax(log p / T)`; 1 switches it off; never changes the most probable option |
| `--temperature-choice` | 1.370 | the global one | 4.672 | the temperature for choice questions |
| `--temperature-noul` | the global one | the global one | the global one | the temperature for yes/no questions |
| `--temperature-score` | the global one | the global one | the global one | the temperature for score questions |
| `--noul-commit` | off | off | off | report an uncommitted yes/no probability at the band's edge (below) |
| `--orders` | 1 | 1 | 1 | 2: two-order averaging, each question also read in a second option order |
| `--branch-log` | none | none | none | a JSONL file for the two orders' disagreement per question |

 

#### Several questions and padding

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--multi-question` | `sequential` | `sequential` | `warm` with the engine in the server's process, else `sequential` | how a request's questions are scored (below) |
| `--register-boundary` | not used (no registration needed) | `before` (`after` from 0.9.0 to 0.11.0) | `before` | when a single question on a new state registers its boundary for the next question: `after`, once the response is out; `before`, ahead of the question, as in 0.8.1; `off`, never ([`docs/running.md`](https://github.com/aminry/decisio/blob/v0.11.1/docs/running.md)) |
| `--engine-process` | `in` (`separate` with a second engine) | `in` (`separate` with a second engine) | `in` | where vLLM's engine runs: `in`, the server's process; `separate`, a process of its own (vLLM's arrangement) (below) |
| `--pad-policy` | `always` | `always` | `always` | `always`: pad every state; `shared`: pad only requests with more than one question; `row`: pad a single-question request's whole row (below); `none`: never pad (the default with `--backend mlx`) |
| `--pad-to` | `block` | `none` | `none` | what a state is padded to: the KV cache block, a token count, or nothing |
| `--pad-where` | `front` | `front` | `front` | where the padding goes: `front`, before the chat template; `user`, at the start of the user turn; `between`, between the state and the question |

 

#### Tasks and abstention

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--tasks` | on | on | on | apply registered per-task calibration and intent heads (`POST /v1/tasks`) |
| `--tasks-file` | none | none | none | a JSON list of tasks to load at start, as `GET /v1/tasks?full=1` returns them |
| `--head-engine` | off | off | off | read the intent head's hidden state from a second copy of the model in vLLM's pooling mode instead of from the serving engine: faster head questions, at a second weight copy; not with `--image-model` |
| `--head-gpu-memory-utilization` | 0.47 | 0.47 | 0.47 | the second engine's share of the card |
| `--abstention` | on | on | on | apply registered abstention thresholds (`POST /v1/abstention/tasks`) |
| `--abstention-tasks` | none | none | none | a JSON list of abstention tasks to load at start |
| `--abstain-option` | none | none | none | offer this extra option (for example "can't tell") on requests that use imajev's extension, reported as `unknown_probability` and `abstained`; untrained |

 

#### Images

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--image-model` | none | none | none | the full multimodal checkpoint, as a second engine serving only requests that carry images; measured with the Qwen base only |
| `--image-gpu-memory-utilization` | 0.5 | 0.5 | 0.5 | the image engine's share of the card |
| `--one-engine` | off | off | off | load only the image engine and serve text requests on it too, for a card that cannot hold both; changes the text route's class |

 

#### The Mac (`--backend mlx`)

  

| Flag | Default | What it does |
| --- | --- | --- |
| `--prefix-cache-mb` | the base's: 2048 (Qwen and `gemma-4-31b`), 7400 (`gemma-4-12b`) | the cross-request prefix cache's budget in MiB (2^20 bytes); 0 turns it off |
| `--tokenizer` | [`Qwen/Qwen3.6-35B-A3B-FP8`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8) | the tokenizer the prompts are built with, so they are the vLLM path's byte for byte |

 

#### Verification

  

| Flag | Default | What it does |
| --- | --- | --- |
| `--debug-readout` | off | honour the `x-decisio-debug` request header (the raw readout in the response); never in production |

 

### What the behaviour flags do

 

#### Several questions in one request (`--multi-question`)

 

With `sequential`, the served default, a request with several questions first prefills the state once, then scores each question in its own engine call, reading the state from the prefix cache, so every answer equals the same question sent alone. With `warm`, for bulk scoring, the questions are scored in one batch after the same prefill: faster (four questions on a new state take about half the time of `sequential` on the Qwen base and about two thirds on the Gemma bases), but a batched answer differs from the same question sent alone, and when two options are close the choice can change ([`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) section 4). With the engine in its own process (`--engine-process separate`) a warm answer can also move between repeats, because that process does not always put a request's questions in one engine step (vllm-project/vllm#59764); with the engine in the server's process (`--engine-process in`) an identical request repeats exactly. `--engine-process` resolves to `in` for a single-engine server (since 0.8.0) and to `separate` when a second engine is configured (`--image-model`, `--head-engine`, `--one-engine`), with which `in` is refused; an explicit choice is honoured, and the server prints the arrangement and why at start-up and reports it in `/health` as `engine_process`. On the Gemma 4 31B base `warm` is the default when the engine runs in the server's process: against `sequential` it changed no choice on the suite, JevBench and the travel requests, by at most 0.0073 in a probability ([`runs/2026-10-05_engine-death-gates/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-05_engine-death-gates)); the other bases keep `sequential`. With `batch`, every question goes in one engine call that prefills the state itself, with no warm-up. Measurements: [`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) section 4, [`runs/2026-10-02_multi-question-and-rendering/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-02_multi-question-and-rendering), and the README's example request in [`runs/2026-10-01_docker-first-gpu-start/repeat_variability/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-01_docker-first-gpu-start/repeat_variability).

 

#### Row padding (`--pad-policy row`)

 

By default every state is padded to end on the KV cache block, so a single question on a state the server has not seen is prefilled in two engine steps, the state and then the question. With `row`, a single-question request is padded so that its whole row, state and question, ends on the block boundary, and vLLM prefills it in one engine step; requests with several questions are padded as by default. It is faster on a new state, and accuracy and calibration stayed within noise on the suite, JevBench, the four Decision Index benchmarks and the intent heads. Its cost is repeatability: a repeated identical request can move slightly on long prompts, where the default padding returns the same probabilities every time, so it is off by default. The demos ran with it ([`docs/demos/README.md`](https://opendecisio.com/docs/demos/)). Measurements: [`runs/2026-10-02_pad-policy-row/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-02_pad-policy-row); not measured on the Gemma base.

 

#### Determinism

 

On a running server with the Qwen base, a question returns the same probabilities every time, alone or among other questions: each question is scored in its own engine call. Across server restarts, single-question answers have matched an earlier record on the whole suite in most starts measured, and moved slightly without changing a choice in the others. The Gemma base does not repeat bit for bit: its logits come out in bf16 after its soft cap, and a state read for the first time and read again from the prefix cache can land a step apart; several questions in one request stay within the bound in the README's settings table. Any change of rounding moves this model's answers, the batch, the padding or where a long row is split, so every bit-identity claim holds for a fixed configuration and request form. Measurements: [`EVAL_CARD.md`](https://opendecisio.com/docs/evaluation-card/) sections 4 and 6.5.

 

#### Committed yes/no answers (`--noul-commit`)

 

For scorers that treat a yes/no probability strictly between 0.20 and 0.80 as no answer, as JevBench v1.5 does, such an answer is reported at the band's edge on its own side: 0.80 above 0.5, and 0.20 at or below it. The answer never changes; its probability does, and calibration pays for it: log loss and ECE rise, while JevBench v1.5's yes/no competence rises. Measurements: [`runs/2026-10-03_noul-commit/`](https://github.com/aminry/decisio/tree/v0.11.1/runs/2026-10-03_noul-commit) and [`docs/handoffs/tasks.md`](https://github.com/aminry/decisio/blob/v0.11.1/docs/handoffs/tasks.md).

 

#### The earlier layout (`--prompt-tail compact`)

 

The compact layout has no blank lines around the options and ends with "Answer with the letter only.". Tasks registered under one layout are not applied under the other, so a task registered before the spaced layout became the default needs `--prompt-tail compact` or registering again. Packed mode reads the compact layout only, and the image route keeps it.

 

## What next

 

- [Make it fast](https://opendecisio.com/docs/guides/latency/): which of these flags change latency, and by how much.
- [Choosing a base](https://opendecisio.com/docs/bases/): what `--base` picks.
- [Deploying and operating](https://opendecisio.com/docs/operating/): running a server with its flags.