# How it works

> A question becomes one prompt and is answered in one forward pass, read at one position; the state is read once and shared.

Source: https://opendecisio.com/docs/how-it-works/

Most language-model APIs write text and make you parse it. Decisio asks the model one question at a time and reads its answer straight off the model’s output, so the answer is a probability for every option, and nothing is generated.

 

[Figure: A question is one prompt; its answer is read at one position, with no text generated. One support-ticket question with 6 lettered options becomes one prompt. The model's distribution over the letters at the answer position is the answer: A billing 0.96, B access 0.01, C bug 0.02, D feature 0.00, E cancellation 0.01, F other 0.00.]

 

## TL;DR

 

With a server running ([Get started](https://opendecisio.com/docs/) starts one), one yes/no question is one prompt and one pass:

 

```
curl http://127.0.0.1:8000/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"state": "I was charged twice for October. Please refund one.", "questions": {"urgent": {"type": "noul"}}}'
# {"model": "...", "answers": {"urgent": {"type": "noul", "noul": ...}}, "usage": {"input_tokens": ..., "output_tokens": ...}}
```

 

Run against the CPU stand-in at v0.11.1 on 2026-10-14.

 

`usage.output_tokens` is one per question: the “token” is the letter that was read.

  

| Word | Meaning |
| --- | --- |
| forward pass | one run of the model over a prompt, which is all a question costs |
| readout | the model’s distribution over the option letters at the answer’s position |
| prefix cache | where the state’s read is kept, so every question about it starts from there |
| hidden state | the model’s internal representation at the answer position, which an intent head reads |

 

## 1. The mechanism

 

Each question becomes one prompt, the state followed by the question and its lettered options, and its answer is the model's distribution over the option letters at one position, so every question costs one forward pass and no generated text. The state is prefilled once per request and every question reads it from vLLM's prefix cache. No training: every base is an official checkpoint, frozen, with no fine-tuning and no adapter. decisio's vLLM plugin registers its model classes through an entry point, so vLLM loads them without a patch, and the served class also returns the hidden state at the answer position for the intent head. The design notes: [`docs/design/vllm-plugin.md`](https://github.com/aminry/decisio/blob/v0.11.1/docs/design/vllm-plugin.md), [`docs/design/hidden-state-readout.md`](https://github.com/aminry/decisio/blob/v0.11.1/docs/design/hidden-state-readout.md) and [`docs/design/mlx-backend.md`](https://github.com/aminry/decisio/blob/v0.11.1/docs/design/mlx-backend.md).

 

## 2. A request

 

One request carries a state, such as a support ticket, and the questions to ask about it. The server answers each question with one forward pass and writes no text.

  

## 3. The cache

 

The state is read once. Every later question about it starts from the cache, which is why asking many questions about one document is cheap.

  

## 4. The threshold

 

Every answer carries a probability for each option. Keep the answers above a threshold you pick and send the rest to a person or a larger model. The tradeoff is yours to set from your own labelled items: how well the number separates right from wrong depends on your questions.

  

## 5. The bases

 

One flag picks the model. The routes and the calibration are the same for each.

  

## What next

 

- [Choosing a base](https://opendecisio.com/docs/bases/): which model to serve, and why.
- [What the probability means](https://opendecisio.com/docs/probability/): temperatures, calibration, and what was measured.
- [Many questions about one text](https://opendecisio.com/docs/guides/many-questions/): one read, then every question.