Decisio
v0.13.0 Release notes

Typed decisions, on your own hardware

Decisions you can trust, on your own hardware. Open source.

Typed questions go in. A probability for every option comes out, in one pass.

  • 81 ms for one question about a new 300-token text
  • 1.08 s for 50 questions about a 3,000-token document
  • 32,000 tokens in one document
ollama pull aminroudaki/decisio-gemma
Get startedSee the proofGitHub
recordedGemma 4 31B, decisio 0.9.0, RTX PRO 6000 at 600 W
Support ticketModerationForm checkTool decision

Incoming support ticket · record 1 of 60

I was charged twice for October. Both charges are 49.00 EUR, same day. Please refund one.

reads it oncethen scores every question in one batch

urgentyes/no

Does this ticket need a response within the hour?

  • yes0.16

queuechoicebilling

Which team should handle this ticket?

  • billing0.96
  • bug0.02
  • access0.01
  • +3 more

impactscorelevel 0.9

Rate the business impact using only the reported facts.

  • 00.26
  • 10.65
  • 20.05
  • 30.04

3 answers in one batch · 91.2 ms server time

Teach it

Teach it your labels. It tells you which answers to trust.

Show the server a few hundred of your own labelled examples. It learns what your labels mean, then marks every answer it is sure of. The model's weights never change.

  1. 1 Learn your labels
  2. 2 Draw the line
  3. 3 Route the answers
Teaching a task, then certifying its answers Labelled examples of BANKING77 go into Decisio; the answer sharpens and accuracy goes from 78.7% to 84.4% on the Gemma 4 31B. Then 200 answers of support tickets across three queues, on the Gemma 4 12B on a Mac, are ordered by confidence; a threshold line moves down until 169 are above it. Those are certified and go to automation, 19 of them wrong; the other 31 go to a person. 1 Your labelled examples label card_arrivallabel exchange_ratelabel lost_or_stolen_cardlabel pending_card_paymentlabel cash_withdrawal_chargelabel declined_transferlabel card_linking Decisiolearns your labelsmodel unchanged An answer, before and after Accuracy, BANKING77 before78.7% after teaching84.4% 770 examples (10 per label), taught in 308 s 2 200 answersmost confident firstSupport tickets across three queues,Gemma 4 12B on a Mac. a right answera wrong one threshold 0.986: 169 above it 3 Certified: automated Not certified: a person CERTIFIED Teaching a task, then certifying its answers Labelled examples of BANKING77 go into Decisio; the answer sharpens and accuracy goes from 78.7% to 84.4% on the Gemma 4 31B. Then 200 answers of support tickets across three queues, on the Gemma 4 12B on a Mac, are ordered by confidence; a threshold line moves down until 169 are above it. Those are certified and go to automation, 19 of them wrong; the other 31 go to a person. 1 Your labelled examples label card_arrivallabel exchange_ratelabel lost_or_stolen_cardlabel pending_card_paymentlabel cash_withdrawal_chargelabel declined_transfer Decisiolearns your labelsmodel unchanged An answer, before and after Accuracy, BANKING77 before78.7% after teaching84.4% 770 examples (10 per label), taught in 308 s 2 200 answersmost confident firstSupport tickets across three queues, Gemma 4 12B on a Mac. rightwrong threshold 0.986: 169 above it 3 Certified: automated Not certified: a person CERTIFIED
Accuracy, BANKING77 (77 labels)
78.7% to 84.4%
Teach time
5 min for 770 examples (308 s)
Certified, at most 15% wrong (90% confidence)
84.5% 169 of 200 answers
Wrong among the certified
11.2% 19 of 169; the other 31 went to a person

Two measured tasks. Learning: BANKING77 on the Gemma 4 31B, after 10 labelled examples per intent, mean of 3 draws, 150 test items. Certifying: support tickets across three queues on the Gemma 4 12B on a Mac, 400 labelled examples, half held back; the dots are its 200 answers on further tickets, one draw. decisio's own records at the pinned release: the Gemma 4 31B's intent heads (EVAL_CARD.md section 7.2) and the certified worked example (docs/tasks.md, runs/2026-10-14_certified-answers/).

10 per intent
  • BANKING77, 77 intents84.4%
  • CLINC150, 150 intents97.0%

Accuracy on the Gemma 4 31B, untaught and taught. Only measured amounts are shown: none, and 10 labelled examples per intent.

  1. Teach.

    Send labelled examples of your question. It takes minutes, and the weights stay as they are.

  2. Certify.

    The server holds half back and finds the confidence above which your wrong answers stay under a rate you pick.

  3. Act.

    Every answer says certified or not. Automate the certified ones and send the rest to a person.

It needs labelled examples, and a certificate holds for one task on one serving setup.

Intent accuracy on the Gemma 4 31B. BANKING77: 78.7% plain, 84.4% after 10 labelled examples per intent (mean of 3 draws, 150 test items). CLINC150: 94.0% plain, 97.0% after 10 labelled examples per intent (mean of 3 draws, 100 test items). These use labelled examples, so they compare the taught server with the untaught one, not with systems that see no labels.

Teach your first task

On the board

What the public board says

  • 71.69JevBench v1.6.1 composite (95% interval 69.86 to 72.81); Jev 1.13.0 71.49 (69.14 to 72.40)
  • 70.4intelligence on JevBench v1.6.1; Jev 1.13.0 63.6
  • 73.3on the sealed questions, never published, on JevBench v1.6.1; Jev 1.13.0 61.6

JevBench v1.6.1, the board's file of October 2026, decisio v0.8.0 rows. The live board has today's standings.

See the board

On JevBench v1.6.1 (the board's file, October 2026), Decisio 31B scored 71.69 on the composite (95% interval 69.86 to 72.81) and Jev 1.13.0 scored 71.49 (69.14 to 72.40); the two intervals overlap. Honest limits

JevBench v1.6.1 composite scores with 95% intervals: ten open-weights systems and Jev 1.13.0Dot and interval chart of JevBench v1.6.1 composite scores with 95% intervals: ten open-weights systems from the board's file and Jev 1.13.0 as a reference row, in alphabetical order; every row is in the table on this page and on the benchmarks page.64666870727476Cygnet68.55 [64.42, 69.94]decider-12b v170.42 [68.09, 71.63]decider-12b v270.88 [68.54, 72.21]decisio v0.8.0, Gemma 4 12B (ours)68.17 [57.37, 69.65]decisio v0.8.0, Gemma 4 31B (ours)71.69 [69.86, 72.81]deck-31B68.68 [67.07, 69.74]H2O-Lightning-4B v1.172.52 [69.79, 73.79]Quyet-1.0-Large71.39 [69.84, 72.21]torchcast-decision-12b69.91 [67.22, 71.47]Winnow-12B Q868.89 [66.45, 70.46]Jev 1.13.071.49 [69.14, 72.40]JevBench score, with its 95% interval
JevBench v1.6.1: ten open-weights systems from the board's file, in alphabetical order, with Jev 1.13.0 as a reference row
SystemScore95% interval
Cygnet68.5564.42 to 69.94
decider-12b v170.4268.09 to 71.63
decider-12b v270.8868.54 to 72.21
decisio v0.8.0, Gemma 4 12B (ours)68.1757.37 to 69.65
decisio v0.8.0, Gemma 4 31B (ours)71.6969.86 to 72.81
deck-31B68.6867.07 to 69.74
H2O-Lightning-4B v1.172.5269.79 to 73.79
Quyet-1.0-Large71.3969.84 to 72.21
torchcast-decision-12b69.9167.22 to 71.47
Winnow-12B Q868.8966.45 to 70.46
Jev 1.13.071.4969.14 to 72.40
The JevBench board: its file for revision v1.6.1, sha256 5cd8c1332226.

How it works

Read once. Answer fast.

No training: one pass for each question.

One request carries a state and its questions.

The requestYour app sends one request with a state and its questions. Decisio reads the state once and, on the default Gemma 4 31B, scores the questions together in one batch; the response is a probability for every option, and the bars are a recorded answer.Your appa state, questionsPOST /v1/systemoneone requestDecisioone read, one batcha probability for every optionanswer0.96search0.01calculate0.02ask_user0.00act0.01

The state is read once, and the questions are scored together in one batch. A later request on the same state starts from the cache.

The cacheOn one 3,000-token document, Decisio 31B answers one question in 490 milliseconds, reading included, and fifty questions in one request in 1.08 s: it reads the document once and scores the questions together in one batch. A later, separate request on the same document starts from the cache and takes 32 to 43 ms (decisio 0.8.1, one RTX PRO 6000 at 585 W). The bars are to scale.One 3,000-token document, Decisio 31B, measured totals to scale (one box, 2026-10-11)1 question: 490 ms, read included50 questions: 1.08 sone read, one batcha later request on the same document, from the cache: 32 to 43 ms (decisio 0.8.1, one RTX PRO 6000 at 585 W)time

Keep the answers above a threshold you pick. Send the rest to a person or a larger model.

The thresholdA bar of the 1,172 questions of example 5: the ones kept at or above the threshold, and the ones escalated to a person or a larger model. The threshold moves through the measured values.Keep answers at or above0.827 of 1,172 questions escalatedThe rest are kept. Drawn to scale.

Speed, measured on one box

Many questions about one document

Fifty questions about one document, on one box.

  • about 44 timesas long for Jeff-1.0-Large (47.1 s) and Quyet-1.0-Large (48.9 s) as for Decisio 31B (1.08 s): 50 questions about a 3,000-token document, one box, 2026-10-11
  • about 66 timesas long at 6,000 tokens: 114.6 s and 114.8 s against 1.73 s, same box, 2026-10-11
  • 10.1 sfor Decisio 31B to answer 50 questions about a 32,000-token document, which neither Jeff nor Quyet accepts
Bar chart of the median time to answer 50 questions about one document, five systems measured in one session on one box. For a 3,000-token document: decisio Gemma 4 31B 1.08 seconds, decisio Gemma 4 12B 1.57, H2O-Lightning-4B 2.05, Jeff-1.0-Large 47.1, Quyet-1.0-Large 48.9. For a 6,000-token document: 1.73, 1.97, 2.82, 114.6 and 114.8 seconds. For a 32,000-token document only decisio Gemma 4 12B (6.76 s), H2O-Lightning-4B (9.05 s) and decisio Gemma 4 31B (10.1 s) answer; Jeff-1.0-Large and Quyet-1.0-Large are marked not supported.Bar chart of the median time to answer 50 questions about one document, five systems measured in one session on one box. For a 3,000-token document: decisio Gemma 4 31B 1.08 seconds, decisio Gemma 4 12B 1.57, H2O-Lightning-4B 2.05, Jeff-1.0-Large 47.1, Quyet-1.0-Large 48.9. For a 6,000-token document: 1.73, 1.97, 2.82, 114.6 and 114.8 seconds. For a 32,000-token document only decisio Gemma 4 12B (6.76 s), H2O-Lightning-4B (9.05 s) and decisio Gemma 4 31B (10.1 s) answer; Jeff-1.0-Large and Quyet-1.0-Large are marked not supported.
The exact wording, from the facts

On one box (one RTX PRO 6000 at 600 W, one session on 2026-10-11), Decisio 31B answered 50 questions about a 3,000-token document in 1.08 s and about a 6,000-token document in 1.73 s; Jeff-1.0-Large took 47.1 s and 114.6 s, and Quyet-1.0-Large 48.9 s and 114.8 s; neither accepts a 32,000-token document (Decisio 31B: 10.1 s). That is about 44 times (3,000 tokens) and 66 times (6,000 tokens) as long for Jeff-1.0-Large and Quyet-1.0-Large as for Decisio 31B.

On one box, the first question about a new 3,000-token document takes a median of 112 ms on H2O-Lightning-4B, 295 ms on Decisio 12B, 490 ms on Decisio 31B, and 895 ms on Jeff-1.0-Large and 895 ms on Quyet-1.0-Large.

The board, column by column

JevBench, axis by axis

Ten open-weights systems from the board's file in alphabetical order, and Jev 1.13.0 as a reference row; each cell is shaded by its value in the column. Higher is better on every axis.

Table of JevBench v1.6.1 scores for 11 rows: ten open-weights systems from the board's file and Jev 1.13.0 as a reference row, in alphabetical order, with six columns (overall, intelligence, sealed questions, calibration, speed, cost). Each cell is a plain value; the table has no shading or marks. Decisio 31B v0.8.0's values are overall 71.69, intelligence 70.4, sealed questions 73.3, calibration 88.7, speed 91.2, cost 51.7. The Decisio rows are the board's v0.8.0 entries: Decisio 31B v0.8.0 (71.69) and Decisio 12B v0.8.0 (68.17). The board's decisio v0.9.0 12B entry (67.76) is not among the rows shown.Table of JevBench v1.6.1 scores for 11 rows: ten open-weights systems from the board's file and Jev 1.13.0 as a reference row, in alphabetical order, with six columns (overall, intelligence, sealed questions, calibration, speed, cost). Each cell is a plain value; the table has no shading or marks. Decisio 31B v0.8.0's values are overall 71.69, intelligence 70.4, sealed questions 73.3, calibration 88.7, speed 91.2, cost 51.7. The Decisio rows are the board's v0.8.0 entries: Decisio 31B v0.8.0 (71.69) and Decisio 12B v0.8.0 (68.17). The board's decisio v0.9.0 12B entry (67.76) is not among the rows shown.
  • On JevBench v1.6.1, the 31B's sealed-set intelligence was 73.3 and Jev 1.13.0's 61.6; the open-set intelligence was 67.6 and 65.6.
  • The sealed set was never published.
  • On JevBench v1.6.1, Decisio 31B's axes were intelligence 70.4 (Jev 1.13.0 63.6), calibration 88.7 (90.6), speed 91.2 (91.5) and cost 51.7 (54.7).
  • On your own hardware you pay for the card, not per token.
Every number
JevBench: ten open-weights systems and Jev 1.13.0 on the composite, the four axes and the sealed questions, with each value in its column
SystemOverallIntelligenceSealed questionsCalibrationSpeedCost
Cygnet68.5554.851.387.091.856.4
decider-12b v170.4260.659.583.890.557.8
decider-12b v270.8863.063.682.090.557.8
Decisio 12B v0.8.0 (ours)68.1752.351.689.287.059.3
Decisio 31B v0.8.0 (ours)71.6970.473.388.791.251.7
deck-31B68.6873.075.882.286.749.7
H2O-Lightning-4B72.5260.059.090.092.660.3
Quyet-1.0-Large71.3973.471.690.086.950.5
torchcast-decision-12b69.9160.561.182.991.756.4
Winnow-12B Q868.8959.559.283.086.756.6
Jev 1.13.0 (reference)71.4963.661.690.691.554.7

Source: the board's file for revision v1.6.1.

Routing

Use the probability

Keep the answers above a threshold you pick and send the rest to a person or a larger model. Example 5 measured what that buys.

Every threshold, as measured
RuleEscalatedRight on the rest95% intervalConfidently wrong (wrong, kept)Wrong answers escalated
No escalation0 (0.0%)97.4%not applicable310 of 31
Keep at 0.8 or more27 (2.3%)98.5%97.6% to 99.1%1714 of 31
Keep at 0.9 or more47 (4.0%)98.8%98.0% to 99.3%1318 of 31
Line chart of accuracy on the questions kept against the share of questions escalated, for thresholds from 0.5 to 0.95 on Gemma 4 31B, with the 0.8 and 0.9 points marked. Example 5, Escalate only when unsure: ARC-Challenge test split, a public benchmark the model may have seen; decisio 0.9.0 on Gemma 4 31B, one NVIDIA RTX PRO 6000 Blackwell at 600 W, recorded 2026-10-08.Line chart of accuracy on the questions kept against the share of questions escalated, for thresholds from 0.5 to 0.95 on Gemma 4 31B, with the 0.8 and 0.9 points marked. Example 5, Escalate only when unsure: ARC-Challenge test split, a public benchmark the model may have seen; decisio 0.9.0 on Gemma 4 31B, one NVIDIA RTX PRO 6000 Blackwell at 600 W, recorded 2026-10-08.

Why

Why Decisio

  • Many questions, one read

    Every question in a request is scored from one read of your text: the same choice as one question at a time on every item measured, probabilities within 0.0073.

  • Long prompts, many options

    Prompts up to 32,768 tokens and 255 options per question.

  • Your hardware, no per-token bill

    It runs on your own GPU, Mac or laptop; the cost is the card, not a bill per token.

Demos

Watch it decide

Decisions in real time, from Decisio's own demos and one recorded run. Each clip is cut at the speed it happened. Nothing is sped up.

The driving clip draws map data from OpenStreetMap (ODbL, © OpenStreetMap contributors).

Get started

Four ways in, one command each.

# the Gemma 4 12B base, for 16 GB laptopsollama pull aminroudaki/decisio-gemma# the Qwen baseollama pull aminroudaki/decisio

The Ollama guide, and every path.

git clone https://github.com/aminry/decisiocd decisiouv sync --extra serve --frozenuv run python -m decisio.serve.vllm_engine

The GPU guide, and every path.

uv build --wheel  # the image installs this wheeldocker compose up --build  # needs the NVIDIA Container Toolkitcurl http://127.0.0.1:8000/health  # answers once the first start has fetched the checkpoint

The Docker guide, and every path.

uv sync --extra mlxuv run python -m decisio.serve.vllm_engine --backend mlx --model mlx-community/Qwen3.6-35B-A3B-6bituv run python -m decisio.serve.vllm_engine --backend mlx --base gemma-4-12b --model mlx-community/gemma-4-12B-it-6bituv run python -m decisio.serve.vllm_engine --backend mlx --base gemma-4-31b --model mlx-community/gemma-4-31b-it-6bit

The MLX guide, and every path.

Read this

Honest limits

  • One box, one sessionEvery race figure is from one RTX PRO 6000 Blackwell Workstation Edition, power limit 600 W (default, enforced), each system on its own server.
  • One questionThe one-question figure in the hero is server time, on decisio 0.8.1's served defaults and one RTX PRO 6000 at 585 W, so a card at 600 W may read new states faster.
  • Time, not accuracyThe race compares how long fifty questions take, on synthetic support tickets.
  • Replayed, not liveThe race replays medians of 5 or 10 documents, and the demo replays decisions recorded on decisio 0.9.0.
  • A snapshot of the boardThe values are for JevBench v1.6.1; the live board has moved since.
  • HardwareGemma 4 31B needs a 96 GB card, or a 64 GB Mac.

Ask it something real.

ollama pull aminroudaki/decisio-gemma
Read the quickstartStar it on GitHub