Skip to content
Decisio

DocsUse cases

2 min read · updated

RAG: answer or abstain

Before a retrieval pipeline answers, check whether the passages it found answer the question; answer when decisio is sure, and say so when they do not.

This script sends the question and the passage as one text and acts only on a certified answer:

import json
import urllib.request


QUESTION = {
    "type": "choice",
    "instructions": "Does the passage answer the question?",
    "criteria": {
        "yes": "The passage answers the question",
        "partly": "It answers only part of it",
        "no": "It does not answer it"
    }
}




def route(text):
    request = urllib.request.Request(
        "http://127.0.0.1:8000/v1/systemone",
        data=json.dumps({"state": text, "questions": {"answered": QUESTION}}).encode(),
        headers={"Content-Type": "application/json"},
    )
    with urllib.request.urlopen(request) as response:
        answer = json.load(response)["answers"]["answered"]
    # A certified answer acts on its own; anything else goes to a person.
    if answer.get("certified"):
        return {"yes": "answer from the passage", "partly": "answer, and say what is missing", "no": "say it is not in the documents"}[answer["choice"]]
    return "to a larger model or a person"




print(route("Question: How long is the warranty?\nPassage: Every product comes with a two-year warranty from the date of purchase."))

Run against the CPU stand-in at v0.18.1 on 2026-10-19. It printed:

to a larger model or a person

Until the task is certified (step 2), every answer goes to a person, as on the stand-in.

Good fit

  • A retrieval step that returns passages, and questions your users ask again and again.
  • Labelled pairs of a question and a passage, to certify from.

Not a good fit

  • Writing the answer itself: decisio reads and decides, it does not write.
  • Questions that need reasoning across many documents at once.

1. Write the question

Section titled “1. Write the question”

One choice question about the text: does the passage answer the question, partly, or not. Put the user’s question and the passage in the text, as the script does.

2. Teach it and certify it

Section titled “2. Teach it and certify it”

Teach the question from labelled pairs, and certify it at your error target.

Send at least 118 labelled question and passage pairs. The server holds back half to certify, and a 5% certificate needs 59 of those. More examples certify a larger share.

POST /v1/tasks
{"id": "rag-gate",
 "examples": [{"request": {"state": "<a past question and passage of yours>", "questions": {"answered": <the question above>}},
               "answer": "yes"},
              ...],
 "certify": {"alpha": 0.05}}

A shape to fill in, not a command to run.

Certify answers runs this call on a server, with its answer. To let the server decline on its own, add a “can’t tell” option and an abstention threshold (Certify answers).

3. Route on the certificate

Section titled “3. Route on the certificate”
  • A certified yes answers from the passage; a certified no says it is not in the documents.
  • Everything else goes to a larger model or a person.
  • When your documents or your users’ questions change, certification is suspended until you certify again (Watch for drift).

What it did

Section titled “What it did”
  • 82.7% right on SciFact, does the evidence support the claim, and 88.7% on BoolQ, yes or no over a passage, on Gemma 4 31B, untaught, in decisio's 1,400-item suite: one RTX PRO 6000, vLLM 0.30.0, 2026-10-04 (its evaluation card, section 7.2).
  • Those measure the reading, not a whole retrieval pipeline: no end-to-end run of this gate is published.

What next

Section titled “What next”