# RAG: answer or abstain

> Before a retrieval pipeline answers, check whether the passages it found answer the question; answer when decisio is sure, and say so when they do not.

Source: https://opendecisio.com/docs/use-cases/rag-gate/

This script sends the [question](https://opendecisio.com/docs/glossary/#question) and the passage as one text and acts only on a [certified answer](https://opendecisio.com/docs/glossary/#certified-answer):

 

```
import json
import urllib.request

QUESTION = {
    "type": "choice",
    "instructions": "Does the passage answer the question?",
    "criteria": {
        "yes": "The passage answers the question",
        "partly": "It answers only part of it",
        "no": "It does not answer it"
    }
}

def route(text):
    request = urllib.request.Request(
        "http://127.0.0.1:8000/v1/systemone",
        data=json.dumps({"state": text, "questions": {"answered": QUESTION}}).encode(),
        headers={"Content-Type": "application/json"},
    )
    with urllib.request.urlopen(request) as response:
        answer = json.load(response)["answers"]["answered"]
    # A certified answer acts on its own; anything else goes to a person.
    if answer.get("certified"):
        return {"yes": "answer from the passage", "partly": "answer, and say what is missing", "no": "say it is not in the documents"}[answer["choice"]]
    return "to a larger model or a person"

print(route("Question: How long is the warranty?\nPassage: Every product comes with a two-year warranty from the date of purchase."))
```

 

Run against the CPU stand-in at v0.18.1 on 2026-10-19. It printed:

```
to a larger model or a person
```

 

Until the [task](https://opendecisio.com/docs/glossary/#task) is certified (step 2), every answer goes to a person, as on the stand-in.

 

Good fit

- A retrieval step that returns passages, and questions your users ask again and again.
- Labelled pairs of a question and a passage, to certify from.

Not a good fit

- Writing the answer itself: decisio reads and decides, it does not write.
- Questions that need reasoning across many documents at once.

 

## 1 Write the question

 

One `choice` question about the text: does the passage answer the question, partly, or not. Put the user’s question and the passage in the text, as the script does.

 

## 2 Teach it and certify it

 

Teach the question from labelled pairs, and certify it at your [error target](https://opendecisio.com/docs/glossary/#error-target).

 

Send at least 118 labelled question and passage pairs. The server holds back half to certify, and a 5% certificate needs 59 of those. More examples certify a larger share.

 

```
POST /v1/tasks
{"id": "rag-gate",
 "examples": [{"request": {"state": "<a past question and passage of yours>", "questions": {"answered": <the question above>}},
               "answer": "yes"},
              ...],
 "certify": {"alpha": 0.05}}
```

 

A shape to fill in, not a command to run.

 

[Certify answers](https://opendecisio.com/docs/tasks/certify/#2-teach-the-task-with-a-certificate) runs this call on a server, with its answer. To let the server decline on its own, add a “can’t tell” [option](https://opendecisio.com/docs/glossary/#option) and an [abstention](https://opendecisio.com/docs/glossary/#abstain) [threshold](https://opendecisio.com/docs/glossary/#threshold) ([Certify answers](https://opendecisio.com/docs/tasks/certify/#let-the-server-abstain)).

 

## 3 Route on the certificate

 

- A certified yes answers from the passage; a certified no says it is not in the documents.
- Everything else goes to a larger model or a person.
- When your documents or your users’ questions change, certification is suspended until you certify again ([Watch for drift](https://opendecisio.com/docs/tasks/drift/)).

 

## What it did

 

- **82.7%** right on SciFact, does the evidence support the claim, and **88.7%** on BoolQ, yes or no over a passage, on Gemma 4 31B, untaught, in decisio's 1,400-item suite: one RTX PRO 6000, vLLM 0.30.0, 2026-10-04 (its evaluation card, section 7.2).
- Those measure the reading, not a whole retrieval pipeline: no end-to-end run of this gate is published.

 

## What next

 

- [Certify answers](https://opendecisio.com/docs/tasks/certify/): the promise, and abstention.
- [Scale](https://opendecisio.com/docs/scale/): long passages, and many questions about one text.
- [Use cases](https://opendecisio.com/docs/use-cases/): the other recipes.