Skip to content
Decisio

DocsQuestions and answers

2 min read · updated

Use the probabilities

Read an answer's probabilities, choose a review policy and evaluate it on labelled examples. Distinguish a manual threshold from a certified answer.

Use the probabilities to decide when your application should act and when it should ask for review. Move the manual threshold to see which illustrative answers your policy would accept:

Figure · Act or review

Choose a probability line; see which example answers your policy accepts.

2 act, 2 review at a manual threshold of 0.90.

  • Ticket Abilling, probability 0.95
    Act
  • Ticket Baccess, probability 0.90
    Act
  • Ticket Cbilling, probability 0.65
    Review
  • Ticket Dbug, probability 0.40
    Review

This is a manual policy. To automate at an error target, certify a task and check its certificate.

Illustration: chosen probabilities for four example answers. The control changes the application policy, not the model.

Read the value for your question type

Section titled “Read the value for your question type”
Question typeReadMeaning
Choiceanswer.probabilities[answer.choice]Probability of the selected option
Yes or noanswer.noulProbability of yes; use 1 - answer.noul for no
Scoreanswer.probabilitiesDistribution across the levels; answer.score is their weighted mean

For a choice, the top probability is the largest value in its distribution. For yes or no, the likelier side has probability max(answer.noul, 1 - answer.noul). For a score, read the distribution as well as the mean: a score of 1.7 is a position on the rubric, not a probability.

Pick an application policy

Section titled “Pick an application policy”

For example, route a ticket when its selected queue has probability at least 0.9, and send the rest to review. This is an illustrative threshold; pick a line using labelled examples from your own traffic and the cost of a wrong route. A higher line accepts fewer answers, and its error rate needs to be measured.

ResultApplication action
Selected queue meets your policyRoute to that queue
Answer falls below your policySend to review
Request fails or times outUse your service-failure fallback

Keep the question, options and action policy together in a module your team can review. A high probability for one answer does not guarantee that it is correct.

Check the policy on labelled examples

Section titled “Check the policy on labelled examples”

Use examples that include ordinary cases, ambiguous cases and inputs outside the categories you intended. Choose the question and policy on development examples, then measure on a separate held-out set. Count wrong automated decisions and how many cases went to review; repeat when the question, model or traffic changes.

Evaluate your task gives a practical checklist. For a complete implementation, follow support routing.

Use certification when you need an error target

Section titled “Use certification when you need an error target”

A threshold chosen by hand is a policy, not a certificate. To automate at an error target, teach and certify a task using representative labelled examples. Your code checks the returned certified field before using its certified-answer path; an answer without it does not qualify.

Certification applies to choice and yes/no tasks, not scores. A gold certificate concerns errors against labelled answers; a learned table’s agreement certificate concerns agreement with the model. If your policy requires a gold certificate, check certificate_kind as well as certified.

Explore a recorded request and answer

Figure · Recorded request and answer

Every question gets its own typed answer.

Request

{
  "state": "I was charged twice for October. Both charges are 49.00 EUR, same day. Please refund one.",
  "questions": {
    "urgent": {
      "type": "noul",
      "instructions": "Does this ticket need a response within the hour?"
    },
    "queue": {
      "type": "choice",
      "instructions": "Which team should handle this ticket?",
      "criteria": {
        "billing": "Invoices, payments, refunds, prices, VAT",
        "access": "Login, passwords, two-factor, SSO, permissions, invitations, account security",
        "bug": "The product behaves wrongly: errors, crashes, wrong numbers, lost data",
        "feature": "A request for something the product does not do yet",
        "cancellation": "Cancelling, pausing or closing an account, deleting data",
        "other": "Press, partners, jobs, sales pitches, wrong address, thanks, general questions"
      }
    },
    "impact": {
      "type": "score",
      "instructions": "Rate the business impact using only the reported facts.",
      "criteria": [
        "No function impaired",
        "One user impaired, with a workaround",
        "Many users blocked from a core function",
        "Data loss or legal exposure"
      ]
    }
  }
}

Answer fields

{
  "answers": {
    "urgent": {
      "type": "noul",
      "noul": 0.159
    },
    "queue": {
      "type": "choice",
      "choice": "billing",
      "probabilities": {
        "billing": 0.962,
        "access": 0.011,
        "bug": 0.016,
        "feature": 0.004,
        "cancellation": 0.005,
        "other": 0.003
      }
    },
    "impact": {
      "type": "score",
      "probabilities": {
        "0": 0.256,
        "1": 0.648,
        "2": 0.053,
        "3": 0.042
      }
    }
  }
}

The text, question, options and typed answer are separate parts of one decision.

Read all field explanations
Text
state is the text to ask about. Keep it separate from the question.
Question
questions names each judgment. Its type and instructions tell the server what to answer.
Options
criteria gives named options for choice or ordered levels for score. A noul question already has yes and no, and can optionally describe true and false.
Answer
answers uses your question names. Read the typed value and its probabilities before your application acts.

Recorded decision: Gemma 4 31B, decisio 0.17.0, RTX PRO 6000 at 600 W, 2026-10-19, on the plain layout; record 1 of 60 of decisio-examples' support-routing run, sent again on that release; the response is abridged and its probabilities are rounded to three places.

Good to know

Section titled “Good to know”
  • Choose thresholds from evaluation on your task, rather than assuming every question type or model needs the same line.
  • Service errors and uncertain answers need separate fallback handling in your backend.
  • Changing the question or serving setup means evaluating again; certify again when required by the task guide.

What next

Section titled “What next”