Send a fixed text once

Every request of a task is often read against the same long text: a policy, a rubric, a catalogue, a set of definitions. Register that text once as the task’s context, and each request sends only the part that changes; the server puts the context in front of it. On a Mac, a 1,176-token moderation policy sent as the context took 373 ms a read, against 1,946 ms when every request carried it:

Figure · How to use it

Save the fixed policy with the task; send only the new message.

1. Register the taskStore the fixed policy as its context, beside the question.
2. Send a messagePut only the changing message in state.
3. Answer with contextThe model reads the task context with this new message.
4. Use the resultYour backend reads the typed answer as usual.

Keep the context out of later messages so the policy is not included twice. Cache reuse depends on the serving engine.

Source: task context in the pinned task and running guides.

1. See where it works great

Engine	The policy at the start of every request	The policy as the task’s context
MLX, an Apple M5 Pro	1,946 ms a read	373 ms a read
vLLM, an RTX PRO 6000	61.4 ms p50	60.8 ms p50

decisio’s measurement with a 1,176-token moderation policy and short comments to judge, the tuned 12B, one question per request, requests one at a time: the MLX conversion on an Apple M5 Pro, the mean of 600 reads; the 12B in bf16 on vLLM 0.31.0 and one RTX PRO 6000, 40 requests, as decisio’s docs/tasks.md states them.

• On a Mac each request computes only its changing part. MLX’s prefix cache finds only exactly the same prefix, so a policy at the start of every request is computed again each time; as the context it is a cache entry of its own, and each request computes only its changing part.
• On vLLM the answers are the same, and the requests smaller. vLLM’s prefix cache already reuses a prefix every request repeats, so the two run at the same speed; the context is held from the task’s first request.
• A fast table learns from the part that changes, since it encodes the state alone.

2. Register it

Prepare a complete task.json registration with your real examples, as Teach a task shows. Save the fixed policy in context.txt; each example’s state contains only the changing input. Run this Python script from the directory containing both files, with DECISIO_BASE_URL set to your endpoint:

import json
import os
import urllib.request

with open("task.json", encoding="utf-8") as source:
    body = json.load(source)
with open("context.txt", encoding="utf-8") as context:
    body["context"] = context.read()

base_url = os.environ.get("DECISIO_BASE_URL", "http://127.0.0.1:8000").rstrip("/")
request = urllib.request.Request(
    base_url + "/v1/tasks", data=json.dumps(body).encode(),
    headers={"Content-Type": "application/json"},
)
with urllib.request.urlopen(request, timeout=300) as response:
    print(json.dumps(json.load(response), indent=2))

Shown, not run: it needs a complete labelled task.json and a context.txt policy supplied by the reader; transport tests check both files.

• Add "context" to the registration: a string, or an object or array read as compact JSON. Labelled, dynamic-option and learning registrations take it.
• Send only the changing part as each example’s state. The server reads each example with the context, so the task is fitted, certified and timed on exactly what it serves.
• Keep in the context only what is the same on every request. What changes belongs in the state.

3. Ask with it

• Send the state without the context. A state that still starts with the context is read with it twice, and the response carries x-decisio-warning.
• Name the task when several share a question. The same question with another context is another task, kept beside the first; "task": "<id>" on /v1/systemone picks one.
• One request, one context. Questions whose tasks have different contexts go in separate requests.

4. Good to know

• A context over the model’s window is refused with 422 before any model work, on every engine, and so is one that fits but not beside an example’s state.
• The context is part of the task’s identity: a certificate made under one context does not hold under another.
• Status shows the context’s sha256 and length, never its text; the export holds the text.
• Abstention tasks take no context.
• Upgrade every replica before registering a task with a context. An earlier release refuses its record rather than serve the task without it.

What next

• Teach a task: register a task and read what it kept.
• Add a decision fast path: a classifier on the server’s CPU, before the model.
• The HTTP API: the task routes, field by field.