Skip to content
Decisio

DocsHow-to guides

3 min read · updated

Serve your own adapter

Load a LoRA adapter you trained on the base, give it its own temperatures, check that the server serves it, and know what it does to certificates.

Serve a LoRA adapter you trained on the base, and ask each question with it or without it. An adapter changes the scale of the model’s readout, so it carries its own temperatures: one per question type, and for choice questions one per band of option count. This writes decisio’s example file and has decisio check it, on any machine:

cat > decisio_adapter.json <<'EOF'
{"schema": "decisio.adapter/1",
 "temperatures": {"choice": 2.141, "noul": 3.075, "score": 4.185},
 "choice_bands": {"2-10": 1.9, "11-26": 2.6, "27-255": 3.1}}
EOF
uv run python -m decisio.serve.adapter_temperatures check decisio_adapter.json

Run against the CPU stand-in at v0.18.1 on 2026-10-19. It printed:

{
 "temperatures": {
  "choice": 2.141,
  "noul": 3.075,
  "score": 4.185
 },
 "choice_bands": {
  "2-10": 1.9,
  "11-26": 2.6,
  "27-255": 3.1
 },
 "fit": null,
 "file": "decisio_adapter.json"
}

1. Load the adapter

Section titled “1. Load the adapter”

Start the server with --adapter name=path, once per adapter, and the adapter’s temperature file:

pip install "decisio[serve]==0.18.1"
python -m decisio.serve.vllm_engine --adapter my-adapter=/models/my-adapter \
  --adapter-temperatures my-adapter=decisio_adapter.json

Shown, not run: it needs Linux with one NVIDIA card and a LoRA adapter, which this check does not have.

  • The adapter is a vLLM-format LoRA adapter, served unmerged, on the text route, in separate mode.
  • A Mac does not serve adapters.
  • A request uses the adapter only when it names it; every other request is answered by the base.

2. Give it its own temperatures

Section titled “2. Give it its own temperatures”
  • Put the file beside the adapter’s weights as decisio_adapter.json, or give it with --adapter-temperatures name=file when the adapter’s folder is not yours to change.
  • temperatures has one per type: choice, noul for yes/no, and score. A type the file does not list keeps the base’s temperature.
  • choice_bands is optional: keys lo-hi, within 2 to 255 and not overlapping. A choice question is served at its band’s temperature; a count in no band is served at temperatures.choice, then at the base’s.
  • Yes/no and score questions are not banded.
  • An adapter with no file is served at the base’s temperatures, and the server says so when it starts.
  • The file is checked before the engine is built, and an unusable one stops the start.

Fit the file from the adapter’s readouts on labelled items, one row per item ({"type": ..., "probs": [...], "gold": <index>}):

python -m decisio.serve.adapter_temperatures fit --reads reads.jsonl \
  --out decisio_adapter.json \
  --data "what the items are"

Shown, not run: it needs your adapter's readouts on your labelled items.

Fit on items that are not a benchmark’s: a temperature holds for the kind of items it was fitted on.

3. Ask with it

Section titled “3. Ask with it”

Name the adapter in the request’s adapter field:

curl -s http://127.0.0.1:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "I was charged twice for the same order.",
  "adapter": "my-adapter",
  "questions": {"refund": {"type": "noul", "instructions": "Does the customer want a refund?"}}
}' -D - -o answer.json | grep -i '^x-decisio-adapter'

Shown, not run: it needs a server with an adapter loaded, on one NVIDIA card.

  • The answer carries "adapter" and the x-decisio-adapter header when the adapter answered.
  • An unknown name is answered 404, with the adapters that are loaded.

4. Check that it took

Section titled “4. Check that it took”
curl -s http://127.0.0.1:8000/health \
  | python3 -c 'import json, sys; h = json.load(sys.stdin); print(json.dumps({k: h.get(k) for k in ("adapter_temperatures", "adapter_identity")}, indent=1))'

Shown, not run: it needs a server with an adapter loaded, on one NVIDIA card.

/health shows each adapter’s temperatures in adapter_temperatures, and the hash of its weights and configuration in adapter_identity.

5. Know what it does to certificates

Section titled “5. Know what it does to certificates”

A certificate covers one serving setup, and an adapter is part of it. The setup a certificate is checked against holds each loaded adapter’s name, weights hash and temperatures, so a different adapter, a retrained one or a changed temperature is a different setup: certify your tasks again on it. A request that names an adapter is never certified, and a question that a registered task matches is refused with an adapter.

What next

Section titled “What next”