On a Mac or a laptop

Run decisio on a Mac with Apple silicon from an MLX build, sized to your Mac's memory, or on any 16 GB laptop with Ollama.

On a 64 GB Mac, install it and start the default model, Gemma 4 31B.

git clone --branch v0.18.1 https://github.com/aminry/decisio
cd decisio
uv sync --extra mlx

Shown, not run: it needs a Mac with Apple silicon, which this check does not have.

uv run python -m decisio.serve.vllm_engine --backend mlx --base gemma-4-31b --model mlx-community/gemma-4-31b-it-6bit

Needs a Mac with Apple silicon: taken as it is from decisio's checked docs, run record.

Figure · Memory on a Mac

Each build against the Mac it is for, at its peak on the longest text it serves; only the 64 GB Mac was measured.

The accent part is the weights, the grey part the rest of the peak while reading a long text, and the track the Mac's whole memory. Measured on one Apple M5 Pro with 64 GB.

Source: docs/running.md of the pinned release.

1. Pick the build for your Mac

Your Mac	Model and build	Memory: weights, then peak	Texts up to	One question on a new text
64 GB	Gemma 4 31B, 6-bit	24.9 GB; 29.6, 31.1 and 33.7 GB at about 8k, 16k and 32k tokens	16,383 tokens	not measured
48 GB	Gemma 4 31B, 4-bit	19.2 GB; 23.9, 25.3 and 28.0 GB, read on a 64 GB Mac	16,383 tokens	9.4 s at 3,000 tokens, on a busy Mac
32 GB	Gemma 4 12B, 6-bit	9.7 GB; 15.1 GB at 32,687 tokens	32,768 tokens	1,312 ms at 1,000 tokens
A 16 GB laptop	Gemma 4 12B on Ollama	a 7.7 GB download; 8.6 GB in Ollama, read on a 64 GB Mac	8,192 tokens	not recorded

Measured on one Apple M5 Pro with 64 GB; no smaller Mac was measured, and GB are 10⁹ bytes. The Qwen base runs from its 6-bit build too; its 4-bit build is for Macs with more than 32 GB.

2. Start it

On a 64 GB Mac, the command above starts the 31B. On a smaller Mac, start the build that fits it instead:

48 GB

--revision v0.15.0 serves the build that was measured, whatever its repository holds later.

uv run python -m decisio.serve.vllm_engine --backend mlx --base gemma-4-31b --model tachara-ai/decisio-gemma-4-31b-mlx-4bit --revision v0.15.0

Needs a Mac with Apple silicon: taken as it is from decisio's checked docs, run record.

32 GB

uv run python -m decisio.serve.vllm_engine --backend mlx --base gemma-4-12b --model mlx-community/gemma-4-12B-it-6bit

Needs a Mac with Apple silicon: taken as it is from decisio's checked docs, run record.

3. Mind the memory

The server sets MLX’s memory limits itself before it loads the model, and prints them in a start-up line that begins MLX MEMORY.
When the model cannot fit, it refuses to start, with the numbers, instead of freezing the Mac:

decisio: the gemma-4-12b base needs about 16.6 GiB under MLX (9.1 GiB of weights, a working set of 5.6 GiB at a 32k-token state, and the MLX cache limit of 2.0 GiB), which is more than the memory limit of 14.0 GiB (--mlx-memory-limit-gb 14). Raise the limit, lower --mlx-cache-limit-gb, or start a smaller base

Recorded output.

Run one model server on a Mac at a time: two servers can each stay under their own limit and still add up to more than the Mac has.
/health reports the memory under memory; watch physical_footprint_bytes, the figure Activity Monitor shows, not the resident size.

4. What a Mac does not serve

Images in a question, adapters (--adapter), packed mode, or the intent head on a second engine (--head-engine).
FP8 checkpoints: a Mac always names an MLX build with --model.
Teaching a task works, slowly: on the 31B 4-bit, about an hour for 770 labelled examples (77 options) and about 2.6 hours for 1,500 (150 options).

On a laptop, with Ollama

Ollama runs decisio’s Gemma 4 12B listing on Mac, Linux or Windows, and its smallest tag fits a 16 GB laptop. You need Ollama 0.35.1 or later.

# the Gemma 4 12B base, for 16 GB laptops
ollama pull aminroudaki/decisio-gemma
# the Qwen base
ollama pull aminroudaki/decisio

Run against Ollama 0.35.1 with aminroudaki/decisio-gemma and aminroudaki/decisio, the listing as published, on 2026-10-09; the answers are Ollama's, not the default model's on a decisio server. Its comments are shown above their commands.

Tag	Download	In Ollama
`q4_k_m`, also `latest`	7.7 GB	8.6 GB, at the listing’s 8,192-token context
`q8_0`	not given	13.6 GB

The Qwen listing’s smallest tag is a 23 GB download, so it needs a machine with more memory. Then ask it a question on Ollama’s decision route, on port 11434, with the listing named in model:

curl http://localhost:11434/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{
  "model": "aminroudaki/decisio-gemma",
  "state": "Hi, since this morning none of our 40 staff can log in to the dashboard. We get \"session expired\" right after entering the password. Payroll is due today.",
  "questions": {
    "urgent": {
      "type": "noul",
      "instructions": "Does this ticket need a response within the hour?"
    },
    "category": {
      "type": "choice",
      "instructions": "Which team should handle this ticket?",
      "criteria": {
        "billing": "Invoices, payments, refunds",
        "access": "Login, passwords, permissions",
        "bug": "The product behaves wrongly",
        "other": null
      }
    },
    "impact": {
      "type": "score",
      "instructions": "Rate the business impact using only the reported facts.",
      "criteria": [
        "No function impaired",
        "One user impaired, with a workaround",
        "Many users blocked from a core function",
        "Data loss or legal exposure"
      ]
    }
  }
}'

Run against Ollama 0.35.1 with aminroudaki/decisio-gemma, the listing as published, on 2026-10-09; the answers are Ollama's, not the default model's on a decisio server.

Ollama builds its own prompt and applies none of decisio’s corrections to the probabilities, so they are not decisio’s measured ones. To teach and certify tasks, run a decisio server.

What next

Get started: send your first question and read the answer.
Requirements and limits: every machine, the context length and what was measured on what.
Teach a task: teach a question you ask again and again, and certify its answers.