Is Ollama Actually Using MLX on Your Mac? GGUF vs Safetensors

Ollama has shipped an MLX engine for Apple Silicon since version 0.19, and since 0.40 it is supposed to be the default. Yet a Mac on the current release can still run every model it has on llama.cpp. The one I checked did, until I pulled a different tag. The explanations floating around are half right: "the model format decides the engine" and "MLX needs 32GB or more." Ollama has no documentation page for MLX (docs.ollama.com/mlx returned a 404 when I checked on 2026-10-22), so this post settles both claims from the source code, the release notes, and what a real Mac prints.

Everything below is pinned to Ollama 0.40.8 (released 2026-10-21) on an Apple M5 MacBook Pro with 32GB, macOS 26.7.1, checked on 2026-10-22. Ollama changes the engine rules from release to release, so re-check on yours.

The short answer

• To see which engine is running a model: load it, then run ollama ps. The RUNNER column says mlx or llamacpp.
• When MLX is used: only when the copy of the model on your disk is in safetensors format. A GGUF model always runs on llama.cpp. That part of the folk rule is true, and the one-line check behind it is identical at the 0.19.0 and 0.40.8 tags.
• What the folk rule leaves out: you rarely choose the format yourself. Since 0.40.0, many library tags (for example qwen3.5:4b and gemma4:12b) hold both a GGUF build and an MLX build. On Apple Silicon, ollama pull downloads the MLX build. Tags that name a GGUF quantization, like qwen3.5:4b-q4_K_M, contain only GGUF. Older families such as llama3.2, qwen3 and mistral have only GGUF builds, so they run on llama.cpp even on a Mac.
• "MLX needs 32GB": there is no fixed 32GB rule in the 0.40.8 source. There is a fit check instead: the MLX runner refuses to load a model whose weights exceed the free GPU memory. The 32GB sentence comes from the March 2026 preview announcement and was about the 35B model it launched with.
• Models you already have do not move to MLX. I found no code path in 0.40.8 that converts or re-downloads a GGUF model as MLX. To get MLX you pull the MLX build.

Step 1: check ollama ps while the model is loaded

ollama ps only lists models that are loaded right now, so send one prompt first. I pulled the same 4B model twice, once by its bare tag and once by its GGUF tag, and ran one prompt through each:

ollama run qwen3.5:4b --think=false 'Say hi in five words.' </dev/null
ollama ps

NAME          ID              SIZE      PROCESSOR    CONTEXT    RUNNER    UNTIL
qwen3.5:4b    7f3251fa6878    4.1 GB    100% GPU     65536      mlx       4 minutes from now

ollama run qwen3.5:4b-q4_K_M --think=false 'Say hi in five words.' </dev/null
ollama ps

NAME                 ID              SIZE      PROCESSOR    CONTEXT    RUNNER      UNTIL
qwen3.5:4b-q4_K_M    d8b0f5e9760c    5.8 GB    100% GPU     65536      llamacpp    4 minutes from now

Same model family, same machine, same Ollama, two different engines. The only difference is the tag I pulled.

Two notes on reading this output:

• 100% GPU does not mean MLX. llama.cpp also runs fully on the Apple GPU through Metal. Only the RUNNER column tells the two apart.
• Don't compare the SIZE column across engines. On my machine the context length is set to 65,536, and the llama.cpp server allocated its whole KV cache when the model loaded. The server log shows llama_kv_cache: MTL0 KV buffer size = 2048.00 MiB. The MLX runner's memory grew with use instead. Its log reported memory peak="3.85 GiB" after the short prompt. Over the series of benchmark prompts below, that figure climbed to 6.59 GiB, because the runner also keeps cache snapshots of earlier requests. The 1.7 GB gap in SIZE largely reflects when each engine allocates memory. It does not show that MLX needs less.

The </dev/null matters. When stdin is not a terminal, ollama run reads all of stdin before it sends anything, even when you pass the prompt as an argument (cmd/cmd.go). In a script, or when an agent runs the command, stdin may never close, and the command hangs with no output. My first scripted attempt hung exactly this way.

Step 2: check the format before you load anything

You can tell which engine a model will use without loading it. ollama show prints the quantization, and the API response says outright:

curl -s localhost:11434/api/show -d '{"model":"qwen3.5:4b"}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["details"])'

{'parent_model': '', 'format': 'safetensors', 'family': 'qwen3_5', 'families': None, 'parameter_size': '4.5B', 'quantization_level': 'nvfp4', 'runner': 'mlx'}

The GGUF tag returns 'format': 'gguf', 'quantization_level': 'Q4_K_M', and 'runner': 'llamacpp'. In plain ollama show output, the giveaways are quantization nvfp4 against Q4_K_M, and the architecture name: qwen3_5 (the Hugging Face spelling, used by the MLX build) against qwen35 (the GGUF spelling).

To list every local model at once:

curl -s localhost:11434/api/tags | python3 -c '
import json, sys
for m in json.load(sys.stdin)["models"]:
    d = m["details"]
    print(m["name"].ljust(58), d.get("format"), d.get("runner"))'

On this Mac, after the two test pulls. The bottom two models were already there, and both are GGUF:

qwen3.5:4b-q4_K_M                                          gguf llamacpp
qwen3.5:4b                                                 safetensors mlx
gemma4:12b-it-q4_K_M                                       gguf llamacpp
fredrezones55/Gemma-4-Uncensored-HauhauCS-Aggressive:e4b   gguf ggml

ggml is the older label for GGUF models created before 0.40. It is not a third engine. In 0.40.8 every GGUF model is served by llama.cpp's llama-server, and the scheduler records the runner as llamacpp once the model loads (server/sched.go: "The loaded process is authoritative").

Step 3: confirm in the server log

The server log on macOS is ~/.ollama/logs/server.log. Each engine leaves a different trail when a model loads. GGUF:

level=INFO source=server.go:107 msg="using llama-server for model" model=/Users/.../blobs/sha256-a9d0a3e1...
level=INFO source=llama_server.go:457 msg="starting llama-server" cmd="/Applications/Ollama.app/Contents/Resources/llama-server --model ...

MLX:

level=INFO source=client.go:508 msg="starting mlx runner subprocess" model=qwen3.5:4b port=53510
level=INFO source=server.go:46 msg="MLX engine initialized" "MLX version"=0.32.3-96-gbebd642 device=gpu
level=INFO source=model.go:137 msg="Model architecture" arch=Qwen3_5ForConditionalGeneration

To find out which engine a model you loaded earlier used:

grep -E 'using llama-server for model|starting mlx runner subprocess|MLX engine initialized' ~/.ollama/logs/server.log | tail -5

The process list shows the same split. The GGUF model runs as /Applications/Ollama.app/Contents/Resources/llama-server --model <blob path> .... The MLX model runs as /Applications/Ollama.app/Contents/Resources/ollama runner --model qwen3.5:4b --port ....

Step 4: check what a tag will give you before you pull it

The registry tells you whether a tag carries an MLX build. This request downloads only a few hundred bytes of JSON:

curl -s -H 'Accept: application/vnd.ollama.manifest.list.v2+json' \
  https://registry.ollama.ai/v2/library/qwen3.5/manifests/4b | python3 -m json.tool

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.ollama.manifest.list.v2+json",
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "digest": "sha256:d8b0f5e9760cd1682034f292d7ef72ec46f432149be0df7574bf2d6e92e38c04",
            "runner": "llamacpp",
            "format": "gguf"
        },
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "digest": "sha256:7f3251fa6878a78a606bcb3c074283306e5ad2ea411a7774a9dd98a4bc7f2d13",
            "runner": "mlx",
            "format": "safetensors"
        }
    ]
}

A manifest.list with an mlx child means a Mac will get MLX. The MLX child's digest, 7f3251fa6878, is exactly the ID ollama list showed after ollama pull qwen3.5:4b. The GGUF child, d8b0f5e9760c, is the same manifest that qwen3.5:4b-q4_K_M serves on its own. If a tag returns a plain manifest.v2+json, it holds a single build, and its layers tell you which: one big image.model layer means GGUF, hundreds of image.tensor layers mean safetensors.

What I found for a few tags on 2026-10-22 (download sizes summed from the manifests):

That table explains this Mac. Its gemma4:12b-it-q4_K_M has ID 6114515d63c1, which is the GGUF child of gemma4:12b. Pulled with a quantization in the tag, it is GGUF and will always run on llama.cpp. Pulled as gemma4:12b, the same model would have arrived as MLX. I did not pull gemma4:12b to confirm this. The prediction comes from the registry manifest and the selection code below.

Does MLX make it faster? One pair, measured

I ran the two qwen3.5:4b builds back to back with the same prompt through /api/generate: temperature 0, a fixed seed, thinking off, and exactly 256 generated tokens (eval_count was 256 on every run). Each prompt started with a random session ID so neither engine's prefix cache could skip the prompt. The MLX log shows at most 5 of about 1,300 tokens matched from cache. The script ran the builds in the order MLX, GGUF, MLX, GGUF. Each block got one warm-up run and then five measured runs, for 10 measured runs per build. The script waited until the 1-minute load average was below 4 before each run. It stayed between 1.65 and 3.21 throughout.

Medians, with min–max in parentheses. The ~1,300-token rows are 10 runs per engine. The ~8,000-token rows are a shorter follow-up with 3 runs per engine after one warm-up, run with the same script and the load average between 2.98 and 3.53.

So on this pair, the MLX NVFP4 build read the prompt 1.65x faster at 1,300 tokens and 1.78x faster at 8,000. The Q4_K_M GGUF build on llama.cpp generated 12–13% faster. The spread inside each build is small next to the gap between them, and both MLX/GGUF rounds agreed. Short prompts with long answers favored the GGUF build on this Mac. Long prompts favored the MLX build.

The plain CLI shows the decode gap, but its prompt numbers are not useful here. Here is ollama run --verbose on a short prompt, second run of each (thinking off):

# qwen3.5:4b (MLX)
prompt eval count:    30 token(s)
prompt eval cached:   26 token(s)
prompt eval duration: 123.818042ms
prompt eval rate:     32.31 tokens/s
eval count:           225 token(s)
eval duration:        6.67247625s
eval rate:            33.72 tokens/s

# qwen3.5:4b-q4_K_M (llama.cpp)
prompt eval count:    30 token(s)
prompt eval cached:   26 token(s)
prompt eval duration: 34.421ms
prompt eval rate:     116.21 tokens/s
eval count:           293 token(s)
eval duration:        8.360133s
eval rate:            35.05 tokens/s

Over three runs each, the median eval rate was 33.72 tok/s for the MLX build and 34.45 for the GGUF build. The answers differed in length, so the CLI gap is noisier than the controlled runs. The prompt eval rate lines point the other way, 32.31 against 116.21. That is not a prefill result: 26 of 30 tokens were cached, so each rate covers a 4-token remainder plus fixed overhead. Two things --verbose will not tell you: which engine ran (there is no engine line in 0.40.8's output), and a meaningful prompt speed when you repeat a prompt. On the third MLX run, 29 of 30 prompt tokens came from cache, and the reported prompt eval rate was 51,174 tokens/s. To measure prefill, change the start of the prompt on every run, as the script above does.

This is not a pure engine comparison, and the two builds are not the same weights:

• The quantizations differ. The MLX build is NVFP4 and the GGUF build is Q4_K_M. Ollama does not publish the same quantization in both formats for this model.
• The embedding matrix differs. The model ties its input embedding to its output layer (tie_word_embeddings: true in the MLX build's config.json). In the MLX build that 248,320 × 2,560 matrix is stored at 2 bytes per weight (1.27 GB on disk). In the GGUF build token_embd.weight is Q6_K. The output layer is read once for every generated token, so this is a plausible contributor to the decode gap. I did not isolate it, so treat it as a hypothesis.
• No speculative decoding on either side. The 0.31.1 and 0.32.6 release notes describe the MLX engine drafting tokens with a model's built-in multi-token-prediction (MTP) head, for Gemma 4 and Qwen3.5. This particular 4B build ships no MTP tensors, and the server log has no speculative decode stats lines from any run. Builds that do ship an MTP head can behave very differently.

Ollama's own headline numbers (1,810 tok/s prefill and 112 tok/s decode for MLX, against 1,154 and 58) compare Ollama 0.19 against 0.18 on a 35B mixture-of-experts model, NVFP4 against Q4_K_M, with the older engine on the 0.18 side. They do not predict what a 4B model does on 0.40.8, and my numbers do not predict yours.

Why it works this way: the code path in 0.40.8

Every claim in this section can be checked against the v0.40.8 source (commit f580b5484a58).

At load time, the format is the only test. The scheduler asks one question of the model's config (server/images.go):

func (m *Model) IsMLX() bool {
	return m.Config.ModelFormat == "safetensors"
}

If that is false, the scheduler reads GGUF metadata and starts llama-server (server/sched.go). If it is true, it starts the MLX runner subprocess (mlxrunner.NewClient, line 633). The choice itself does not look at RAM size, chip model or any setting. (A separate memory fit check runs afterwards; see below.) The one precondition for choosing MLX is that the MLX library loaded (CheckRuntime in mlxrunner/client.go). The same IsMLX() body exists at the v0.19.0 and v0.35.1 tags.

At pull time, the platform picks the format. A manifest list carries one child per runner, and the client takes the first child that matches its preference order (manifest/manifest.go):

func runnerPreferences() []string {
	if runtime.GOOS == "darwin" && runtime.GOARCH == "arm64" {
		return []string{RunnerMLX, RunnerLlamaCPP, RunnerGGML}
	}

	return []string{RunnerLlamaCPP, RunnerGGML, RunnerMLX}
}

PullModel then downloads only the selected child (server/images.go). Pulling qwen3.5:4b on this Mac fetched 4.0 GB, not both builds. On Linux, Windows, or an Intel Mac, the same tag resolves to GGUF.

No fixed RAM threshold, but a fit check. Before pulling an MLX child, Ollama checks that the MLX library loads (mlx.CheckInit()). If it does not, the pull fails with this model requires MLX support, but the MLX runtime is not available (server/images.go). When the model loads, the MLX client compares the size of the weight files against the first GPU's free memory, minus a minimum reserve and OLLAMA_GPU_OVERHEAD. On an integrated GPU like Apple's, it also uses system free memory when that is lower. If the weights do not fit, the load fails with model requires X but only Y are available, or the scheduler first tries evicting other loaded models (mlxrunner/client.go, called from server/sched.go). After loading, the runner pins the model in memory up to Metal's recommended working-set size and logs MLX model exceeds the recommended working set; performance may be degraded if it is bigger (mlxrunner/runner.go). None of this is a fixed number. It scales with your Mac's memory and with whatever else is loaded. On this machine the server reported 25.0 GiB of GPU memory free at load time. The app bundle ships two MLX builds, mlx_metal_v3 and mlx_metal_v4. The loader prefers v4 and skips it on macOS releases older than 26 (mlx/dynamic.go). This Mac loaded mlx_metal_v4. The runner's startup log loads its structured-output library from .../Resources/mlx_metal_v4/, and the code takes that directory from the path of the loaded MLX library (mlxrunner/grammar.go).

There is a hidden way to choose. ollama pull and ollama run accept --runner mlx|llamacpp|ggml. The flag is marked hidden, so it does not show up in --help (cmd/cmd.go). It selects a child from a manifest list. I did not use it. Pulling a different child under a tag you already have rewrites that tag and prunes the blobs it no longer references, and I did not want that on someone else's model store. Where the library offers them, use explicit tags instead (for this model, -q4_K_M for GGUF and -mlx or -nvfp4 for safetensors) and keep both side by side, as I did.

There is no off switch. The 0.40.8 envconfig package has no MLX setting. Outside tests, the code reads two environment variables that touch MLX, and neither decides whether a model uses it. OLLAMA_LLM_LIBRARY set to a value like mlx_metal_v3 forces which MLX build loads. MLX_METAL_RESIDENCY_REFRESH_INTERVAL_MS tunes how Metal keeps weights resident. I did not set or test either, because changing them means restarting the server.

How the rules got here

From the GitHub release notes:

The 0.40.0 wording, "architectures supported by the MLX runtime will automatically run on MLX," is easy to misread. In practice it means: tags that Ollama has republished with an MLX child will pull as MLX. The 0.40.8 runner registers LlamaForCausalLM and Qwen3ForCausalLM, yet llama3.2:3b and qwen3:8b still serve GGUF only, so they run on llama.cpp. The 0.40.2 upgrade rewrites old GGUF files into newer GGUF for llama.cpp (compatmigrate/writer.go is a GGUF writer). It never produces MLX. In 0.40.8 the call that starts it is commented out with // TODO: enable compatibility migration (server/routes.go).

The two claims, settled

"The model format decides the engine." True, for the copy on your disk: safetensors runs on MLX and GGUF runs on llama.cpp, with no exceptions in 0.40.8. It is misleading as advice, because the tag and the platform decide the format when you pull. On a Mac you get MLX by pulling a tag that has an MLX build, usually the bare tag or one ending in -mlx or -nvfp4. You do not get it by changing a setting.

"MLX needs 32GB or more." Not as a fixed rule. The source checks whether this model's weights fit in the free GPU memory at load time, which is a different thing from a 32GB minimum. The sentence comes from Ollama's March 30 preview post, which says "Please make sure you have a Mac with more than 32GB of unified memory" right next to instructions to run qwen3.5:35b-a3b-coding-nvfp4, a model that needs that much. On this 32GB Mac the 4B MLX model's logged peak never went above 6.59 GiB in the runs above. What you do need is enough free memory for whichever model you pick, on either engine. On MLX, a model whose weights do not fit is refused at load time.

If you want a model on MLX

1. Check the tag in the registry (Step 4). No mlx child means MLX is not available for that tag yet.
2. Pull the bare tag or the -mlx / -nvfp4 tag: ollama pull qwen3.5:4b. This is a second download. It does not convert anything.
3. Load it and confirm RUNNER says mlx in ollama ps.
4. Measure your own workload before deleting the GGUF copy. On the pair above, the GGUF build generated text faster.

To remove a copy you no longer want, ollama rm <tag> deletes that tag. Blobs another tag still references stay on disk (manifest/manifest.go, RemoveUnreferencedBlobs).

Limitations

• One model family (Qwen3.5 4B), one Mac (M5, 32GB), one Ollama version. The speed gap could be smaller, larger, or reversed for other models, other quantizations, an M1–M4 chip without the M5's GPU neural accelerators, or a later release.
• The two builds differ in quantization and in how the embedding is stored, so the measurement compares what you get when you pull each tag, not the engines alone.
• The machine was shared and not idle. Load averages ranged from 1.65 to 3.53 during measured runs (the raw uptime line is logged next to every run). All but one run landed within 4% of its median. The exception was one MLX prefill run, 8% under. An idle machine might read somewhat higher on both sides.
• I did not test --runner, OLLAMA_LLM_LIBRARY, ollama create, the MLX memory fit check, or any Intel Mac, Linux or Windows behavior. I also did not load the ggml-labeled model, so the statement that it runs as llamacpp rests on the scheduler code. Those statements come from the source code only.

How I checked this

• Version: ollama --version → 0.40.8. The server log reports Listening on 127.0.0.1:11434 (version 0.40.8).
• Source: shallow clone of github.com/ollama/ollama at tag v0.40.8 (commit f580b5484a58c8370c35d669555bf887783a3f68), plus tags v0.19.0, v0.35.1 and v0.40.0 for comparison. Code was read and searched with grep. Nothing was built.
• Release notes: every release from v0.17 to v0.40.8 fetched with gh release view on 2026-10-22 and searched for MLX, safetensors and runner.
• Registry: manifests and child manifests fetched from registry.ollama.ai/v2/... with curl. Sizes are the sums of the layer sizes in each child manifest.
• Models pulled: qwen3.5:4b (MLX child, 4.0 GB) and qwen3.5:4b-q4_K_M (GGUF, 3.3 GB). Nothing else on the machine was changed. The Ollama server was not restarted or reconfigured. It runs with this machine's existing settings, including a 65,536-token default context.
• Engine identification: one prompt per model, then ollama ps, the process list, and the server log lines written during that load.
• Timing: a Python script that calls /api/generate with stream: false and computes tok/s from the server's own eval_count/eval_duration and prompt_eval_count/prompt_eval_duration, minus cached tokens. It logs uptime before and after every run. The --verbose excerpts come from piping one fixed prompt into ollama run --verbose --think=false, one warm-up and then three runs per model.

References

• Ollama release notes: v0.19.0 · v0.30.0 · v0.34.1 · v0.40.0 · v0.40.2 · v0.40.3 · v0.40.8
• Ollama blog, "Ollama is now powered by MLX on Apple Silicon in preview" (2026-03-30): ollama.com/blog/mlx
• Source at v0.40.8: server/images.go (IsMLX, PullModel, requiresMLX) · manifest/manifest.go (runnerPreferences) · server/sched.go (runner start and the RUNNER label) · llm/server.go · mlxrunner/client.go · mlx/dynamic.go · cmd/cmd.go
• Ollama build notes on MLX ("On macOS arm64, MLX is enabled by default"): docs/development.md
• Model tags: ollama.com/library/qwen3.5/tags · ollama.com/library/gemma4/tags (checked 2026-10-22)