# Local AI on a Strix Halo mini PC: our setup, tuning and benchmarking recipe

Updated 2026-10-14 by Meauxsoft. Companion to the benchmark page:
https://www.meauxsoft.com/News/AI_Benchmark.html

This is what worked for us on one machine, written so a person or their AI assistant can
follow it. Versions move quickly; re-check anything marked with a version before relying
on it. Where we measured something, the number is ours, from this machine.

Tested with: Windows 11 Pro, AMD Adrenalin 26.9.1, Lemonade 2026.41.1 (llama.cpp on
Vulkan), OpenCode 1.18.33.

## 1. The machine

- Minisforum MS-S1 MAX: AMD Ryzen AI Max+ 395 ("Strix Halo") with Radeon 8060S graphics.
- 128 GB LPDDR5X at 8000 MT/s, shared by the CPU and GPU. That is the chip's rated maximum;
  anything faster is overclocking, which we did not do.
- Memory bandwidth (about 256 GB/s) is what limits writing speed: each new token reads the
  whole active model. Prompt reading is compute-bound and is where tuning helps.

## 2. Windows and GPU memory

1. **Update the AMD driver first.** The factory driver was months old. Install the current
   Adrenalin from amd.com (Full Install). We skipped the optional "AI Bundle" (it adds
   other model runners we did not need).
2. **Give the GPU 96 GB.** AMD Software → Performance → Tuning → System → Variable Graphics
   Memory → Custom → 96 GB → reboot. Windows then sees about 31.6 GB. This is a fixed
   split: memory given to the GPU does not come back to Windows while it is unused.
   Check: `(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory/1GB` ≈ 31.6.
3. **Long unattended runs: keep the screen on.** With the GPU at full load, the display
   timeout put Windows into Modern Standby and the driver missed its power watchdog:
   blue screen `0x19C WIN32K_POWER_WATCHDOG_TIMEOUT`. Set Settings → System → Power →
   Screen timeout (plugged in) to Never, or have your runner hold the display on
   (`SetThreadExecutionState`) for the length of a job.
4. Windows' "commit charge" counts GPU allocations, so `llama-server` shows a 35–48 GB
   commit size. That is bookkeeping, not RAM; keep the page file system-managed.
5. With two models loaded, Windows' remaining ~31 GB gets tight. Unload the model you are
   not using if the desktop feels slow.

## 3. Lemonade (AMD's local model server)

- Install: `winget install AMD.LemonadeServer` (not `Lemonade.Lemonade`, a different
  product). Open a new terminal afterwards.
- API: `http://127.0.0.1:13305/api/v1` (OpenAI-compatible). Each loaded model also gets its
  own llama.cpp server port (8001, 8002, …).
- **Keep the backend on `auto` (Vulkan).** With Adrenalin 26.9.1, the ROCm backend reported
  "no ROCm-capable device", and Lemonade then ran the model on the CPU (~18 tok/s reading,
  ~4.5 tok/s writing). Back to Vulkan: `lemonade config set llamacpp.backend=auto`, then
  reload the models.
- `lemonade config set max_loaded_models=2` keeps two chat models loaded.
- Context size is set per model in the model's settings (wrench) dialog. We use 262144 for
  the Qwen models and 131072 for Gemma 4.
- Models from Lemonade's Suggested list get Qwen's recommended sampling automatically.
  Models imported from a Hugging Face link do not; add to that model's llama.cpp arguments:
  `--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 --repeat-penalty 1.0`
- Check what is loaded and with which arguments:
  `curl.exe -s http://127.0.0.1:13305/api/v1/health` (each model's `launch_command`).
- **Put the model files on a fast, unencrypted drive.** Lemonade downloads to
  `%USERPROFILE%\.cache\huggingface`. We moved that folder to another drive and left a
  junction in its place: `mklink /J "%USERPROFILE%\.cache\huggingface" "D:\Lemonade\huggingface"`
  (Lemonade closed, folder moved first). A BitLocker drive read the same file at about
  half the speed.
- **After a Lemonade update, quit and restart it once.** The installer starts it in a mode
  that refuses links like the junction above ("untrusted mount point"); a normal start is
  fine.
- **A model whose `.gguf` has no built-in MTP head** (some Unsloth releases ship it as a
  separate file) fails to load when Lemonade adds `--spec-type draft-mtp` ("model doesn't
  contain MTP layers"). Add `--spec-type none` to that model's arguments.

## 4. How we tune, before any benchmark

**Method.** Time each change on the same 6,000-token code prompt with a 256-token reply and
no prompt cache, three runs. Compare runs two and three; the first run after a load or an
idle spell is a slow warm-up. Confirm a winner on a 20,000-token prompt. Change one thing
at a time. A small script that posts the prompt to the model's llama.cpp port and reads
its reported timings is all you need.

**What paid off: the micro-batch size, `-ub`,** set per model in its llama.cpp arguments.
The best value is opposite for dense and mixture-of-experts (MoE) models, so measure each:

| Model | `-ub` 128 | 256 | 512 (default) | 1024 | 2048 | We use |
|---|---|---|---|---|---|---|
| Qwen3.8-27B (dense), reading tok/s | **~242** | ~234 | ~225 | ~208 | ~194 | `-ub 128` (+7%) |
| Qwen3.6-35B-A3B (MoE), reading tok/s | ~500 | ~686 | ~850 | **~1,024** | ~917 | `-ub 1024` (+20%) |

`-ub 64` also slowed the 27B model's writing. For a very large model (75 GB on the GPU) we
kept the default: a larger `-ub` read faster on short prompts but was unstable on a long one
(not re-tested cleanly, so treat that as unresolved).

**Already at their best, nothing to change:** flash attention (llama.cpp's `auto` turns it
on; forcing it off was ~4% slower); full GPU offload (`-ngl auto` puts the whole model on
the GPU when it fits); Windows power mode (Best performance vs Balanced: no measurable
difference).

**Advice written for discrete graphics cards that does not apply here:** splitting MoE
layers to the CPU (everything already runs on the GPU), `--fit` settings (our models fit),
and GPU minimum-clock tweaks (no such setting for this GPU; it runs at full clock under load).

**Gotchas while testing big models:**
- After a killed or stuck `llama-server`, Windows can take a while to release its GPU
  memory. A model loaded too soon spills into system RAM (loads 3× slower, reads ~4 tok/s).
  Wait until the `\GPU Adapter Memory(*)\Dedicated Usage` counter is back near 2 GB.
- Prompt reading is the slow part of local agent work: one 138,000-token review spent 23
  minutes just reading. Keep agent contexts small (diffs and targeted reads, not whole files).

## 5. The models we use, and for what

Speeds below are averages over our real agent sessions (long contexts, tool calls), so they
are lower than short-prompt tests (Qwen3.8-27B reads ~240 tok/s on the 6K tuning prompt).
GPU memory is measured after loading with one short reply.

| Model | Quant (file) | Reading / writing tok/s in agent runs | GPU memory at 32K / long context | Our use |
|---|---|---|---|---|
| Qwen3.8-27B | UD-Q6_K | 120 / 15 | 24 / 39 GB at 256K | **Best local reviewer and coder.** 94% on our three coding benchmarks, close to the best cloud models, at 20–40× the time |
| Qwen3.8-27B | UD-Q4_K_XL | 130 / 18 | 20 / 35 GB at 256K | A little faster, a little less careful |
| Swift-Qwen3.8-27B | Q6_K | 126 / 16 | 25 / 40 GB at 256K | Same model tuned to think less: best value for plans |
| Qwen3.6-35B-A3B | UD-Q4_K_XL | 449 / 61 | 24 / 29 GB at 256K | Fast "hands" for precise, spelled-out edits; not a reviewer |
| Nemotron 3.5 Lightning 30B-A3B | UD-Q4_K_XL | 579 / 70 | 25 / 29 GB at 256K | Fastest; precise edits only |
| Gemma 4 26B-A4B | UD-Q4_K_M | 104 / 49 | 19 / 21 GB at 128K | Precise edits only; vision untested yet |
| Gemma 4 31B | Q4_K_M | 25 / 17 | 23 / 31 GB at 128K | Reads very slowly here (~110 tok/s at 7K, ~35 at 50K+) |

Thinking level: for Qwen3.8 UD-Q6_K, `low` was not faster on agent work (it wrote more tokens),
and `xhigh` took longer on our hardest review without finding more. `medium` is our setting.

## 6. OpenCode (the coding agent)

- Install: `winget install SST.opencode`; run `opencode` in the project folder.
- Configure the provider yourself in `%USERPROFILE%\.config\opencode\opencode.jsonc` and set
  `"model"` to a local model, so nothing falls back to a cloud model. (`lemonade launch
  opencode` started OpenCode on its own cloud model for us.) A running OpenCode does not
  reload its config: exit and start it again.
- Provider: `@ai-sdk/openai-compatible`, `baseURL` `http://127.0.0.1:13305/api/v1`, model
  ids exactly as Lemonade's `/api/v1/models` lists them, `limit.context` equal to the
  context loaded in Lemonade.
- Thinking level per request, as variants:

```jsonc
"variants": {
  "medium-nopreserve": { "chat_template_kwargs": { "enable_thinking": true, "reasoning_effort": "medium", "preserve_thinking": false } },
  "low":    { "chat_template_kwargs": { "enable_thinking": true, "reasoning_effort": "low" } },
  "xhigh":  { "chat_template_kwargs": { "enable_thinking": true, "reasoning_effort": "xhigh" } },
  "off":    { "chat_template_kwargs": { "enable_thinking": false } }
}
```

  Qwen3.8 accepts `low`, `medium` and `xhigh`. `preserve_thinking: false` follows Qwen's own
  advice to keep earlier turns' reasoning out of the history.
- **Slow readers time out.** A model that reads a 50,000-token step slowly can hit
  Lemonade's request timeout or OpenCode's 5-minute header and chunk timeouts ("CURL error:
  Timeout was reached"). For those, add a second provider that talks straight to the
  model's llama.cpp port with the timeouts off:

```jsonc
"llamaserver": {
  "npm": "@ai-sdk/openai-compatible",
  "name": "llama-server direct (no timeouts)",
  "options": { "baseURL": "http://127.0.0.1:8001/v1", "timeout": false, "headerTimeout": false, "chunkTimeout": false },
  "models": { "Gemma-4-31B-it-MTP-GGUF": { "limit": { "context": 131072, "output": 32768 } } }
}
```

- For unattended runs we allow edits and commands but deny every git command that changes
  shared state (`git add/commit/push/reset/checkout/restore/clean/stash/rebase`, including
  the `git -C <path> …` forms) and deny work outside the project folder
  (`"external_directory": "deny"`).

## 7. How we benchmark (so the numbers mean something)

- **Real tasks with fixed answer keys.** Each benchmark is a frozen copy of a real change
  from our own repository: a standalone git repo with one snapshot commit and no history,
  so the later fix cannot be looked up. The answer key comes from the real review, written
  before any model runs, and is kept outside the benchmark folder.
- **Same prompt, same starting state.** A reset script restores the exact snapshot after
  every run and saves the model's output for scoring.
- **No outside help.** The model may not use MCP servers or files outside the benchmark.
- **Turn off assistant memory for benchmark runs.** We found that Codex's memories carried
  earlier runs' findings into later runs, which inflated scores until we re-ran them clean.
  In `%USERPROFILE%\.codex\config.toml`:

```toml
[features]
memories = false

[memories]
generate_memories = false
use_memories = false
```

  Check other tools too (for example Claude Code's per-project memory). Before scoring a
  cloud run, check that tool's session log: no memory loaded, no MCP tool called.
- **Score against the key, penalize wrong claims.** A false statement about the code costs
  points; a build or test failure caps the score at 50%. Run each setup more than once where
  you can: the same model's reviews vary a lot from run to run.
- **Windows Smart App Control judges every unsigned file by cloud reputation, and the verdict
  can change.** On 2026-10-12 it first blocked a freshly built test DLL ("An Application Control
  policy has blocked this file", 0x800711C7; hundreds of tests failed for a reason unrelated to
  the change), then Lemonade's own `llama.dll`, unchanged for weeks, so no model could load
  ("Bad Image", 0xC0E90002). llama.cpp builds, local model servers and your own test builds are
  all unsigned. Our development machine only: we turned Smart App Control off there (Windows
  Security → App & browser control); Defender and SmartScreen stay on. It lowers that PC's
  protection, so decide for your own machine, and check Microsoft's current guidance on turning it
  back on before you switch it off.

Results, per benchmark and combined, with speed and memory charts:
https://www.meauxsoft.com/News/AI_Benchmark.html