Muse Glimmer 30B on Strix Halo: Vulkan passed the 8K test

Test record: I qualified Muse Glimmer 30B on a 128GB GMKtec EVO-X3 with a Ryzen AI MAX+ 395 / Radeon 8060S (gfx1151). I built the same llama.cpp b10362 commit for ROCm 7.14 and Vulkan. ROCm processed a 2,048-token prompt at 434.59 tok/s and reached 42.38 tok/s on a favourable DFlash workload. Vulkan was much slower at prompt processing, but it was the only backend to pass the fresh 8K passkey test. I kept Vulkan without speculative decoding for the correctness-first text service and retained ROCm plus DFlash as the next short-context candidate to qualify.

For the first half of this test, ROCm looked like the easy choice. It was roughly twice as fast as Vulkan at prompt processing on the dynamic quant, and DFlash could push a favourable generation task beyond 40 tokens/s.

Then I moved the passkey beyond roughly 4,000 tokens. ROCm stopped returning the right value; Vulkan returned it exactly at 8,000 tokens. That one failure changed the deployment decision.

Muse Glimmer is a dense 29.6B-parameter multimodal model with tool calling, a 131,072-token context window and a separate DFlash draft model. Both published quants and the vision projector fit comfortably in the EVO-X3’s 128GB unified memory. Fit was not the problem. The test was about how much speed I could use without narrowing the model’s working context by accident.

The numbers that shaped the decision

The distinction between “highest favourable result” and “representative set” is deliberate. Speculative decoding accelerates accepted draft tokens; its benefit changes with the prompt and output. A single arithmetic loop is useful for tuning the ceiling, but it is not an honest forecast for coding, explanation and operational writing.

The exact build I tested

The test host was the same evox3 workstation used throughout this series:

I tested both model quants published in the official repository rather than treating the filename as provenance:

The dynamic quant is the quality-first choice because this machine has ample capacity. The 17GB quant remains useful for a speed profile: it decoded 17% faster than dynamic under the ROCm target-only baseline and leaves more memory for other services.

Muse Glimmer support landed through llama.cpp PR #26841. The model card requires b10353 or newer; using b10362 keeps the model architecture, chat parser, vision path and DFlash integration on one exact source revision for both backends.

Building both backends

The ROCm build targets the APU explicitly and disables HIP graphs, because the graph-enabled control was fractionally slower in every retained benchmark:

cmake -S . -B build-rocm \
  -DGGML_HIP=ON \
  -DGGML_HIP_GRAPHS=OFF \
  -DAMDGPU_TARGETS=gfx1151 \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build-rocm --config Release -j

The Vulkan build uses the same source tree and release mode:

cmake -S . -B build-vulkan \
  -DGGML_VULKAN=ON \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build-vulkan --config Release -j

The common benchmark shape was pp2048 and tg128, repeated three times with all layers on the GPU, flash attention enabled, an 8,192-token batch and 2,048-token micro-batch. Model loading was excluded from the timed result.

ROCm won prefill; Vulkan narrowly won baseline decode

ROCm was 78% faster than Vulkan in 17GB-quant prompt processing and just over twice as fast with the dynamic quant. Vulkan reversed the target-only generation result, but only narrowly: 4.3% on the 17GB quant and 1.8% on dynamic.

HIP graphs did not create a useful ROCm win. Disabling them improved prompt processing by 0.8% on the 17GB quant and 0.2% on dynamic; generation changed by around one tenth of one percent. I retained the simpler graph-disabled build rather than adding a moving part for a negative result.

These are microbenchmarks, not complete chat-request rates. They isolate prompt processing and autoregressive generation so the backend differences are visible; the later correctness and feature tests decide whether a configuration is deployable.

DFlash made ROCm much faster on the right output

The official DFlash model is not plug-and-play with the current GGUF metadata. Its muse-glimmer.attention.sliding_window_pattern is encoded as an array of booleans, while the current DFlash binding expects the scalar pattern. That mismatch is tracked in llama.cpp issue #26894 and can terminate model loading with a vector::_M_range_check exception.

The non-destructive workaround is an explicit metadata override:

--override-kv muse-glimmer.attention.sliding_window_pattern=int:4

That scalar represents the published local/local/local/global attention cycle. DFlash also needs an explicit speculation type in this build:

-md dflash-kquant.gguf
--spec-type draft-dflash
--spec-draft-n-max 15
-ngld 99

The last tuning control was decisive. The default maximum of three draft tokens left most of the opportunity unused. Seven was slower on the coding control, while 15 matched the DFlash block shape and won clearly.

The three representative 512-token tasks averaged 23.46 tok/s, 1.89 times the target-only baseline. The favourable integer set averaged 42.38 tok/s, 3.41 times baseline. Both are true; only the first is a reasonable planning number for mixed work.

Dynamic plus DFlash showed the same shape. A favourable integer run reached 43.87 tok/s, coding reached 31.03 and the B-tree explanation reached 17.68. With seed 123, the speculative and target-only paths also returned identical final content for an exact addition check: 12345 + 67890 = 80,235.

This is the strongest ROCm performance result in the study. I still did not make it the default, because it only helps inside the context range that returns the right answer.

The context test changed the deployment decision

I inserted a unique passkey at the start of progressively longer prompts and asked for that exact value at the end. Each important boundary was repeated from a fresh server so the result did not depend on a reused slot cache.

The ROCm slot really had a 131,072-token context and processed the full prompt; this was not a server-side truncation. Turning flash attention off did not repair the 8K result. The override is also equivalent to the model’s repeated local/local/local/global metadata, so the evidence does not point to a changed attention pattern.

I am describing this narrowly as a current b10362 / ROCm 7.14 / gfx1151 long-context correctness regression. The test does not prove a universal ROCm fault or identify the kernel responsible. It does establish a deployment boundary on this exact machine: I will not expose the ROCm route as a long-context endpoint until a newer build passes the same retrieval gate.

Vulkan returned HALO-MUSE-VULKAN-8000 exactly without speculation. At that depth it processed the prompt at 198.59 tok/s and generated at 10.71 tok/s. DFlash also returned its passkey, but generation fell to 6.96 tok/s because acceptance was poor. On a favourable short integer task, Vulkan DFlash improved dynamic generation by only about 15%. Those controls make the Vulkan choice straightforward: run the target model alone.

I tested 8K retrieval, not the model’s full advertised 131K window. The server can be configured for 131,072 tokens, but that is capacity rather than a published correctness claim. I would qualify 16K, 32K, 64K and 128K before promising the full window to users.

Vision and tool calling

The feature checks used the dynamic quant on ROCm with DFlash 15. They did not use the Vulkan/no-DFlash profile that passed the 8K retrieval test.

With mmproj-kquant.gguf, Muse Glimmer correctly identified a supplied GitHub icon. Loading the first image increased GPU-visible GTT by about 346MiB. I then compared two 10,016-token prefill controls before and after the image: 390.52 and 400.91 tok/s. That does not reproduce the first-image prefill collapse reported for CUDA in llama.cpp issue #26873, although the extra retained memory was visible.

The Jinja tool path also returned a valid OpenAI-compatible tool_calls response for a weather lookup, with the expected get_weather function and {"city":"Paris"} arguments. Reasoning appeared separately from final content, and the ordinary arithmetic smoke test returned 391 for 17 × 23.

Those results establish that the features worked on the tested ROCm route. They do not establish multimodal or tool-call support on Vulkan merely because both backends came from the same source revision. I would repeat the image and tool checks on the exact Vulkan profile before exposing either feature there.

What I would run next

Correctness-first text service

The retained default is the dynamic quant on Vulkan, without DFlash. This text-only command exposes loopback, keeps one slot and caps the qualified retrieval endpoint at 8K:

./build-vulkan/bin/llama-server \
  -m muse-glimmer-30B-kquant-dynamic.gguf \
  -ngl 99 -fa on \
  -c 8192 -np 1 \
  -b 8192 -ub 2048 \
  --load-mode none --no-host --fit off \
  --jinja --temp 1.0 --top-p 0.95 --top-k 64 \
  --host 127.0.0.1 --port 18090

The model supports a 131,072-token context, so the context can be raised after each longer retrieval and multi-turn compaction gate passes. Keeping -np 1 avoids silently dividing the configured context across slots. A public or LAN-facing service also needs authentication and a deliberate network policy; loopback is the safe baseline. The projector is deliberately absent here: adding images or publishing tool calls requires a fresh check on this exact Vulkan/no-DFlash profile.

Fast short-context candidate

The 17GB ROCm quant plus DFlash produced the useful speed result, but the retained passkey checks used the dynamic target without DFlash. I would use the following command as a qualification canary, not yet as a service profile:

./build-rocm/bin/llama-server \
  -m muse-glimmer-30B-kquant-17gb.gguf \
  -md dflash-kquant.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 15 \
  --override-kv muse-glimmer.attention.sliding_window_pattern=int:4 \
  -ngl 99 -ngld 99 -fa on \
  -c 4096 -np 1 \
  -b 8192 -ub 2048 \
  --load-mode none --no-host --fit off \
  --jinja --temp 1.0 --top-p 0.95 --top-k 64 \
  --host 127.0.0.1 --port 18091

The 4K cap is a conservative test ceiling, not a validated safety boundary for this configuration. The 4,086-token pass in the dataset belongs to the dynamic quant without DFlash. Before serving the 17GB/DFlash combination, I would run a fresh 4K retrieval gate on that exact command and only then repeat the 8K and longer tests.

The profile I kept and the candidate I did not promote

ROCm is the quicker option in the shallow, well-tested lane. Its dynamic-quant prefill was twice as fast as Vulkan, and DFlash lifted the mixed 512-token set from a 12.43 tok/s target-only baseline to 23.46 tok/s. On favourable output it exceeded 40 tok/s.

I kept Vulkan as the text default because it returned the exact 8K passkey where the current ROCm path did not. It is slower at prompt processing and only slightly faster at target-only decode. DFlash adds too little on Vulkan—and can make generation substantially slower when acceptance is poor—so the correctness-first text route uses the dynamic target model by itself.

That leaves one qualified text profile and one benchmark candidate:

• Vulkan / dynamic / no DFlash for text retrieval up to the currently qualified 8K context. Multimodal and tool use remain unqualified on this exact profile.
• ROCm / 17GB / DFlash 15 as the next short-context candidate. It is not a deployment recommendation until the exact combination passes the 4K retrieval gate.

The complete retained measurements are available as CSV. The production DeepSeek service was unloaded for the isolated tests and restored afterwards; the benchmark server was not left competing for unified memory.

Benchmark date: 11 August 2026. All figures were measured locally on the same EVO-X3. No backend failure is represented as zero throughput, speculative results are labelled separately from target-only generation, and the 8K Vulkan pass is not presented as a 131K qualification.