DeepSeek V4 Flash on Strix Halo: tuning ROCm, matching Vulkan

Test record: I repeated the DeepSeek V4 Flash Vulkan-versus-ROCm comparison on evox3, a Ryzen AI MAX+ 395 / Radeon 8060S (gfx1151) system with 128GB unified memory. The model was the same 97.05GiB four-shard UD-IQ3_XXS GGUF. ROCm 7.14 gained 44.3% in shallow prompt processing after moving from a 512 to 2,048 micro-batch, but an equally configured patched Vulkan build still finished ahead in all four tests: by 11.5% to 47.9%. Both final runs exited cleanly, and the captured kernel log contained no timeout, DeviceLost, wedge or reset.

I started with a result that appeared to show Vulkan ahead of ROCm on three of four DeepSeek V4 Flash tests. Then ROCm tuning seemed to change the answer. Matching that tuning on Vulkan changed it again.

That is the useful part of this benchmark. The final table matters, but so does the sequence: a backend comparison can be technically reproducible and still be unfair because one ordinary runtime setting was left at its default on only one side.

The original result and the question it left open

The original test came from an ASUS PX13 running Fedora 44 on the same Strix Halo GPU architecture. Its Vulkan build included a LIGHTNING_INDEXER implementation that avoided the DeepSeek V4 fallback behind llama.cpp issue #25664. Without that operator, the fallback performs a permute and contiguous copy with a pathological access pattern; at sufficient depth, the work can exceed the AMD compute-ring watchdog and end in vk::Queue::submit: ErrorDeviceLost.

The patched Vulkan run completed instead of losing the device. Its published numbers were:

All rates in this article are tokens per second. The original post identified Vulkan build 867 at revision 4a1fb6c and ROCm build 824 at cd0fa60. It did not state the ROCm userspace version, so those ROCm numbers cannot honestly be labelled “ROCm 7.14”. My tests below did use ROCm 7.14.60850.

The original Vulkan patch set is available in the author’s vk-indexer-plus-hc branch; the indexer-only version is in vk-lightning-indexer. The published result tested all three patches together, so it was evidence for the combined result rather than an isolation test of any one patch.

Repeating it on evox3

My host differs from the original machine in enough ways that I treat this as a reproduction, not a pooled dataset:

Both machines used -mmp 0. At roughly 99GiB loaded, memory mapping can cause the page cache and GPU-visible allocation to compete for the same unified memory budget. This is a capacity requirement on these configurations, not a throughput tweak.

My first run retained the default 512 micro-batch and used one measured repetition. Vulkan was the patched d93e2df7 build 10329; ROCm was cd0fa605 build 10286 under ROCm 7.14.

The shape was familiar: Vulkan led generation, while prompt processing was closer and changed with depth. The important next question was whether ROCm had unused headroom in configuration or whether the gap sat inside the quantised kernels.

The ROCm settings that mattered

The safe ROCm configuration was simple:

-fa on -ub 2048 -mmp 0

Flash attention remained enabled, the micro-batch rose from 512 to 2,048, memory mapping stayed off, and HIP graph behaviour remained at its default. The one-repetition screen was:

After qualification over three repetitions, 2,048 versus the fresh 512 screen improved shallow prefill by 44.3%, shallow generation by 5.6%, and deep prefill by 25.0%. Deep generation was 3.9% lower, within a notably noisy measurement. Disabling flash attention was a clear loss and nearly halved the deep generation rate.

Changing graph behaviour did not produce a meaningful win:

At this point, comparing tuned ROCm with the earlier Vulkan default would have made ROCm look decisively faster at both prompt-processing tests. It would also have been the wrong comparison.

Match the micro-batch and the result changes

I repeated Vulkan at the same 2,048 micro-batch. The common arguments were:

-ngl 99 -p 2048 -n 64 -ub 2048 -r 3 -mmp 0

ROCm used -fa on; the Vulkan build accepted -fa 1. The deep tests added -d 24576. Both used the same four model shards, the same host power policy, and no speculative decoding.

The final matched result was:

The tuned ROCm gain is real. So is the Vulkan win. A larger micro-batch substantially accelerated prefill on both backends, and the matched Vulkan build led all four workloads.

Here is the complete progression in one place:

Those six columns are three comparison stages, not six interchangeable samples. The original figures come from another chassis and software stack. The initial evox3 numbers use ubatch=512 and one repetition. Only the final pair is the three-repetition, same-host, same-setting backend comparison.

Why I did not keep a ROCm kernel patch

Configuration tuning improved prompt processing, but it did not explain the remaining generation gap. I profiled the exact IQ3_XXS matrix-vector shape used by this model: m=4096,n=1,k=14336.

rocprofv3 attributed 98.62% of kernel-dispatch time to mul_mat_vec_q; quantize_q8_1 accounted for the remaining 1.38%. The unprofiled upstream microbenchmark took 122.09µs per call. That is a strong signal to work on the quantised matrix-vector kernel, but a strong signal is not the same as an easy win.

Every candidate passed all 11 IQ3_XXS CPU-reference cases. Extra wavefronts and rows nevertheless increased lookup or register pressure enough to lose performance. Shared-memory staging occasionally won, then reversed direction in several paired trials. I rejected it rather than retaining a fragile change that I could not distinguish from noise.

The practical outcome is that there is no local ROCm kernel patch behind these published results. The isolated worktree was returned to pristine upstream and rebuilt successfully.

Correctness and stability came before the score

The Vulkan indexer patch exists because “fast until the driver kills the context” is not a usable result. I applied the same standard to the ROCm experiments:

• LIGHTNING_INDEXER passed 144 of 144 supported ROCm cases in normal asynchronous mode and again with HIP_LAUNCH_BLOCKING=1;
• the DeepSeek host-combine operators plus CONT passed 48 of 48 cases in both modes;
• every IQ3_XXS kernel candidate passed 11 of 11 CPU-reference tests;
• the final ROCm and Vulkan benchmark commands both exited zero;
• the captured kernel log contained no ring timeout, DeviceLost, wedge or GPU reset.

I did not change clocks, firmware, the kernel, the memory split or the ROCm installation during this work. That matters because changing the platform under the benchmark can turn a tuning exercise into an attribution problem.

What I would run

For this particular DeepSeek V4 Flash UD-IQ3_XXS GGUF, the patched Vulkan build is the faster llama.cpp path on evox3. It wins prompt processing and generation at both tested depths. ROCm 7.14 remains stable and materially better with -ub 2048, but configuration alone does not overtake Vulkan.

That does not make ROCm the wrong production backend. My specialised ROCmFP3 service, restored after the benchmark with the same model profile and options, returned the deterministic smoke output BENCHMARK RESTORE OK at 24.9 decode tokens per second. It is not an apples-to-apples comparison with upstream llama.cpp, a different quantisation and a patched Vulkan build, but it remains the practical production route when I want more decode throughput from this model on this machine.

So I keep both paths:

The last row is important. Another report still crashed with a related patch applied on different hardware, a Gemma model and --parallel 5. The DeepSeek fallback fixed here is a concrete failure mode, not proof that every Vulkan device loss shares one cause.

My earlier Vulkan-versus-ROCm article concluded that backend choice was model- and phase-specific. DeepSeek V4 Flash makes the same point more sharply: it is also configuration-specific. Tune one side, then carry the winning setting across the boundary before declaring a winner.

The numbers changed twice. The disciplined conclusion changed only once the comparison stopped moving.

Benchmark date: 9 August 2026. Model revision fbbb5b93; patched Vulkan build 10329 at d93e2df7; initial ROCm build 10286 at cd0fa605; final upstream ROCm build 10333 at 08659901c; ROCm userspace 7.14.60850. All evox3 figures come from retained same-machine benchmark and profiler artefacts.