Test record: I measured the pinned DeepSeek V4 Flash 0731 target on
evox3, an AMD Ryzen AI MAX+ 395 system with 128GB physical memory, using ROCm 7.14.0. A sparse four-expert experiment processed 32,512 prompt tokens at 146.65 tok/s, 10.06 times the first working configuration, and completed a one-pass 130,816-token run at 110.22 tok/s. Repeated requests were not answer-stable, so I did not deploy that fast path as the default.
The first deployment did not produce a slow benchmark. It did not load.
The new official DeepSeek V4 Flash 0731 release replaced the preview checkpoint, but the 102.3GB ROCmFP3-MIX quant exposed a backend gap: qtype 105 had no implementation for the non-zero H32 rotations used by the model’s p4mix experts. The server stopped at layer 3, expert 7. A process that never becomes ready has no honest throughput number.
That failure set the order of work: make the new weights load, measure each cumulative change, and keep testing the answer after the attractive number appears.
The optimization ladder
Every stage used the same request: 32,512 prompt tokens, temperature zero, target-only batch one, caches disabled and up to 64 output tokens. The experimental profile used sparse prefill and four routed experts. Each row began in a fresh server and had to return exactly OK on its first request.
| Cumulative stage | Startup | 32K prefill | First-request OK |
Decision |
|---|---|---|---|---|
| First deployment | failed | — | — | implement H32 support |
| H32 correctness baseline | 26.794s | 14.58 tok/s | yes | retain |
| Fused H32 expert matvec | 28.783s | 14.61 tok/s | yes | fused basis |
| Shared H32 input | 24.784s | 14.53 tok/s | yes | no standalone gain |
| Fused multi-token H32 prefill | 26.693s | 24.70 tok/s | yes | retain |
| Streaming model loader | 22.722s | 24.71 tok/s | yes | retain |
| Decode specialization | 22.655s | 24.75 tok/s | yes | reject after decode test |
| Monolithic layer-major prefill | 22.689s | 145.49 tok/s | yes | investigate |
| Four host-compute workers | 22.667s | 146.12 tok/s | no | reject |
| Packed24 FP3 Q1 | 32.811s | 145.96 tok/s | no | reject |
| Fastest one-pass configuration | 24.834s | 146.65 tok/s | yes | reject after repeats |
The fused multi-token H32 kernel produced the first useful gain: 69.1%, from 14.61 to 24.70 tok/s. Streaming model loading then reduced startup by 14.9% without claiming an inference improvement.
The large measured step came from monolithic layer-major prefill batching. It moved the same 32K request from 24.71 to 145.49 tok/s, a 5.89-times jump. The best first request reached 146.65 tok/s, or 10.06 times the first working baseline.
Two variants already demonstrated why output checks mattered. Four host-compute workers produced an unrelated instruction and x; the packed24 FP3 selector repeated control text and returned OK.. Both were rejected despite their competitive prompt rates.
Then I repeated the fast result
The remaining sparse profile passed its fresh-server request, but it did not remain correct:
| Sparse four-expert check | 32K prefill | Exact OK |
|---|---|---|
| Fresh-server ladder request | 146.65 tok/s | yes |
| Consecutive request through Lemonade | 147.62 tok/s | no |
| Consecutive direct-backend request | 148.02 tok/s | no |
| Fresh request with graph-reset experiment | 138.01 tok/s | no |
The last experiment explicitly discarded the cached graph at a new request boundary. It still echoed or corrupted prompt material, so the patch was rejected. The important distinction is that the throughput was real while the configuration was not production-safe. One exact answer from one fresh process had been too weak a qualification rule.
That changed the deployment decision. Lemonade on evox3 now defaults to accuracy-ar: exact prefill, all six routed experts, target only, a 32,768-token context and caches off. The uncached production profile processed the same 32,512-token prompt at 17.62 tok/s in 30.75 minutes and returned exact OK through Lemonade. Without restarting the container, it then processed an uncached 16,128-token prompt at 19.75 tok/s in 13.61 minutes and returned exact OK again. That consecutive pass is the behaviour the sparse profile could not sustain.
What the 128K number means
Before the repeat problem appeared, I also ran the fast sparse image once with a 131,072-token maximum. A calibrated 130,816-token prompt completed in 19.78 minutes at 110.22 tok/s and returned OK on that first request.
Manual checks near completion showed about 13–14GiB of available memory and roughly 10.24GiB of GTT in use. Total swap use reached about 728MiB, although evox3 already had roughly 411–504MiB used before or early in the run. This is evidence that the 102.3GB quant can process nearly 128K on the machine; it is not a claim that the sparse profile became repeat-stable at that size.
Decode also received its own test. The proposed specialization managed a 19.6 tok/s median over three measured 512-token completions after a warmup; its parent produced 19.7 tok/s. I rejected the change. Those figures are diagnostic because Radeon performance policy remained at auto.
Running 0731 under Lemonade
Lemonade remains the registry, lifecycle manager and OpenAI-compatible API. The pinned 0731 filename is dispatched to a ROCm 7.14 Lucebox container built from e1cd3c9; every other GGUF still falls through to the existing ROCm llama.cpp backend.
The registered model is DeepSeek-V4-Flash-0731-ROCmFP3-MIX. The 10.8GB DSpark draft is downloaded and hash-verified, but remains opt-in because this ladder did not qualify it. I also pinned ROCm 7.14.0, the Lucebox commit, both Hugging Face revisions and both GGUF hashes.
The lifecycle boundary is tested too: Lemonade unloaded the attached container cleanly, reloaded the exact profile in 23 seconds, and returned exact OK from a short post-reload smoke test. I left the model loaded and healthy on evox3.
The useful result is therefore more nuanced than “10× faster”. DeepSeek V4 Flash 0731 now loads and runs under Lemonade on evox3; the sparse path established a compelling performance ceiling; repeated requests exposed a correctness boundary; and the production default moved back to the slower exact path. That is the optimization outcome I would rather operate—and publish.
For the host and runtime choices behind this deployment, see ROCm on Strix Halo without folklore.