Source record: open llama.cpp PRs 28213 and 28244 independently compact the K/V cells selected by Qwen4Exp QSA before decode attention. Both report increasing gains with context depth. Both were tested on NVIDIA CUDA, and neither is ready for an EVO-X3 production build.
Qwen3.8 Flash Next already uses a sparse indexer to choose roughly 2,048 useful K/V cells. The surprising part is what happens next: current decode turns that selection into a mask over the complete cache, then still runs attention across the full window.
The model chooses a small working set, but the kernel keeps paying a cost that grows with context length. These two pull requests gather the selected cells into compact buffers and run attention over those instead.
The same idea, two implementations
PR 28213 derives its compact attention mask from existing per-cell bias values. It also avoids uploading a reported 17 MB full-window mask per decoded token at 130K. On dual RTX A6000 GPUs with IQ4_XS and Q8_0 K/V, its published decode results are:
| Context | Existing path | Gather path | Change |
|---|---|---|---|
| 31K | 36.5 tokens/s | 38.5 tokens/s | +6% |
| 62K | 26.5 tokens/s | 31.6 tokens/s | +19% |
| 130K | 15.7 tokens/s | 23.6 tokens/s | +50% |
The author reports byte-identical retrieval and short factual answers at all three depths. Long open-ended generations diverged after about 150 tokens, but repeated unpatched runs also diverged, so the PR does not attribute that nondeterminism to the gather path.
PR 28244 keeps the existing mask as the source of truth and gathers its entries alongside the selected K/V cells. It changes less shared graph-input code. On an RTX PRO 6000 with UD-Q4_K_XL, Q8_0 K/V and 1,024 generated tokens, the reported rate is unchanged at 4K and rises from 73.2 to 78.9 tokens/s at 50K: about 8%.
That alternative explicitly says deep-context output is not bit-identical to main because Flash Attention reduces 2K gathered cells in a different order from the full masked window.
Why I would not choose between them yet
The first PR is open and blocked; the second is open with merge conflicts at the revision checked. More importantly, neither includes a Radeon Vulkan measurement. The gather itself may map cleanly to backend primitives, but allocation, transfer and Flash Attention costs differ sharply between dual discrete GPUs and 128 GiB of unified memory.
My current service exposes 262K per slot. That is exactly where avoiding full- window attention should matter most, but it is also where spare memory and correctness margins are smallest. Compact buffers are not free merely because they are smaller than the mask they replace.
The eventual EVO-X3 test
I would wait for one upstream direction to settle, then keep the candidate isolated from MTP and the newly merged Q8 K/V dequant fix. The test needs actual 4K, 32K, 64K, 128K, 192K and 256K occupancy, not only a large configured context.
At every depth I would record decode, per-token latency, GTT, available memory, page faults and output hashes. Retrieval and exact short answers need to be paired with long deterministic generations, ABCCBA state restoration and two-slot cross-content checks. A speed-up that subtly changes selection or mask semantics is not a usable result.
Where I would stop claiming
The CUDA data strongly supports the diagnosis that current QSA decode leaves long-context performance on the table. It does not prove either implementation is correct or faster on Vulkan, and the two proposals do not promise the same numerical path.
I would monitor both rather than backport both. The right candidate is the one that survives upstream review, passes the recurrent/cache matrix and produces a repeatable gain outside noise on the full 125B EVO-X3 service.
Pull-request state and measurements checked 3 September 2026. No local binary was built and no production setting was changed.