AMD · Strix Halo · DeepSeek V4 Flash

ROCm beats Vulkan on Strix Halo

Both engines ran on the same Strix Halo on the same day with the same prompt. With the DSpark drafter Lucebox ROCm decodes at 41.9 tok/s at 8K and 38.0 tok/s at 123K prompt tokens. That is 28% to 47% faster than llama.cpp Vulkan v0.7.5 running the same drafter and its prefill is 17% to 47% faster. Plain decode with no drafter is measured on both engines too.

By Davide CiffaSeptember 14, 2026Engineering report

A Strix Halo mainboard on a wooden deck under a starry sky, with the ROCm and Vulkan logos above it — One Strix Halo GPU with all six routed experts and no prompt compression.

283.9tok/s prefill
at 123K prompt tokens

38.0tok/s speculative decode
adaptive width, q5 chosen on every step

+47% / +35%prefill / decode over llama.cpp Vulkan
at 123K prompt tokens

Single runs at temperature 0 with native server timings; every answer was checked against the same expected hash. Lucebox is the plain launch of PR 729. Vulkan is the Nathanw1014/strix-halo-llamacpp fork v0.7.5 with the published DSpark drafter.

Watch it run

Both engines on one Strix Halo, replayed from the 123K-token timings measured for this post. Prompt reading is shown sixty times faster than real time. The answer is typed in real time at the measured decode speed, and the clocks show real time.

Side-by-side replay of llama.cpp Vulkan and Lucebox ROCm reading a 123K-token prompt and typing a Python answer with DeepSeek V4 Flash on one Strix Halo — Vulkan reads the prompt at 193.6 tok/s and answers at 28.1 tok/s. Lucebox reads it at 283.9 tok/s and answers at 38.0 tok/s. The code is a real DeepSeek V4 Flash answer to a short Dijkstra prompt, typed at those speeds.

Speculative decode

Both engines ran on the same Strix Halo box on September 14, 2026 with the same five-key retrieval prompt at 8K, 32K and 123K prompt tokens, one request at a time on a fresh server per context. The published v0.7.0 table is in the plots as a dashed line for reference; it did not reproduce on this box.

Prefill throughput by prompt length

Lucebox ROCmllama.cpp Vulkan v0.7.5llama.cpp Vulkan v0.7.0 published table

Prompt length on a linear axis. Only the three marked points were measured and the lines only connect them. The dashed line is the published v0.7.0 table, which did not reproduce on this box.

Speculative decode throughput by prompt length

Lucebox ROCm with DSparkllama.cpp Vulkan v0.7.5 with DSparkllama.cpp Vulkan v0.7.0 published table

Native server-reported rates at temperature 0. Lucebox ran adaptive width q2 to q5 and chose q5 on every step. Vulkan ran a fixed draft length of four because the fork has no adaptive mode. Only the three marked points were measured.

Speculative decode on our Strix Halo box in tok/s. The published v0.7.0 table reads 254 / 37.7, 251 / 39.5, 227 / 35.7 and did not reproduce here.
Prompt	Lucebox prefill	Vulkan prefill	Lucebox decode	Vulkan decode
8K	283.3	241.1	41.9	28.5
32K	306.6	237.2	41.4	32.3
123K	283.9	193.6	38.0	28.1

Lucebox prefill is 17% to 47% faster than Vulkan and speculative decode 28% to 47% faster, with the widest prefill gap at 123K. The two engines run different quantizations: Lucebox its ROCmFPX-MIX target with its own DSpark draft, Vulkan the UD-IQ3_XXS target with the published drafter. This compares what each engine delivers as configured, not identical arithmetic. The 8K prefill row is the first request after a server start and pays that cost once; warm re-runs at 8K read 323 tok/s, in line with 32K.

Plain decode

The baseline is the same prompt with no drafter on either side: same box, same contexts, autoregressive decode on both engines, measured on September 14.

Plain decode on our Strix Halo box in tok/s. The last column is each engine’s speculative decode divided by its plain decode.
Prompt	Lucebox prefill	Vulkan prefill	Lucebox decode	Vulkan decode	Speculative gain Lucebox / Vulkan
8K	321.5	254.3	21.2	17.3	2.0× / 1.6×
32K	318.0	247.7	18.4	17.4	2.3× / 1.9×
123K	287.4	194.8	12.9	15.8	2.9× / 1.8×

Without a drafter, Lucebox prefill is 26% to 48% faster at every context. Plain decode is 23% faster at 8K, 6% faster at 32K and 18% slower at 123K, where attending over a long history costs our plain path more than Vulkan’s. With the drafter, Lucebox verifies up to five tokens per step and its speculative decode is 2.9× its plain decode at 123K; Vulkan gains 1.8× with the same drafter at a fixed draft length of four.

Adaptive verify width

Retrieval accepts almost every draft token, so it does not test the verify width. Code, math and prose do. Lucebox picks the width per step, two to five draft tokens, from the drafter’s confidence head, calibrated against what the target actually accepts. On code and math it stays at q5 on almost nine steps in ten; on prose it drops to q2 on nearly every step. Vulkan has no adaptive mode and always drafts four.

Lucebox decode in tok/s on four prompts per suite with a 512-token cap at temperature 0, before and after the adaptive width. Code and math answers were 4/4 on both settings.
Suite	Lucebox adaptive q2 to q5	Lucebox before PR 729 fixed q4
HumanEval	41.5	33.6
GSM8K	36.8	30.6
MATH	40.6	30.1
Agent	38.9	29.6
Prose	24.2	18.1

Pooled over code, math and the agent suite that is 39.4 tok/s, 29% more than the fixed width of four the engine shipped before. On prose it is 24.2 tok/s, 34% more than fixed q4. Vulkan on the same four prose prompts reads 17.1 tok/s, so Lucebox is 42% faster there; the other suites were not run on Vulkan.

What changed in the engine

Packed weights, multiplied directly. The mixed ROCmFP2/3/4 path runs quantized matmul on the packed expert weights instead of expanding them first.
Layer-major sparse prefill. Gate and up projections batched across prompt bands, attention streamed through rocWMMA.
Fused five-token verify with cached graphs. Up to five draft tokens checked per step; the verify graphs are built once and reused across hundreds of steps.
Exact rollback. Rejected drafts restore the affected rows of the rolling cache, seed row included; fixed-width prose output is byte-reproducible run to run.

All of it is PR 729, installed as the gfx1151 device profile, so the plain launch gets it.

Run it

Build the engine for gfx1151 as in the README, then launch with the two DSpark switches. Everything else on this page is the device profile’s default.

DFLASH_DS4_SPEC=1 \
DFLASH_DS4_DRAFT=/path/to/DeepSeek-V4-Flash-0731-DSpark-draft-Q4RMFP4-denseF16.gguf \
DFLASH_DS4_SPARSE_DECODE_FLASH=1 \
dflash_server /path/to/DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf \
  --target-device hip:0 --max-ctx 131072 --chunk 8192 \
  --cache-type-k q4_0 --cache-type-v q4_0 \
  --ds4-fused-decode --ds4-fused-verify-f16-kv \
  --ds4-expert-top-k 6 --ds4-prefill sparse

Setup

Host. AMD Ryzen AI MAX+ 395, Radeon 8060S / gfx1151, 128 GB, ROCm 7.2.4, GTT capped at 110 GiB. The box also holds an R9700, which stayed idle and out of the run.
Lucebox. PR 729 defaults, plain launch: no tuning environment, only the speculative switch and the draft path. The 98 GB ROCmFPX-MIX target with the 10.6 GB Q4RMFP4 DSpark draft and a 131K-token allocation at every point.
Vulkan. Nathanw1014/strix-halo-llamacpp v0.7.5 portable build with the DeepSeek-V4-Flash-0731-UD-IQ3_XXS target and the published DSpark drafter at a fixed draft length of four; q8_0 K/V, batch 2048, microbatch 1024 (512 at 123K), flash attention on, GGML_VK_FA_WAVE32=1 and GGML_VK_LIGHTNING_INDEXER_SMALL_CM=1; a 32K-token allocation for 8K and 32K and 131K at 123K, under a memory guard.
Prompt and timing. The same five-key retrieval prompt on both engines, thinking off, no prefix-cache reuse, a fresh server per context with the page cache dropped. Every run found all five keys and produced the same answer. Native server timings, single runs.
Limits. Each engine runs its own published or default model files, so this compares the two engines as delivered rather than identical arithmetic. Nothing here says anything about prompts longer than 123K.

Bottom line

On one Strix Halo with one prompt, Lucebox ROCm with the DSpark drafter beat llama.cpp Vulkan with the same drafter at 8K, 32K and 123K by +17% to +47% on prefill and +28% to +47% on decode. Without a drafter it is ahead on prefill everywhere and on decode up to 32K and behind at 123K. With the drafter and the adaptive verify width it is ahead at every context on retrieval, code, math and prose. All of it is the default launch: install the engine and point it at the model and the draft.

Source: github.com/Luce-Org/lucebox (PR 729). Vulkan: Nathanw1014/strix-halo-llamacpp v0.7.5 portable build; published v0.7.0 tables here with pinned data, not reproduced here. All chart values and table rows are read from one dataset module; nothing is baked into artwork.

Run DeepSeek V4 Flash on one Strix Halo

Open-source. One command. The model's own output, faster.

GitHub DeepSeek on Strix Halo Discord