July 2026
DeepSeek V4 Flash: 284B model, up to 32 tok/s on AMD Ryzen AI MAX+ 395
AMD-Powered Lucebox runs the full DeepSeek V4 Flash target locally on AMD Ryzen AI MAX+ 395 with 128 GB unified memory: up to 32.0 tok/s decode and roughly 250 tok/s with indexed sparse prefill. The code used for both paths is on Lucebox main.
DeepSeek V4 Flash is a 284B mixture-of-experts model. Its target and draft both fit because the CPU and Radeon 8060S share the same 128 GB LPDDR5X pool. There is no discrete GPU, second machine, or remote inference service in this run: prompts and model execution stay on the local system.
The LocalMaxxing result
We submitted the run to LocalMaxxing. On July 18, its next-fastest DeepSeek V4 Flash entry for the Radeon 8060S was HipFire at 18.99 tok/s. The previous best in the site’s Ryzen AI Max 395 unified-memory group was DwarfStar at 15.6 tok/s.
That puts our run 68.5% ahead of HipFire and at 2.05× the DwarfStar result. These are comparisons against the public LocalMaxxing entries shown above, not controlled A/B tests.
ROCmFPX: fitting 284B weights into 128 GB
ROCmFPX is not one quantization format. It is a family of block formats built around the AMD ROCm/HIP path. Each block holds 32 weights as packed low-bit codes plus one or two small scales. ROCmFP2 stores a block in 10 bytes, or 2.50 bits per weight; ROCmFP3 uses 3.50 bits per weight; and the fast ROCmFP4 layout uses 4.25.
For DeepSeek V4 Flash, we added the missing 2-bit format and its HIP kernels, then built a Strix-specific mixed-precision recipe. The enormous routed-expert gate and up matrices use ROCmFP2, expert down projections use ROCmFP3, and dense or more sensitive projections keep ROCmFP4 or higher precision. We used an importance matrix during quantization and kept the model’s MTP head. The final 102.3 GB target works out to roughly 2.88 bits per parameter; the filename says ROCmFP2 because that is the dominant format, not because every tensor is 2-bit.
(95.3 GiB)
(10.5 GiB)
| Piece | Measured configuration |
|---|---|
| Hardware | Ryzen AI MAX+ 395, Radeon 8060S (gfx1151), 128 GB LPDDR5X |
| Target | DeepSeek-V4-Flash-ROCMFP2-STRIX.gguf, 102.3 GB |
| Draft | DeepSeek-V4-Flash-DSpark-draft-Q4RMFP4-denseF16.gguf, 11.3 GB |
| Runtime | ROCm 7.2.4, HIP gfx1151, platform performance, Radeon high (2.9 GHz observed), q=4 verification cap |
| Server context | 8,192 tokens in the published setup |
The format is also part of the speed result. At batch one, every generated token streams the active weights across all 43 layers, so decode is mostly limited by memory traffic. The ROCmFPX kernels read the packed blocks directly, expand their small codebooks in registers with AMD byte-permute instructions, and feed integer dot products without a separate codebook gather. In practice, the file layout and the GPU kernel are designed as one path.
Decode: up to 32 tok/s
ROCmFPX handles the weight traffic. We then added a DeepSeek-specific HIP decode path for the model’s hyper-connections, attention, routing, and expert work. With no speculative draft, that target runs at 25.31 tok/s autoregressive.
DSpark is the next layer. With a q=4 batch, its small draft proposes up to three new tokens and the 284B target verifies four positions, including the current seed, in one fused pass.
With a q=4 cap and adaptive width disabled, the public run reached 32.0 tok/s, 26.4% above the 25.31 tok/s autoregressive result. The gain varies with how many draft tokens the target accepts.
We also worked with Geometric, which helps ML teams find and ship faster GPU kernels. For Lucebox, DeanoC optimized the HIP routing path with dedicated ROCm TOP_K and ARGSORT kernels. In our A/B test, they made expert selection 3.5–7× faster in the kernel microbench and improved end-to-end decode by 0.44%. That work was measured separately and is not part of the public 32 tok/s main-branch run.
On main, the q=4 ROCmFP4 path now decodes each packed dense weight once and applies it to all four verification columns instead of unpacking it four times. That change added 2.1–2.3%. The ROCmFPX format, fused decode path, DSpark verifier, sparse prefill, and q=4 weight reuse are all included in the reproduction below; no integration branch or private patch is required.
Test setup and quality checks
The LocalMaxxing request used 2,048 prompt tokens, 510 output tokens, an 8,192-token context, temperature zero, and batch one. We also reran five GSM and five MATH prompts after the kernel changes; all ten completed with the expected answer. That is a regression check, not an accuracy benchmark.
The 32 tok/s profile uses --ds4-expert-top-k 4 instead of the model default of six experts. This changes model execution and trades some quality margin for speed. Before using it for a specific workload, compare it with six-expert autoregressive output on that workload.
Sparse prefill: roughly 250 tok/s
The public LocalMaxxing request reports 245 tok/s prefill with --ds4-prefill sparse. In a separate 7,960-token validation, indexed sparse prefill reached 251.79 tok/s; the 8K cases ranged from 246.8 to 255.9 tok/s. At roughly 24K tokens, throughput was 221.9 tok/s.
Sparse prefill uses DeepSeek V4’s learned indexer to limit compressed-history attention. It also batches work layer by layer, which changes floating-point reduction order. The output is not byte-identical to tokenwise exact prefill, so sparse mode remains opt-in. It scored 10/10 on our small GSM8K set and 3/3 on a HumanEval smoke set; we have not run a broad quality evaluation yet.
For reference, tokenwise exact prefill measured 22.5–23 tok/s on our short GSM/MATH prompts and 16.46 tok/s on the paired 7,960-token test. Those figures use a different execution path. Use the roughly 250 tok/s result only for indexed sparse prefill.
Reproducing the run
Starting from a 128 GB Strix Halo machine with ROCm 7.2.4 already installed:
sudo apt-get update
sudo apt-get install -y build-essential cmake git ninja-build curl \
hipblas-dev hipcub-dev rocblas-dev rocprim-dev rocwmma-dev
git clone --branch main --recurse-submodules \
https://github.com/Luce-Org/lucebox.git
cd lucebox
cmake -S server -B server/build-hip -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_HIP_COMPILER=/opt/rocm/lib/llvm/bin/clang++ \
-DDFLASH27B_GPU_BACKEND=hip \
-DDFLASH27B_HIP_ARCHITECTURES=gfx1151 \
-DDFLASH27B_HIP_SM80_EQUIV=ON \
-DCMAKE_HIP_FLAGS=-DDFLASH_WAVE_SIZE=32 \
-DGGML_HIP_MMQ_MFMA=ON \
-DGGML_HIP_NO_VMM=ON \
-DGGML_HIP_GRAPHS=OFF
cmake --build server/build-hip --target dflash_server -j"$(nproc)"Download the ROCmFPX target and DSpark draft, then start the measured profile:
mkdir -p models
curl -L -C - --retry 5 \
-o models/DeepSeek-V4-Flash-ROCMFP2-STRIX.gguf \
"https://huggingface.co/Lucebox/DeepSeek-V4-Flash-ROCMFPX/resolve/main/DeepSeek-V4-Flash-ROCMFP2-STRIX.gguf"
curl -L -C - --retry 5 \
-o models/DeepSeek-V4-Flash-DSpark-draft-Q4RMFP4-denseF16.gguf \
"https://huggingface.co/Lucebox/DeepSeek-V4-Flash-DSpark-Drafter-GGUF/resolve/main/DeepSeek-V4-Flash-DSpark-draft-Q4RMFP4-denseF16.gguf"
MODEL="$PWD/models/DeepSeek-V4-Flash-ROCMFP2-STRIX.gguf"
DRAFT="$PWD/models/DeepSeek-V4-Flash-DSpark-draft-Q4RMFP4-denseF16.gguf"
echo performance | sudo tee /sys/firmware/acpi/platform_profile
sudo /opt/rocm/bin/rocm-smi -d 0 --setperflevel high
printf '0\n' > /tmp/ds4_awidth
printf '4\n' > /tmp/ds4_spec_q
DFLASH_DS4_SPEC=1 \
DFLASH_DS4_FUSED_VERIFY=1 \
DFLASH_DS4_SPEC_Q=4 \
DFLASH_DS4_TIMING=1 \
DFLASH_DS4_DRAFT="$DRAFT" \
LUCE_MMVQ_MAX_NCOLS=4 \
./server/build-hip/dflash_server "$MODEL" \
--target-device hip:0 \
--host 127.0.0.1 --port 8000 \
--max-ctx 8192 --default-max-tokens 2048 \
--chunk 2048 --ds4-prefill sparse \
--ds4-fused-decode \
--ds4-expert-top-k 4 \
--prefix-cache-slots 0 --prefill-cache-slots 0 \
--disk-prefix-cache offWarm the model once and use temperature: 0. The server prints decode speed on its [deepseek4] DSpark decode line. DFLASH_DS4_SPEC_Q=4 sets the DS4 verification cap; --verify-width is a Laguna option and is not used here. The implementation may shorten a batch at a compressor boundary, which is required for correct state handling.
The two clock commands are also part of the measured setup. On our validation machine, Radeon high held the graphics clock at 2.9 GHz; automatic clocks can cost several tokens per second. The published request reported 245 tok/s sparse prefill and 32.0 tok/s decode. Throughput varies with prompt shape and, for decode, how many DSpark proposals the target accepts. If you switch to exact prefill or restore the model’s six experts, those numbers no longer apply. No integration branch or private patch is required.
Measured July 2026 on a Ryzen AI MAX+ 395 / Radeon 8060S (gfx1151), 128 GB LPDDR5X, ROCm 7.2.4, platform profile performance, and Radeon performance level high (2.9 GHz observed). Target: DeepSeek-V4-Flash-ROCMFP2-STRIX.gguf. Draft: DeepSeek-V4-Flash-DSpark-draft-Q4RMFP4-denseF16.gguf. Public LocalMaxxing entry: 32.0 tok/s output and 245 tok/s sparse prefill, 2,048 prompt tokens, 510 output tokens, 8,192 context, batch one, temperature zero. Community submissions use different protocols; comparisons in this article are explicitly limited to published decode throughput.