August 2026

By Lucebox, with Geometric

Lucebox × Geometric: DeepSeek V4 Flash 0731 reaches 32.7 tok/s on AMD Strix Halo

Lucebox Engine now serves DeepSeek V4 Flash 0731 from one 98.29 GB GGUF on a 128 GB AMD Strix Halo system. The artifact scores 82/92 on ds4-eval-92 and reaches 32.7 tok/s with DSpark. Across our fixed HumanEval, GSM8K, and MATH evaluation, it averages 27.9 tok/s. At 60K context, sparse prefill reaches 173.0 tok/s and DSpark decode reaches 18.0 tok/s.

We added support for DeepSeek-V4-Flash-0731 to Lucebox Engine together with Geometric. The collaboration covers the adaptive ROCmFPX formats, HIP and CUDA kernels, embedded codebook loading, runtime validation, and the monolithic device load needed by this artifact. Geometric PR #28 records the adaptive-quantization contribution; Lucebox PR #593 carries that work into Lucebox Engine.

Geometric builds tools that automatically discover and verify CUDA and Triton kernels, then ships the resulting speedups as GitHub pull requests. Their work is aimed at teams deploying machine learning across GPU fleets.

The supported target is available from Lucebox on Hugging Face. It is a single 98.29 GB GGUF for a 128 GB AMD Ryzen AI MAX+ 395 / Radeon 8060S configuration. Per-expert quantization data, codebooks, and runtime metadata are embedded in the same file.

Quality benchmark

The release validation, documented on the Lucebox model card, ran the complete 92-item ds4-eval suite with a 15,488-token thinking budget, matching the published reference. The supported artifact scored 82/92 while using 2.766 bits per weight instead of 2.88.

The suite combines COMPSEC-17, AIME 2025, GPQA Diamond, and SuperGPQA. COMPSEC scored 17/17. All 92 evaluation items were excluded from calibration, and the outputs were rescored with the reference grader.

How the model fits on AMD Strix Halo

The GGUF is 98.29 decimal GB, or about 91.54 GiB. That is below the 96 GiB single-allocation window available to the Radeon 8060S in our 128 GB Strix Halo system, so the full model can stay resident on one local device. The 2.766 bpw figure is all-in: it divides the complete file, including its header and embedded codebooks, by the artifact’s 284.33 billion parameters.

To reach that size, the mixed-precision layout described in PR #593 chooses each tensor’s format from measured layer-output damage under real mixture-of-experts routing. The routed experts use learned per-expert codebooks: 2.5 bits per weight for gate/up experts and selected down projections, and 3.5 bits per weight for the remaining down projections. Attention, dense weights, and the shared expert stay at 4.25 bits per weight or higher.

Six-expert performance on AMD Strix Halo

Every result in this article uses all six experts routed by the model for each token. In the published target-only measurement, the 98.29 GB target reaches 18.1 tok/s with fused decode disabled. This is also the routing configuration used for the 82/92 quality evaluation.

The DSpark evaluation below keeps the same six-expert routing and enables fused decode. Fused decode changes how the work is executed; it does not reduce the number of routed experts.

DSpark speed evaluation: HumanEval, GSM8K, and MATH

On August 11, we ran a fixed 30-prompt evaluation: ten prompts each from HumanEval, GSM8K, and MATH. The Radeon 8060S handled both the 98.29 GB target and the draft on Strix Halo. The run used exact prefill, native six-expert routing, fused decode, DSpark q=4, temperature zero, and batch size one.

All 30 requests completed. Per-request draft acceptance ranged from 67% to 96%, with 82.4% token-weighted acceptance overall.

The 29/30 score is after auditing the saved responses without rerunning inference. The audit corrected one GSM8K answer-extraction error and one incorrect MATH gold label; one MATH response remained wrong. Both evaluator fixes have regression tests.

32.7 tok/s at near-100% draft acceptance

In our high-acceptance DSpark run, the drafter predicted almost every token correctly. Acceptance stayed between 99% and 100%, and streaming decode reached 32.7 tok/s with all six routed experts active.

This is what the 32.7 tok/s result means: when the drafter is right almost every time, the target can accept nearly every proposed token. On the mixed HumanEval, GSM8K, and MATH evaluation, acceptance averaged 82.4% and server-side decode averaged 27.9 tok/s.

Long-context performance

We tested the same six-expert model with 23,943 and 59,933 input tokens.

DSpark improved decode by 57% at 24K and 25% at 60K. Sparse prefill remained at 173.0 tok/s for the 59,933-token prompt.

Both runs used the model-specific sparse decode path, sparse prefill, Q4 K/V cache, DSpark q=4, guarded high mode, the platform performance profile, and all six routed experts. No prompt state was reused between runs, and both completed the requested integer sequence correctly.

Lucebox Engine integration with Geometric

Geometric PR #28 contains the reviewable adaptive-quantization and serving contribution. Lucebox PR #593 brings that work into Lucebox Engine and records the corresponding Lucebox review and validation.

• CUDA and HIP kernels for the new 2.5-bit and 3.5-bit expert weights.
• Embedded codebook loading, so the supported model ships as one GGUF.
• Validation for missing, duplicated, truncated, or mismatched codebooks before serving starts.
• Automatic monolithic loading when the mixed formats are detected.
• A fix for the DeepSeek V4 thinking budget so generation continues into the visible answer.

The pull requests report a 2.2× speedup for the fused 3.5-bit matrix-vector kernel over the generic expand-then-multiply fallback, plus a separate 14.7% kernel-time gain from the sync-free MoE decode path. These are kernel measurements, not end-to-end token-rate multipliers. The Strix Halo serving results above were measured separately.

Run with Lucebox Engine

Download the target from Lucebox on Hugging Face. Use a Lucebox Engine build containing PR #593; the measured long-context profile also includes PR #599. A native six-expert target-only launch is:

dflash_server DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf \
  --target-device hip:0 \
  --max-ctx 8192 \
  --ds4-expert-top-k 6

Six experts per token is the model default; the explicit flag makes the configuration visible and reproducible. At startup, the server should report 101 registered Q2 mixed tensors, 28 registered Q3 mixed tensors, and 256 experts (6/6 routed).

Reproduce the six-expert speed evaluation

The fixed HumanEval, GSM8K, and MATH evaluation used the following equivalent launch profile. Download the draft from the Lucebox DSpark repository, then set MODEL and DRAFT to the downloaded files. On a system with both a discrete GPU and the Strix iGPU, expose only Strix with HIP_VISIBLE_DEVICES=<Strix device index> before launch; the selected device is then hip:0 inside the process.

MODEL=/path/to/DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf
DRAFT=/path/to/DeepSeek-V4-Flash-3107-DSpark-draft-Q4RMFP4-denseF16.gguf

DFLASH_DS4_SPEC=1 \
DFLASH_DS4_SPEC_Q=4 \
DFLASH_DS4_FUSED_VERIFY=1 \
DFLASH_DS4_DRAFT="$DRAFT" \
DFLASH_DS4_DRAFT_GPU=0 \
DFLASH_DS4_DRAFT_CONTEXT_KV_CACHE=1 \
DFLASH_DS4_TIMING=1 \
dflash_server "$MODEL" \
  --host 127.0.0.1 --port 18109 \
  --target-device hip:0 --max-ctx 8192 \
  --default-max-tokens 2048 \
  --prefix-cache-slots 0 --prefill-cache-slots 0 \
  --disk-prefix-cache off \
  --ds4-fused-decode --ds4-expert-top-k 6 \
  --ds4-prefill exact

Strix Halo uses automatic unified-memory allocation with stock memory settings. The 30-prompt evaluation used exact prefill; the long-context table used sparse prefill. A healthy load reports 101 qtype-106 tensors, 28 qtype-105 layers, 256 experts (6/6 routed), and DSpark enabled.

The measured runs loaded the current draft whose filename contains 3107. The runtime proves that exact file was used, but not that it was trained for the 0731 target. We therefore report the artifact name and do not describe this as a matched 0731 drafter.

Availability

Download the 98.29 GB target from Lucebox on Hugging Face. The DSpark helper is also available from Lucebox. The model integration is in PR #593, with the long-context serving update in PR #599.

Artifact details, quality, and target-only results are documented on the Lucebox model card. The adaptive formats and serving contribution are documented in Geometric PR #28; the Lucebox Engine integration and validation are in Lucebox PR #593. The 82/92 result uses the reference grader at a matched thinking budget with all 92 evaluation items held out of calibration. The DSpark measurements were recorded on August 11, 2026, on a Ryzen AI MAX+ 395 / Radeon 8060S with 128 GB unified memory. The 30-prompt evaluation is a fixed sample, not a full-dataset benchmark.

Related

• DeepSeek V4 Flash on AMD Ryzen AI MAX+ 395
• Asymmetric DeepSeek V4 expert parallelism on Lucebox
• Download the Lucebox target model
• Download the Lucebox DSpark helper
• Related integration work with Geometric
• Official DeepSeek V4 Flash 0731 model card
• Review the Lucebox implementation
• Review the long-context serving update