August 28, 2026

By Davide Ciffa

Ling 3.0 Flash on DGX Spark: Up to 141.9 tok/s with Adaptive DSpark and FlashKDA

We got AntLing’s 124B model running on one NVIDIA DGX Spark. The result is mixed: Lucebox reads prompts faster than llama.cpp, while normal token-by-token generation is about tied. Our speculative path reaches 141.9 tok/s in a favorable test, but that is not the speed every prompt will see.

What we found

• Prompt reading is the clearest win. Lucebox is 36.4% faster on the shortest matched prompt. With FlashKDA enabled, it stays ahead from 4K to 64K tokens: the lead starts at 14.0% and narrows to 5.7%.
• Normal generation is essentially tied. Lucebox reaches 46.0 tok/s and llama.cpp reaches 45.38 tok/s on the short test. Through 16K context, the difference remains below 0.5%.
• Speculative decoding can go much faster when its guesses are good. Our best matched case reaches 141.9 tok/s, 3.3% ahead of llama.cpp on the same 128-token output. On the saved ten-problem coding set, the more representative full-engine result is 74.90 tok/s.
• FlashKDA is roughly four times faster by itself. The kernel is 3.82× to 4.55× faster than our old KDA kernel. That does not make the whole server four times faster because the model still spends most of its time elsewhere.
• There is still work to do. Two simultaneous requests barely improve total throughput, the cross-engine HumanEval comparison is not ready, and the optional expert-skipping mode changes model execution and remains off by default.

What we built

Ling 3.0 Flash has 124 billion parameters, but it uses about 5.1 billion for each token. This design is called a mixture of experts: the model has many specialist blocks and selects only a few at a time. Our Q4_K_M build uses 72.59 GiB when loaded, so it fits comfortably in the DGX Spark’s 128 GB of unified memory.

We added native model loading, ordinary token-by-token generation, and support for AntLing’s official DSpark helper model. We also added NVIDIA-specific paths for prompt processing and expert layers. The implementation and benchmark scripts are under review in Lucebox Engine PR #661.

Watch it run

This replay turns the strict autoregressive control into one simple terminal race. Both sides read the same 83-token prompt and write the same 128-token response. Lucebox finishes first because FlashKDA shortens prompt processing; normal token-by-token generation is nearly even.

Ordinary generation: effectively tied

This is the plain token-by-token test, with all shortcuts turned off. Both engines use the same model file, cache format, prompt template, sampling settings, and one active request. Prompt caching is disabled. This is the fairest way to compare ordinary generation.

Lucebox is ahead by 1.36%, which we consider a tie in practice. An earlier build produced a faster single result, but we use the newer five-run median instead. For the complete request, Lucebox spends 204.9 ms reading the prompt and 2,780.1 ms writing the answer. Llama.cpp takes 322.8 ms and 2,798.3 ms. That is 2,985.0 ms versus 3,121.1 ms in total. Lucebox finishes 136.1 ms sooner, almost entirely because it reads the prompt faster.

Prefill: +36.4% early, then ahead through 64K

Next we measured prompt reading on its own. We sent each engine the same generated prompts, asked for one output token, disabled prompt caching, and kept one request active at a time. Each point is the middle result from three requests. Both engines returned the same one-token output.

The first version of our prompt path loses most of its lead by 4K. We then integrated FlashKDA and repeated the test with longer prompts. The updated path is 14.0% ahead at 4K and still 5.7% ahead at 64K. The full long-context chart appears below.

Adaptive DSpark: up to 141.9 tok/s

DSpark is a small helper model. It proposes several future tokens at once, and the main model checks them. When most guesses are accepted, generation can be much faster. Lucebox changes how many guesses it checks based on how well the recent guesses worked.

This is our highest measured generation speed. Lucebox is 3.3% faster, and both engines produce exactly the same 128 tokens. But this prompt is unusually easy for the helper model: 94.1% of its guesses are accepted. Treat 141.9 tok/s as a best case, not an average.

Why changing the block size helps

Lucebox starts cautiously. After two fully accepted blocks, it checks the largest block. If guesses begin to fail, it quickly reduces the block size; after clean blocks, it tries a larger one again. The review branch lets users disable this behavior or force a fixed size for testing.

A small coding test: changing versus fixed block size

The first number starts counting after the first token arrives. The second includes the complete server-side generation loop. Changing the block size improves them by 6.1% and 2.7%. Each setting produces 567 tokens per repetition and passes the same cases.

This saved set has ten problems, not the full 164-problem HumanEval suite. The 10/10 result is a local regression check, not an official HumanEval score.

Fixed-size checks

A verifier optimization raises the fixed-size result from 47.15 to 50.75 tok/s, a 7.6% gain. Different block sizes do not always produce identical text, even though all saved coding cases still pass. We therefore report this as a serving-speed result, not as proof that speculative output always matches ordinary generation byte for byte.

A real tool call

We also tested the model through Lucebox’s OpenAI-compatible API. We gave it a get_weather(location) function and asked for the weather in Rome. Ling selected the function, sent valid JSON arguments, read the result, and used it in its final answer.

This shows that the basic tool-call loop works on the real model. It does not show that Ling can reliably choose among many tools or recover from broken calls; we tested one controlled function. The complete record is in the DGX Spark integration report.

Optional expert skipping: 10.5% faster, but approximate

Ling normally uses eight specialist blocks for each token. Our optional mode skips the specialists that contribute the least. This saves work, but it also changes the model’s calculation. It is separate from DSpark, disabled by default, and never included in the exact-serving results above.

The selected 75% setting reaches 50.3 tok/s, compared with 45.5 tok/s when all eight specialists are used: +10.5%. Because this mode changes the calculation, we also checked it on small saved math sets.

The aggressive 50% setting falls from 17/20 to 14/20, so we reject it. The selected 75% setting reaches 18/20, but twenty local problems are far too few to claim equal quality. Expert skipping remains deterministic, optional, and off by default.

What profiling found

We recorded matched runs and used NVIDIA Nsight Systems to see where the GPU spent its time. This led to two changes that matter for long prompts.

First, Lucebox was converting the same attention cache twice. We made the value cache reuse data already stored in the key cache. Second, the general KDA path was doing bookkeeping needed for speculative trees even during ordinary generation. The DGX Spark now selects a simpler path when that bookkeeping is unnecessary.

After the changes, cache-conversion launches fall from 2,240 to 1,120, attention time falls from 132.1 to 62.9 ms, and average KDA time falls from 256.5 to 226.6 microseconds. Llama.cpp records 1,134 launches, 61.2 ms, and 217.8 microseconds in the same trace. Lucebox keeps its previous output at 4K, 8K, and 16K.

Why DSpark changes block size

The 141.9 tok/s run has 68 generation steps. The saved coding run has 117 steps, 107 of them speculative. Both traces are complete, and all ten saved coding cases pass under the local scoring setup.

Checking a large block costs roughly 40–50 ms, while producing the draft costs roughly 14–15 ms. A large block only pays off when acceptance is stable, which is why Lucebox waits for two clean blocks before trying the maximum size.

Why expert skipping helps

The matched traces cover all 128 generation rounds in both modes. They show that most of the gain comes from doing less setup work before the expert layers.

FlashKDA: faster prompt reading

Most of Ling’s layers use Kimi Delta Attention (KDA). Our old KDA kernel reads a prompt one token at a time, which leaves much of the GPU idle during this phase.

FlashKDA groups prompt tokens into 16-token chunks and prepares many chunks in parallel. A second step then carries the model state from one chunk to the next. This gives the GPU much more work it can do at the same time.

We compiled this path for the DGX Spark’s Blackwell GPU and connected it to the GGML graph. It is used while reading prompts; ordinary one-token generation keeps the existing exact path.

The official full-size FlashKDA test matches its Torch reference exactly. Our separate stress test compares a BF16 kernel with Lucebox’s older FP32 path, so the numbers are close but not identical:

Long prompts and multiple users

We repeated the prompt-reading test at 4K, 8K, 16K, 32K, and 64K tokens. Both engines use the same model, cache, prompt template, and one active request. Caching and speculation are disabled. Lucebox stays ahead at every measured length.

The prompt-reading lead shrinks because FlashKDA speeds up 35 KDA layers, while Ling’s seven conventional attention layers become more expensive as the prompt grows. Those untouched layers take a larger share of the total time. The lead does not disappear in the measured range: it is 6.2% at 16K and 5.7% at 64K.

Handling more than one request is a clear weakness today. Two simultaneous requests mostly run one after the other instead of sharing the GPU efficiently.

We measured prompt reading through 64K and ordinary generation through 16K. We have not yet tested the model’s full 262K limit or four and more simultaneous requests, so we do not claim results beyond those points.

What else we tested

Speed is only part of the story. We also ran saved coding problems, one complete tool call, and a real streamed response. The table separates what worked from what we have not tested yet.

We have not yet run a full repository-level coding task through an agent framework. That would be a useful follow-up, but we do not claim it here.

Hardware and run configuration

Exact test setup and commands

This section is for readers who want to rerun the numbers. The benchmark scripts record the machine setup, hash the output, and refuse to start if another inference server could interfere. The ordinary-generation script gives both engines the same model, cache settings, and number of active requests.

export LING_SERVER_BINARY=/path/to/dflash_server
export LLAMA_SERVER_BINARY=/path/to/llama-server
export LING_BENCH_MAX_CTX=4096
export LING_BENCH_PROMPT_PROFILE=official-longform
export LING_BENCH_DECODE_WORKLOAD=longform
export LING_BENCH_SECTIONS=decode
export LING_BENCH_DECODE_TOKENS=128
export LING_BENCH_DECODE_RUNS=5
export LING_BENCH_WARMUPS=2

server/scripts/bench_ling3_ar.sh \
  luce /path/to/Ling-3.0-flash-Q4_K_M.gguf luce-ar.json

server/scripts/bench_ling3_ar.sh \
  llama /path/to/Ling-3.0-flash-Q4_K_M.gguf llama-ar.json

export LING_BENCH_MAX_CTX=8192
export LING_BENCH_SECTIONS=context
export LING_BENCH_CONTEXT_TOKENS=128,1024,4096
export LING_BENCH_CONTEXT_RUNS=3

export LING_SERVER_BINARY=/path/to/dflash_server
export LING_BENCH_SUITES=he
export LING_BENCH_N_SAMPLE=10
export LING_BENCH_MAX_CTX=8192
export LING_DSPARK_ADAPTIVE_WIDTH=1
export LING_DSPARK_ADAPTIVE_MAX_WIDTH=9

server/scripts/bench_ling3_heldout.sh \
  luce-dspark \
  /path/to/Ling-3.0-flash-Q4_K_M.gguf \
  /path/to/Ling-3.0-flash-dspark-BF16.gguf \
  adaptive-humaneval.json

Links

• Lucebox Engine PR #661 — model support, DGX Spark optimizations, benchmark scripts, and tests.
• inclusionAI/Ling-3.0-flash — target weights and official model card.
• inclusionAI/Ling-3.0-flash-dspark — official DSpark helper model.
• sudoingX/dgx-spark-ling — upstream DGX Spark serving recipe.
• vllm-project/FlashKDA — the prompt-reading kernel used by our DGX Spark path.

Verdict

Ling 3.0 Flash runs well on one DGX Spark, but the result depends on the workload. Lucebox’s clear win is prompt reading: it is ahead at every measured point through 64K. Ordinary generation is about tied with llama.cpp. Speculative generation reaches 141.9 tok/s in the best matched case and 74.90 tok/s across the complete server loop on our small saved coding set.

Run it if you want a large local model for coding, tool experiments, or GPU work on a compact NVIDIA box. If you need high throughput from many simultaneous users or identical output across different runtimes, a more mature serving stack is the better choice today.

We measured every Lucebox and llama.cpp number on the same DGX Spark from August 24–28, 2026. Percentages use the original unrounded results. We label full-model, after-first-token, approximate, and kernel-only numbers separately. We removed the invalid cross-engine HumanEval comparison, and we do not reuse vendor benchmark numbers as our own.