Blog
Engineering notes on custom kernels, local inference, and hardware design.
Lucebox × Geometric: DeepSeek V4 Flash 0731 reaches 32.7 tok/s
On AMD Strix Halo, the 98.29 GB model scores 82/92 and reaches 32.7 tok/s with DSpark. Across our fixed HumanEval, GSM8K, and MATH evaluation, it averages 27.9 tok/s.
Tool prefix caching: 48× faster warm prefill for agent loops
Agent turns often resend thousands of tokens of tool definitions. Lucebox can now restore that stable prefix and process only the new conversation. On Qwen3.6-27B, median warm prefill was 1.04 seconds after a 50.35 second cold turn.
Lucebox (AMD Radeon AI PRO R9700 + Strix Halo) Beats NVIDIA DGX Spark by 3.63x on DeepSeek V4 Flash Decode Speed
51.1 tok/s on Lucebox versus our 14.09 tok/s average on one NVIDIA DGX Spark for the full 284B model. The complete $6,499 Lucebox costs 31% less than two DGX Sparks.
DeepSeek V4 Flash: 284B model, up to 32 tok/s on AMD Ryzen AI MAX+ 395
The full DeepSeek V4 Flash target runs locally on AMD Ryzen AI MAX+ 395 with 128 GB unified memory, reaching up to 32 tok/s decode and roughly 250 tok/s indexed sparse prefill.
Laguna XS 2.1 on a RTX 3090: 296 tok/s peak, 152 tok/s at 256K context
poolside's coding MoE with its official DFlash drafter: 296 tok/s peak at short context, a flat 152 tok/s at 256K tokens, prefill at 3,500 tok/s. Lossless speculative decoding plus KVFlash paging plus two new model-agnostic engine optimizations, on one 24 GB card.
Luce KVFlash: 256K context with 72 MiB of KV on the GPU
On Qwen3.6-27B the KV cache costs 4.6 GiB at 256K and drags decode to 13 tok/s. KVFlash pages cold 64-token chunks to host RAM bit-exact: decode holds a flat 38.6 tok/s from 64K to 256K on a 3090, accuracy unchanged. One flag, every model family.
Lucebox in a container: one image for every supported GPU
A prebuilt image spanning the RTX 2080 Ti to the RTX 5090 (sm_75 to sm_120, CUDA 12.8). The ~25-minute fat-binary compile happens once in CI, not on your box. Two host deps (docker + nvidia-smi), self-tuning, provenance at /props. docker run --gpus all.
Luce Spark: fit Qwen3.6 35B and Laguna XS.2 on a 16 GB GPU
A 33-35B MoE fires ~8 of 256 experts per token but pays for all of them in VRAM. Spark keeps resident only the experts traffic uses and swaps the rest: Qwen3.6 35B-A3B in 13.3 GiB, Laguna XS.2 in 14.6 GiB, self-tuning, one flag.
Gemma 4 26B edges out DeepSeek V4 Flash (284B) on ds4-eval-92, at 5x the speed
ds4-eval-92 head-to-head: Gemma 4 26B (26B, 4-bit) on a 24 GB RTX 5090 Laptop ties DeepSeek V4 Flash (284B, ~2-bit) on a 192 GB Mac at 78.3%, and decodes about 5x faster.
Launch and tune Lucebox with real agent harnesses
Real-client profiles, launch scripts, and TQ3/DDTree results for OpenCode, Hermes, OpenClaw, Open WebUI, Codex, Claude Code, and Pi.
Laguna XS.2 on a 3090: 111 tok/s, 5.4x prefill, first MoE target for PFlash
Poolside Laguna XS.2 (33B-A3B) ported into dflash + PFlash in ten days as the first MoE target supported by PFlash. ~107 tok/s decode at short context, 15.91 s TTFT at 128K on a single RTX 3090, 5.4x faster prefill than llama.cpp.
DFlash + PFlash on AMD Strix Halo: 2.5× end-to-end vs llama.cpp HIP
PR #119 lands DFlash + PFlash on the Ryzen AI MAX+ 395 iGPU (gfx1151, 128 GiB unified). Qwen3.6-27B Q4_K_M: 26.85 tok/s DFlash decode (2.23×), 20.2 s PFlash prefill at 16K (3.05×), 2.51× end-to-end at 16K + 1K gen vs vanilla llama.cpp HIP on the same silicon.
PFlash: 10× prefill speedup over llama.cpp at 128K on a RTX 3090
Long context overwhelms Q4 27B targets on 24 GB GPUs. PFlash compresses 128K → 2.6K with a small drafter before dflash sees the prompt. Head-to-head cold-vs-cold: 24.8 s TTFT vs ~257 s llama.cpp (10.4×); NIAH retrieval preserved at every measured context.
DFlash on ggml: up to 207 tok/s Qwen3.5-27B on a RTX 3090
Standalone C++/ggml speculative decoder for Qwen3.5-27B Q4_K_M with DFlash block-diffusion draft + DDtree verifier. 3.43x AR, 2.8x SGLang AWQ, 128K context on 24 GB.
The eGPU Myth: Why a ~$300 Dock Won't Turn Your GPU Into an AI Workstation
tinygrad wrote an NVIDIA driver from scratch. We ran real models on an RTX 3090 over USB4. The engineering is brilliant. The numbers aren't there yet. Full benchmarks and profiling.
Megakernel: Matching Apple Silicon Efficiency at 2x the Throughput on a RTX 3090
The first megakernel for hybrid DeltaNet/Attention LLMs. All 24 layers fused into a single CUDA dispatch. 1.87 tok/J, matching M5 Max efficiency at 1.8x the throughput on a 2020 GPU.