翻訳待ち：Speed Up LLM Inference with DSpark Speculative Decoding

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

--> Speed Up LLM Inference with DSpark Speculative Decoding - KDnuggets --> Join Newsletter There are many ways to get more from the models and GPU infrastructure you already have. Quantization, optimized kernels, and better inference engines can all help, but speculative decoding is especially useful because it can increase generation speed without simply adding more GPUs. There are now several approaches to speculative decoding. Traditional methods use a smaller draft model, while Multi-Token Prediction (MTP) predicts several future tokens at once. Methods such as Medusa and EAGLE improve how those drafts are produced, while DFlash generates blocks of candidate tokens in parallel. DSpark takes another approach by combining parallel drafting with a lightweight sequential component. This helps later draft tokens use information from earlier predictions while keeping much of the speed advantage of parallel generation. In this guide, we will test DSpark with Qwen3-8B and llama.cpp. We will benchmark the model normally, enable DSpark with a matching draft model, and compare the generation speeds to see how much performance we can gain from the same GPU. How DSpark Works DeepSeek's DSpark improves the drafting part of speculative decoding. Parallel draft models can predict a whole block of tokens in one pass, which is fast, but later predictions can become less accurate because they do not fully depend on the tokens predicted earlier in the block. DSpark combines a parallel backbone with a lightweight sequential component, allowing later draft positions to incorporate information from earlier predicted tokens while retaining much of the speed of parallel generation. In simplified terms: DSpark can also estimate how likely draft tokens are to survive verification, allowing low-confidence parts of a block to be dropped instead of wasting verification compute. llama.cpp exposes this through its DSpark implementation and optional confidence threshold. DeepSeek reports that DSpark improved per-user generation speed by 60–85% compared with its previous MTP-1 production baseline when deployed with DeepSeek-V4. Those numbers should not be treated as expected results for our small local model, so we are going to measure the difference ourselves. 1. Building llama.cpp and Downloading the Models We will build the latest llama.cpp from source so we can use its current DSpark implementation with CUDA acceleration. Install the required tools: apt-get update apt-get install -y git cmake build-essential Clone the official llama.cpp repository: cd /workspace git clone https://github.com/ggml-org/llama.cpp Build it with CUDA support enabled: cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF \ -DGGML_CUDA=ON cmake --build llama.cpp/build \ --config Release \ -j \ --clean-first \ --target llama-cli llama-mtmd-cli llama-server llama-gguf-split This creates the binaries we need while allowing the models to run on the GPU. Next, create a directory for the model files: mkdir -p /workspace/models We will download the GGUF files manually using the Hugging Face CLI so the download time does not affect our benchmarks. Install the CLI: pip install -U huggingface_hub If your Hugging Face token is stored in HF_TOKEN, authenticate with: hf auth login --token "$HF_TOKEN" Download the Qwen3-8B Q4_K_M target model: hf download \ Qwen/Qwen3-8B-GGUF \ Qwen3-8B-Q4_K_M.gguf \ --local-dir /workspace/models Then download the matching DSpark Q8_0 draft model: hf download \ ggml-org/Qwen3-8B-GGUF \ dspark-Qwen3-8B-Q8_0.gguf \ --local-dir /workspace/models The first file is the main model that generates the final output. The smaller DSpark model will generate speculative draft tokens for the target model to verify. Confirm that both files are available: ls -lh /workspace/models You should see something similar to: 4.7G Qwen3-8B-Q4_K_M.gguf 1.2G dspark-Qwen3-8B-Q8_0.gguf With llama.cpp built and both models downloaded, we can first measure the normal Qwen3-8B generation speed before enabling speculative decoding. 2. Measuring the Baseline Speed Before enabling DSpark, we need a baseline. We will run Qwen3-8B normally and record its generation speed so we can compare it against the speculative-decoding run. Move into the llama.cpp directory: cd /workspace/llama.cpp Run Qwen3-8B without speculative decoding: ./build/bin/llama-cli \ -m /workspace/models/Qwen3-8B-Q4_K_M.gguf \ -ngl all \ -fa on \ --temp 0 \ --top-k 1 \ -n 512 \ -st \ -p "Write a complete Python implementation of merge sort. Explain how it works and include its time and space complexity. /no_think" Here, -ngl all offloads all model layers to the GPU, while -fa on enables Flash Attention. We also use deterministic decoding: --temp 0 --top-k 1 This is important because we will use the same prompt, token limit, and decoding settings when testing DSpark, giving us a cleaner apples-to-apples comparison. When generation finishes, look for the benchmark summary printed by llama.cpp: [ Prompt: 294.6 t/s | Generation: 95.0 t/s ] For this guide, the important number is Generation: 95.0 tokens/s. We will use this as our baseline when measuring the DSpark speedup. 3. Running the Same Test With DSpark Now we will repeat the benchmark with DSpark enabled. The goal is to keep the target model, prompt, token limit, and decoding settings the same so we can directly measure the effect of speculative decoding. Run the same Qwen3-8B model, this time with the DSpark draft model attached: ./build/bin/llama-cli \ -m /workspace/models/Qwen3-8B-Q4_K_M.gguf \ -md /workspace/models/dspark-Qwen3-8B-Q8_0.gguf \ --spec-type draft-dspark \ --spec-draft-n-max 3 \ -ngl all \ -ngld all \ -fa on \ --temp 0 \ --top-k 1 \ -n 512 \ -st \ -p "Write a complete Python implementation of merge sort. Explain how it works and include its time and space complexity. /no_think" Here, -md loads the DSpark draft model, while --spec-type draft-dspark enables DSpark speculative decoding. --spec-draft-n-max 3 allows DSpark to draft up to three tokens at a time, and -ngld all offloads the draft model to the GPU. When the run finishes, record the generation speed: [ Prompt: 88.0 t/s | Generation: 124.9 t/s ] Now compare it with our baseline: Configuration Prompt Speed Generation Speed Qwen3-8B baseline 294.6 t/s 95.0 t/s Qwen3-8B + DSpark 88.0 t/s 124.9 t/s DSpark increases generation throughput from 95.0 to 124.9 tokens/s. That is about a 1.31× speedup, or roughly 31.5% faster generation, using the same target model and GPU. The prompt-processing speed is lower in the DSpark run, but the main benefit we are measuring is autoregressive generation speed. For workloads that generate longer responses, the higher token-generation throughput can have a much larger impact on overall inference time. Final Thoughts For local LLM acceleration, I still think MTP is often the more practical option, especially because it is simpler and available across a wider range of models. However, DSpark can have an edge over basic multi-token prediction in cases where better draft quality leads to more accepted speculative tokens. The good thing is that DSpark is very easy to set up in llama.cpp. The bigger limitation is model support: only a small number of models currently have compatible DSpark draft models available. Support in llama.cpp is also still relatively new, so you may run into bugs or instability depending on the model and build you are using. For now, DSpark is an interesting acceleration technique to experiment with, but MTP remains the more broadly useful option for local inference. Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in technology management and a bachelor's degree in telecommunication engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness. Our Top 5 Free Course Recommendations --> Latest Posts Speed Up LLM Inference with DSpark Speculative Decoding 7 Python Mistakes Beginners Make (And What to Do Instead) The Local AI Stack for Productive SLMs Quantization and Pruning Methods to Make Your LLM Leaner What We Can Learn From Google Engineers’ Indispensible Prompts Understanding the Impact of AI on Job Markets Top Posts The Local AI Stack for Productive SLMs How to Leverage Local Small Language Models for Your Projects How to Build a Career in AI: 3 Distinct Pathways 10 Rules for Getting Better Results from AI Coding Agents 5 Real-World Use Cases for AI Agents Transforming Industries Building an End-to-End Data Science Portfolio Project Top 10 Open-Source Benchmarks for AI Coding Agents in 2026 Python Data Classes Beyond the Boilerplate What We Can Learn From Google Engineers’ Indispensible Prompts Run Qwen3.8-27B as a Local AI Coding Agent in Just 3 Commands Published on August 31, 2026 by No, thanks!