Best LLM for a single DGX Spark: Qwen3.8 Flash Next DeepSWE benchmark results

On DeepSWE it solves more tasks than DeepSeek V4 Flash and runs four of them at once on a single 128 GB Spark. Results from all 113 tasks.

October 8, 2026

Why this model

Artificial Analysis scores Qwen3.8 Flash Next at 40 on its Intelligence Index. The DeepSeek V4 Flash 0731 build I used before scores 34. Every open model that scores higher is 320B+ and needs two or more Sparks at a reasonable quantization. Qwen fits in 99 GB at 4-bit, leaving room for the full 262K context and several concurrent requests.

Setup

• One NVIDIA DGX Spark (GB10, 128 GB).
• MiaAI-Lab single-Spark vLLM recipe: NVFP4 weights, MTP speculative decoding (3 tokens), FP8 KV cache, 262K context, PLE table offloaded to CPU memory.
• chat_template_kwargs: {enable_thinking: true, reasoning_effort: "xhigh"}. Qwen’s template accepts only low, medium and xhigh; max returns HTTP 400.
• Temperature 1.0, top_p 0.95, top_k 20.
• mini-swe-agent via the datacurve pier runner, stock prompt.

A bug in the recipe

My first pilot solved 2 of 6 tasks. The serving recipe was at fault. It streams Qwen’s PLE (per-layer n-gram embedding) table from CPU memory, and a buffer-width bug sent stale or zero rows to the GPU on every forward pass longer than one token, which covers every prefill chunk and speculative-decoding verify step. Upstream fixed it on Sep 25, measuring mean NLL 0.86 → 0.71 and HumanEval 155 → 160 of 164. My checkout predated the fix. After pulling and restarting, I discarded all earlier Qwen results.

Stock vs. custom prompt

My DeepSeek runs used a custom prompt with extra constraints and step-budget reminders. With Qwen, 2 of the first 6 tasks ran out of context. The stock mini-swe-agent prompt used less context on the same tasks. I changed the prompt and fixed the server at the same time, so I can’t attribute the improvement to either one alone.

Running four tasks at once

DeepSeek’s recipe can serve concurrent requests, but its KV pool is split between them: two requests get about 169K context each, well below the 384K a long agent run needs. Qwen3.8 Flash Next is a hybrid model. Most of its layers keep a fixed-size recurrent state rather than a per-token KV cache, so its 1.1M-token pool holds about 4.2 full 262K contexts. The prefix cache keeps each agent’s history between turns (84–98% hit rate).

I added a CONCURRENCY option to the runner. Total generation speed on the server went from about 45 tok/s with one worker to 58 with two and 81 with three. With four workers the full run completed 98 tasks in about 40 hours, roughly 2.4 per hour. DeepSeek’s 113 tasks add up to about 160 hours run one at a time, more than six days.

Per-task time barely changed, because agents spend much of their time in builds and tests rather than generation. Across all 113 tasks, Qwen averaged 95 minutes per task and DeepSeek 85. Docker Desktop needed 8 CPUs and 20 GB. The runner prunes old task images when Docker’s disk runs low; with four workers, the first version of that cleanup deleted another worker’s freshly built image, so it now only removes unused DeepSWE images older than two hours.

DeepSWE results: Qwen3.8 Flash Next vs DeepSeek V4 Flash

15-task pilot, stratified by language and DeepSeek outcome: Qwen 8/15, DeepSeek 7/15.

Full run, all 113 tasks (every task, with tests passed, steps, time and context):

Many of Qwen’s wins are tasks DeepSeek missed by one or two tests (actionlint 54/55, bandit 87/88, dasel 144/146, pebble 58/59). Qwen’s margin is widest on Go and Python. On TypeScript the two are even, because that is where Qwen runs out of context.

Context limit

20 of 113 tasks hit the 262K limit, 9 of them in TypeScript. One repository, koota, accounts for four of its five tasks. Three of the 20 still passed, because their edits were complete before the limit was reached. The other 17 scored zero, since the agent never finished. DeepSeek, with 384K, solved 7 of those 17. Another 13 tasks finished above 84% of the window. Context is now the main reason Qwen loses tasks that DeepSeek solves. YaRN can extend the window to 524K, but it applies to every request and halves concurrency, so I’ll test it separately rather than change the running setup.

Caveats

• Each task is scored once at temperature 1.0. One task flipped between solved and unsolved across Qwen runs.
• The DeepSeek numbers come from its completed run with the custom prompt at max reasoning. This compares two configurations, not two models under identical settings.
• One task (gql-incremental-graphql-delivery) failed before the agent started, because a disk cleanup deleted its Docker image, and was re-run. On the re-run it passed all 17 target tests but broke one existing test, which scores zero.
• The pilot scored 53% and the full run 54.9%. Qwen’s published DeepSWE score is 58.7. DeepSeek V4 Flash scored 45.1% over all 113 tasks in my earlier run.

Next

• Per-task results are on their own page, as for the DeepSeek run.
• Rerun the 20 context-limited tasks, plus a control sample, with YaRN at 524K.
• Look at the TensorFold recipe. Its 1.0 release is 25–35% faster than vLLM here, but it serves a pruned 125B checkpoint with 5 experts per token, which its authors measure at about 10% lower general capability. Its earlier 0.6 release keeps the full model.