Can a local model on a DGX Spark match a frontier model? A DeepSWE case study

Running DeepSeek V4 Flash on a single NVIDIA DGX Spark against GPT-5.6 Terra and Claude Sonnet 4.6 on one hard coding task.

September 7, 2026

TL;DR

On a hard DeepSWE task (adding using / await using declarations to the Meriyah JavaScript parser), a local DeepSeek V4 Flash on a single DGX Spark fully solved the task: 49/49 tests, matching frontier GPT-5.6 Terra and ahead of Claude Sonnet 4.6 and a lower-effort tier of Terra. It only reached that score with reasoning set to max. With the default config it under-reasoned and lost by a single edge-case test.

The setup

• Hardware: one NVIDIA DGX Spark (GB10, 128 GB).
• Local model: DeepSeek V4 Flash 0731, served with the MiaAI-Lab EXL3 single-Spark recipe (EXL3 3.0-bpw, REAP-pruned to 216/256 experts, on a vLLM/sparkinfer container). Roughly Q4–Q5 quality in ~100 GB, ~384K context, OpenAI-compatible API.
• Hosted comparisons: GPT-5.6 Terra and Claude Sonnet 4.6, driven through the Cursor CLI agent.
• Benchmark: DeepSWE, a set of original, contamination-free, long-horizon software-engineering tasks graded by real test suites. I ran a single task (meriyah-explicit-resource-declaration, seed 0) so every model got the identical problem and the identical verifier.

Grading is binary: you score 1.0 only if every “fail-to-pass” test passes and no previously-passing test breaks. The task has 49 fail-to-pass tests and 51,469 pass-to-pass tests.

Results

Steps count model calls for the DeepSeek runs (mini-swe-agent) and tool calls for the Cursor runs (cursor-agent), so compare them within each agent, not across. Times are wall-clock. And the harness difference is itself a caveat: the local runs used mini-swe-agent while the hosted runs used cursor-agent, so this is a directional comparison, not a controlled one.

Two observations:

1. The local model matched the frontier on quality, at a large cost in time. DeepSeek V4 Flash reached the same 49/49 as GPT-5.6 Terra, but took 110 minutes to Terra’s 19. At max reasoning the deepest steps ran close to nine minutes each.
2. Reasoning effort decided the outcome. Terra at xhigh solved it; at high it missed a test and added 4 regressions. DeepSeek at max solved it; at its default it missed the single hardest edge case.

Reasoning config

The first local run lost because the reasoning controls never reached the model. On this vLLM/sparkinfer build, top-level reasoning_effort and the various think flags have no effect. What works is chat_template_kwargs:

{
  "extra_body": {
    "chat_template_kwargs": { "thinking": true, "reasoning_effort": "max" }
  }
}

The server also has to run with --max-model-len 393216. “Think Max” is gated behind that context size; below it, max effort does nothing. With the correct config, peak reasoning per step went from ~14.5K to ~66K characters, and the score went from 48/49 to 49/49.

Beyond one task

One task is thin. To check whether DeepSeek V4 Flash holds up more broadly, I ran three more DeepSWE tasks (a different seed, no overlap) under the same max-reasoning config:

Strict pass@1 was 2/4, with zero regressions on any task and an average fail-to-pass of ~98% — the two misses each failed a single test. Two of four solved lines up with DeepSeek V4 Flash’s published DeepSWE score of 54.4, so the EXL3 3-bpw quant on one Spark roughly reproduces the full-precision number on this small sample.

A smaller local model, same harness

I also ran Qwen3.8-27B (FP8, SGLang) under the same mini-swe-agent scaffold. It scored 0/49 — not a near-miss like Sonnet or the default runs, but zero passing target tests: it made edits, passed none of the 49, and grew its context past 220K without converging until the agent errored out. The harness matched the DeepSeek solve, thinking was on, and the timeout budget was generous, so this was not a scaffold or timeout artifact.

Qwen3.8-27B and DeepSeek V4 Flash tie on the Artificial Analysis composite index (~52 each), which is what makes this surprising at first. On the DeepSWE component alone they do not tie: 42.2 vs 54.4. The composite parity comes from the non-agentic categories. At 42.2, Qwen fails most DeepSWE tasks, and this was a hard one.

A faster local model that still fell short

I also tested Qwen3.8-Flash-Next (NVFP4, ~99 GB, vLLM) — the local model with the best DeepSWE score on paper (58.7, above DeepSeek’s 54.4). It had the fastest prefill of the three local models: context stayed under 16% of the KV pool, it never stalled, and it finished cleanly in ~70 minutes with no timeout. But on Meriyah it passed only 38/49 tests — far better than Qwen3.8-27B’s 0, yet below every near-miss here (Sonnet 47, DeepSeek default 48) and well short of a solve.

Running it on the same three extra tasks changed that picture. It matched DeepSeek test-for-test — Awilix 23/24, Cliffy 36/37, sqlite-utils 60/60 (solved) — identical fail-to-pass counts, including the same solve and the same two one-test-short misses. So across all four tasks the two are even except on Meriyah, where DeepSeek solved and Flash-Next did not:

The single-task result oversold the gap. On most tasks the two were interchangeable; DeepSeek’s edge came down to one hard task.

Takeaway

A single DGX Spark running a quantized open-weight model can reach the same result as a frontier hosted model on a hard, contamination-free coding task, provided you give it the reasoning budget it can use. The cost is time: 110 minutes here against 19 for Terra. Model choice still matters, and leaderboard rank does not settle it: DeepSeek V4 Flash solved the task, while Qwen3.8-Flash-Next — which ranks higher on DeepSWE — only reached 38/49, and Qwen3.8-27B scored 0.