A context setting is not a context test
A server can accept a large context configuration without demonstrating useful work at that length. I separated configured capacity, an actual request, and a successful result before making another speed claim.
Key finding
The largest verified successful coding prompt was 49,124 tokens; larger configured windows remained unproven.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
The table joins historical receipts from different runtimes and concurrency conditions; it is not a controlled quantization leaderboard.
Data
Reviewed aggregate results with explicit units, sources and qualification limits. Different cohorts remain separate.
Two draft tokens led this fixed-output sweep
One GPU · three fixed 512-output-token requests · configured context 8192
- GPTQ INT4 · drafting off45.30
- GPTQ INT4 · MTP265.70
- GPTQ INT4 · MTP361.60
- CyanKiwi · base38.90
tokens / second
Decode-only sweep. Coding-task outcomes and context capacity require separate evidence.
View data & source
| Configuration | Value |
|---|---|
| GPTQ INT4 · drafting off | 45.30 |
| GPTQ INT4 · MTP2 | 65.70 |
| GPTQ INT4 · MTP3 | 61.60 |
| CyanKiwi · base | 38.90 |
Origin: reviewed report table. Revision 7e6fdbe3b1d1.
Write-ups
Report qwen38/context-speed-audit-2026-10-06/README.md
Why audit the receipts?
A server can accept a large context configuration without demonstrating useful work at that length. I separated configured capacity, an actual request, and a successful result before making another speed claim.
The largest successful coding receipt contained 49,124 prompt tokens and 51,239 total tokens on the older GPTQ INT4 setup. Its configuration was 81,920. The newer HQ512 build was still in progress, so it supplied no new long-context result.
Speed depends on the workload
In a separate fixed-512-output-token, one-GPU sweep, GPTQ without drafting measured 45.3 tokens/s, MTP2 65.7, and MTP3 61.6. The CyanKiwi profile measured 38.9. These figures belong to that sweep; coding-request rates include prefill and varying concurrency.
An INT8 receipt generated 32,337 tokens but failed retrieval on a 7,758-token prompt. Its 260k setting cannot be counted as a successful context test. The 335k AWQ receipt explicitly marked long-context testing false.
The finding
Capacity, decode rate, retrieval and task completion need separate evidence. The useful next run is a fixed retrieval and coding protocol over increasing prompt lengths, with the same output budget and concurrency.
Evidence and publication boundary
This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.
More in Language models
An unfinished gauntlet cannot name a winner
This campaign compared Qwen UD-Q4_K_XL with MTP2 against Bonsai through coding harnesses. It stopped at the 70 GiB free-space guard, retaining the incomplete evidence rather than risking the workstation.
The 15 matched cells tied: six passes and nine failures for each model.
Finding a local model for the inbox
Inbox work needs accurate structured answers and reasonable response times. The benchmark ran the same 50 tasks across local model, runtime and hardware configurations, with thinking on and off, to make that tradeoff visible.
Bonsai-2-27B PQ2_0 with MTP passed 48/50 tasks with thinking on, at a 15.8-second median on an RTX 3060.
Balancing GPU experts when the model will not fit
The quant was larger than the three GPUs could hold on their own. The experiment tested explicit GPU expert splits with CPU offload and measured the decode effect under the same large-context workload.
An explicit 12/12/12 GPU expert split plus 12 CPU blocks measured 31.1 tokens per second.