Qwen3.8-27B and its Swift-1.5 and Pi fine-tunes on a 24 GB RTX 3090: measured settings, speeds and energy

This page tells you how to run one model family — Qwen3.8-27B and two fine-tunes of it, Swift-1.5 and bytkim’s Qwen3.8-27B-pi — on one graphics card, a 24 GB RTX 3090, and what to expect when you do. Every setting it recommends was measured on that machine, and the measurement that decides each recommendation is printed beside it. It also carries settings for other cards, marked clearly as calculated rather than measured. It is not a review, and not a set of numbers you should expect to match exactly on different hardware. It does compare three files of one architecture side by side in one sweep (2026-10-04) — the anchor (unsloth UD-IQ4_XS of the base model), Swift IQ4_XS and Swift IQ3_XXS — and every difference it finds belongs to the file (fine-tune, quantizer and size together), not to Swift-1.5 as a model (§17). A fourth file, the Pi fine-tune’s IQ4_XS, was measured on 2026-10-06/07 against Swift IQ4_XS in its own sweep and against the 2026-10-04 suite cells (Appendix B). If all you want is a working configuration, go to §03 and copy the block for your card.

The limits, stated plainly: one machine, one model family, one operating system. Every measured row on this page is the same RTX 3090 on Windows 11 with driver 596.36. Nothing here has been checked on Linux, on AMD, on Apple silicon, or on a second copy of the same card. Where a number was calculated instead of measured, it says so in the cell.

Run Swift IQ4_XS for single-turn coding and reasoning tasks at xhigh. It generated 0.446 of the total tokens of the anchor (unsloth UD-IQ4_XS of the base model) on this page’s 175-prompt suite (greedy, xhigh, drafter off, single-turn, 2026-10-04) der — a total that leans on the anchor’s capped items: 0.52 over the 171 items both finished, median per-item ratio 0.87, fewer tokens on 139 of 175 items — with no quality difference larger than the detectable gap at 25 questions per set n=25, a smoke-test scale (§10): the 5-set composite is 82.7 for Swift IQ4_XS against 81.3 for the anchor meas, difference +1.49 der, paired 95% bootstrap interval −0.2 to +3.9 (10,000 resamples, seed 42) including zero. With the drafter off it decodes at the same speed at 1.5k depth (ratio 1.005 der), so derived time over the suite is 0.44 of the anchor’s der (about 0.86 for the typical item) and measured energy 0.47 der. For the four agents this page tested only on the base model (aider, OpenCode, Qwen Code, DeepSeek Harness), the high effort level and the whole August evidence base, stay on the anchor’s default card. For two users or the longest single-slot Swift window, take Swift IQ3_XXS, at the cost of a perplexity gap against Swift IQ4_XS on their shared tokenizer (§08); it is ranked on perplexity, speed and appetite only — its quality suite arm was cut (§15).

With the drafter on (n4/p0.75, the setting the cards shipped until 2026-10-08; the Swift cards now run n3/p0 with one slot and n4/p0 with two, §06), both Swift files’ drafted thinking was accepted less on one prompt n=4 — reasoning-regime decode 59.96 t/s for Swift IQ4_XS and 50.61 for Swift IQ3_XXS against the anchor’s 76.23 meas — and the reasoning-regime measurement across depths is void (its passes disagreed beyond the 3% band in the sweep and again in a re-run on 2026-10-05), so with the drafter on part of the time saving may be given back while the model reasons. A third drafter setting measured on 2026-10-07, n-max 3 / p-min 0, decoded Swift IQ4_XS’s reasoning tokens ×1.153 faster than n4/p0.75 at the cards’ sampler (4 prompts × 2 seeds, -c 32768, text only) and ×1.087 greedy on the same one prompt der; the anchor did not run in that sweep, so how much of the gap to it this recovers is not measured (§08). The token saving is greedy: at the cards’ temperature-1.0 sampler, two appetite runs per file on one task did not show Swift IQ4_XS generating fewer tokens than the anchor n=2 (ranges, §15). Every difference in this section belongs to the file (fine-tune, quantizer and size), not to Swift-1.5 as a model (sameness, limits).

For the Pi coding agent there is one more file to consider, bytkim’s Qwen3.8-27B-pi, and nothing measured makes it the better choice. bartowski’s IQ4_XS of it shares Swift IQ4_XS’s tensor layout and chat template; on the same 175 prompts it shows no difference the suite can detect against the anchor or Swift IQ4_XS (5-set composite 80.6 at xhigh, 81.2 at medium; ALPACA and MT-Bench not judged) n=25, with the drafter off it decodes at Swift IQ4_XS’s speed, and in Pi it passed a three-probe smoke test n=1. No agent-task benefit was measured, and the author’s agent results are cited, not reproduced. Its launch command and its limits are in Appendix B.

Choosing a file and card by the job

Find your job in the table; each row names the file, the card to copy and the measurement behind the pick. The token, score and speed figures are from one sweep, 2026-10-04, and the two window figures from the knee sweep of 2026-10-03/04 and Appendix A’s arithmetic; speed bands are greedy-only (temperature 0) and are not measured at the temperature-1.0 sampler the cards ship (§15).

Three files, one architecture

Three GGUF files of the same architecture run every comparison in this section; the anchor is named throughout as “the anchor” (files hashed and headers read 2026-10-04).

Every tensor name and shape is identical across all three files — 866 tensors each meas, with one presence-only array in the Swift files, so the architecture and the per-token memory of the window are measured once and apply to all three; file size and the drafter head differ per file (sameness). The three files share a tokenizer (gate passed 2026-10-04), so raw perplexity across files is a legal drift measure (§08).

What was held constant

The three files ran on one machine, one llama.cpp build (10502, commit 0adcc3bb5), one driver (596.36) and the same flags per instrument, in one sweep (2026-10-04) with the anchor interleaved between Swift arms wherever it ran (the knee sweep before it, 2026-10-03/04, ran its picks in a fixed order and carries no cross-model claim) — so a difference in any table below belongs to the file, not to the rig. Two quantities were not held constant: the quantizer and file size differ between files (limits), and the chat templates differ: they render the suite’s single-turn prompts identically, and the Swift template rejects high (sameness).

Four departures from the plan apply to this sweep; the first two bear on every Swift table below. The quantizer and file size differ between the anchor (unsloth UD-IQ4_XS) and the Swift files (bartowski IQ4_XS and IQ3_XXS), so every speed or quality difference mixes fine-tune, quantizer and size. The scored suite ran with the drafter off, while the published cards ship it on: the drafter’s effect on speed is measured in the drafter-regime arms, its effect on scores is not. Every GPU step ran under a job memory cap (MEASURED_INFERENCE_MEM_CAP_GB); a step that needs more is stopped, not slowed, so the cap changes no measured quantity. Appetite ran before the speed probes, drafter regimes and deep fills, so its token counts could size the suite’s plan; the order changes no measured quantity either. The limits that follow are listed below.

Where the three files are the same

The model’s architecture, tokenizer arrays, KV cache geometry, suite prompt rendering and sampling defaults are identical across all three files (header diff, 2026-10-04), and the per-token memory slope measured on Swift IQ4_XS matches Appendix A’s within the 2% check. Where the base guide measured one of these on the anchor, that measurement applies to both Swift files without re-running it.

What does not transfer. The MTP draft head is quantized to Q4_0 in both Swift files (227.91 MiB) against Q6_K and Q8_0 in the anchor (334.75 MiB), so drafter VRAM and drafter acceptance do not carry over — this page measured both for Swift (§08, §06). The token_embd and output tensors differ in type and byte count: the anchor carries token_embd Q3_K (546,304,000 B) and output Q5_K (874,086,400 B); Swift IQ4_XS carries IQ4_XS (675,430,400 B) and Q6_K (1,042,944,000 B); Swift IQ3_XXS carries Q4_K (715,161,600 B) and Q5_K (874,086,400 B) — yet with the drafter off Swift IQ4_XS, 8.58% larger than the anchor, decodes at the anchor’s speed (ratio 1.005), so decode speed is measured per file rather than predicted from size. Two tokenizer scalars differ at the header: padding_token_id is 248055 in the anchor and 248044 in both Swift files, and add_bos_token (false) is present only in the Swift files (header diff, 2026-10-04). The chat template diverges outside the suite’s single-turn prompts, and the high effort level, which the Swift template rejects, does not transfer (§14).

The provenance chain: each sweep is read against the anchor that ran in it

Absolute decode speeds do not survive across sweeps (§06), so a Swift figure on this page is set beside the anchor only inside one sweep and one build, and is carried across sweeps only as a ratio to the anchor. A new model that wants to join the comparison runs the anchor beside it, in its own sweep, and the anchor’s reproduction of its known figures is what ties that sweep back to the August data in §06–§09. The chain below records what each sweep measured, what the anchor returned in it, and what the sweep therefore licenses. The August sweeps’ own absolutes live in §15; they are not restated here as October numbers. The Pi fine-tune’s study (2026-10-06/07) is tied in differently: its speed and window steps ran Swift IQ4_XS (and, for one window, Swift IQ3_XXS) beside it, not the anchor, so its speed figures are ratios to Swift inside that sweep, and its suite cells are paired with the anchor’s and Swift IQ4_XS’s cells of row R7, reused rather than re-run (row R9).

Rows R2–R8 are one sweep, 2026-10-04, whose reasoning-regime probes were re-run, void again, on 2026-10-05; row R1 is the knee sweep before it, 2026-10-03/04. Row R9 is the Pi fine-tune’s own sweep, 2026-10-06/07, with no anchor arm. The speed absolutes of rows R2–R8 meet the August sweeps of row R0 only as the anchor’s ratio in row R3 (§06).

The anchor file is 14,252,845,984 bytes with sha256 40fac4050e940397dbf13087afd50f4734a11805bf9d65ef8ddd7483470e6199, hashed 2026-10-04 meas. Its perplexity reproduces at 6.5956 to the printed digit (0.0% drift, exact reproduction). Its transcripts reproduce: 171 of 171 comparable (non-truncated) anchor items are byte-identical in response text and generated token count to the August transcripts. August recorded no sha256, so the identity of the anchor across the two sweeps rests on byte size, perplexity and 171 transcript matches, not on a hash recorded in August.

The single-arm characterisations — the Swift IQ4_XS deep fills, vision on both Swift files, the needle tests (which did not run; §15), and the Swift IQ3_XXS knee loads — license no cross-model claim. Appetite is a same-sweep comparison at n=2, reported as ranges: the anchor generated 22,718–80,213 tokens, Swift IQ4_XS 55,621–60,991, Swift IQ3_XXS 52,929–67,455 (xhigh, temperature 1.0, -c 131072, uncapped, 2026-10-04).

What this comparison cannot say

Every difference measured in this sweep mixes a fine-tune, a quantizer and a file size. The list below names the claims that do not follow from the data, with the reason each one fails and, where one exists, the measurement in the register (§15) that would close it.

• “Swift-1.5 thinks less” or “Swift-1.5 is faster” as a property of the fine-tune. The fine-tune, the quantizer and the bit-width all differ between the files: the anchor (unsloth UD-IQ4_XS) packs token_embd as Q3_K (546,304,000 B) against IQ4_XS (675,430,400 B) in Swift IQ4_XS, and output as Q5_K (874,086,400 B) against Q6_K (1,042,944,000 B); Swift IQ4_XS is 1,223,105,952 B larger (+8.58%). A same-quantizer pair, or the Swift BF16 weights, would separate them.
• The anchor’s drafter VRAM or acceptance rate applied to the Swift files. The MTP draft heads differ in precision and size: the anchor’s head is Q8_0 + Q6_K + F32 at 334.75 MiB; the Swift heads are Q4_0 + F32 at 227.91 MiB. Drafter VRAM and acceptance are measured per file (§06, §08).
• An August absolute set beside an October absolute. This machine shows two throughput levels across sessions (§03); the anchor sat on the lower one in October. Only ratios to the anchor within one sweep are comparable.
• The divergence (KL) of these files against the Swift-1.5 BF16 weights. The BF16 file is 54.66 GB; this machine has 31.8 GB of RAM. ukisai’s card KLD is for ukisai’s files, not bartowski’s. A KLD run against a BF16 or Q8_0 reference would close it (§15).
• Perplexity as a quality ranking between the base model and the fine-tune. The tokenizer gate passed, so raw perplexity is comparable as drift from wikitext, but across fine-tunes with different training data it is not a quality rank.
• “The same quality” beyond single-turn tasks of this suite. The 175-prompt suite has no read-then-edit task and no multi-turn agent loop. A read-then-edit task would narrow it (§15).
• The suite’s quality scores with the drafter on, as the cards ship. The scored arms ran drafter off; on the one drafter prompt with reasoning off, greedy text matched across drafter settings for Swift IQ4_XS (evidence, not proof) and differed for the anchor and Swift IQ3_XXS (cannot say either way). Suite runs with the drafter on would close it (§15).
• The judged pair as an independent witness. The judge seats are Claude, as is this page’s author. A second-vendor or human judge would close it (§15).
• Judged scores compared across judge sessions. The same 25 MT-Bench answers scored 79.7 and 87.0 in two sessions; on ALPACA the same answers moved +0.59 and +1.33 against the two earlier passes. Only paired comparisons inside one session are read (§15).
• A thinking-length difference at the cards’ own sampler. Appetite is n=2 per file at temperature 1.0 on one task; the token saving is greedy-only. More appetite samples would narrow it (§15).
• Swift IQ3_XXS results applied to another quantizer’s IQ3_XXS. bartowski’s quantizer and recipe differ from other quantizers’ IQ3_XXS files.
• Swift speed at the cards’ temperature-1.0 sampler as a band. Every Swift speed band on this page is greedy; the temperature-1.0 decode is a cross-reference from two appetite runs per file, not a band. A sampler band would close it (§15).
• The reasoning-regime speed of any file in this sweep. That probe group is void in both runs: in the sweep its first pass overlapped downloads on the test host, and a re-run on 2026-10-05 failed the same 3% agreement between passes in five of its six Swift cells. More passes per cell would let the spread be measured rather than tested against the band (§15).
• The energy per token of the drafter-on cards beyond one prompt, and at the pairs they carry now. The drafter-regime energy (n10/p0.5 and n4/p0.75, 2026-10-04) rests on one prompt of 700 generated tokens (14.5–33.1 s per pass), and the board sat at its software power cap in 97–100% of active samples, so joules per token restates throughput. A drafter-regime sweep over more prompts, with the power logger running, would close it.
• The drafter settings as optimal for the Swift head — narrowed 2026-10-08. The n/p sweep ran a grid at the cards’ sampler and xhigh, shallow and after a 57,540-token prefix, at -c 81920: on Swift IQ4_XS no drafter, n2–n6 and n10 at p-min 0, n4 at p-min 0.25, 0.5 and 0.75, and n10/p0.5; on Swift IQ3_XXS n3, n4 and n5 at p-min 0 and n4/p0.75 with one slot, and n3/p0, n4/p0 and n4/p0.75 with two slots busy. Best score: n3/p0 with one slot (shallow reasoning tokens 1.189 and 1.097 of n4/p0.75; after the prefix 1.052 and 1.007, intervals including 1) and n4/p0 with two (1.337 per slot, second pass, one load per setting) der (§06). Still open: the optimum at each card’s own deep-filled window with an image, n2 on Swift IQ3_XXS, other prompts and agent use, quality with the drafter on, and n-gram or draft-model drafting (§15).
• Swift behaviour in aider, OpenCode, Qwen Code or DeepSeek Harness. None has been tested; the Swift template differs from the anchor’s (unsloth) template and rejects high as a reasoning effort. It is byte-identical to the template in lmstudio-community’s Q4_K_M of the base model (8,952 characters, header read 2026-10-06), so the difference is the unsloth file’s, not the base model’s.^C40 A probe per agent would close it (§15).
• An agent benefit from the Pi fine-tune. bytkim’s Qwen3.8-27B-pi ran only a three-probe smoke test in Pi (n=1 each); its suite scores show no difference the suite can detect, and the author’s agent results are cited, not reproduced. An agent-task benchmark with repeats would close it (Appendix B, §15).
• ukisai’s Swift files against bartowski’s on quality. The two repos carry the same recipe in 21 of their 22 shared tiers, with a different imatrix (Appendix A). A KLD pass per tier for each repo’s file against the same reference would close it (§15).
• The quality of the 12 and 16 GB Swift files in Appendix A. No Swift file below IQ3_XXS ran any quality instrument. A suite run per file would close it (§15).
• Swift IQ3_XXS’s quality on the 175-prompt suite, and quality at depth for any Swift card. Swift IQ3_XXS was cut from the suite for time when the recipes were fixed, and ranks on perplexity, speed and appetite only; the needles did not run, so the deep-window cards are speed-verified but quality-unverified at depth. The register carries both gaps (§15).

These are the technical words this page uses without stopping to explain them again. Each is defined once, here, with one of this campaign's own numbers attached wherever a number makes it concrete. Every later section links back to this box instead of defining anything again.

One recipe per class of card. Find yours below, copy the block. The reasoning behind every flag is in §04 to §09.

24 GB readers: two files compete. Take the default (UD-IQ4_XS) unless you want the highest measured quality at a 122k window, in which case take the Q4_K_M and accept that it leaves only about 0.3 GiB for your desktop. The two are statistically tied on quality and the default is smaller and faster in every comparison — details at the Q4_K_M recipe.

NVIDIA 24–32 GB · 3090 / 4090 / 5090 — THE DEFAULT · UD-IQ4_XS (vision, speed, big window)

:: TAKE THIS ONE for everyday use: quality statistically TIED with Q4_K_M (6.596 vs
:: 6.535, +0.9%, smaller than the +/-0.063 combined error bar), measured FASTER
:: everywhere (decode 42.97 vs 39.99 with no drafter and 93.9 vs 81.7 at n10/p0.5 at -c 32768,
:: both from one matched sweep; prefill 1365 vs 1303), 2.1 GiB smaller — and
:: this block ships VISION on. The Q4_K_M block below is kept so that an option you
:: have already heard of is answered rather than absent; take it only if you want the
:: highest measured quality at a 122k window and can free the VRAM your desktop is
:: using - it leaves only about 0.3 GiB. Caveat: IQ-format decode (files whose name contains IQ) was measured on
:: CUDA only - verify on Vulkan or Metal before trusting these speeds there.
:: file: unsloth/Qwen3.8-27B-GGUF (UD-IQ4_XS, 14.25 GB = 13.3 GiB) + BF16 mmproj
llama-server.exe -m Qwen3.8-27B-UD-IQ4_XS.gguf  --alias qwen/qwen3.8-27b
  --mmproj mmproj-Qwen3.8-27B-BF16.gguf  --image-min-tokens 1024  --image-max-tokens 10580
                                :: min 1024 is a FLOOR only — it lifts small images and never
                                :: shrinks a big one (measured: 720p 922 -> 1,077 tokens, 1440p
                                :: unchanged at 3,602). max 10580 = 4K-class detail. Per-image
                                :: token costs: section 12. The projector costs 1,138 MiB of
                                :: VRAM and 0% of decode (0.04-0.09% at a 91k fill, section 06)
  -c 122880                     :: 20,658 MiB of server VRAM by the measured budget model
                                :: (section 05), leaving 2.7-3.7 GiB for a desktop that itself
                                :: measured 1,179-1,669 MiB idle, no server. The largest window
                                :: whose every byte still lives on the card in THIS configuration
                                :: is 163,840; 122,880 is the daily value, chosen to leave a
                                :: desktop room
  -ngl 99  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0
                                :: -ngl 99 is not a guess: llama.cpp counts the output layer as
                                :: layer 65, so -ngl 64 leaves the output layer on the CPU and
                                :: costs 29.5% of decode with NO memory signature (section 11)
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0  :: n3/p0 since
                                :: 2026-10-08: the best score for this file at this card's
                                :: sampler and xhigh (n/p sweep, -c 81920, text only, two
                                :: sessions): shallow reasoning tokens 1.111x n4/p0.75
                                :: (paired 95% 1.075-1.156), answers 1.120x; after a
                                :: 57,540-token prefix 1.061x, interval including 1.
                                :: n10/p0.5 read 0.961x and 1.049x, and 1.079x after the
                                :: prefix, intervals overlapping (section 06).
                                :: About 150 MiB LESS than n4, so this window only gains
                                :: margin. n10/p0.5 was the August grid's peak on GREEDY,
                                :: thinking-off code: 93.9 vs 83.5 t/s at -c 32768 (section 06).
                                :: On PROSE (the August greedy probe), drop speculation:
                                ::   1.16x for ~1.8 GiB, measured at n4/p0.75
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"  :: all three levels fit this window ONLY IF you leave
                                :: room for them. The window is one POT: prompt + thinking +
                                :: answer share it. xhigh wants 61,500-75,800 tokens of
                                :: THINKING, so your prompt and history must stay under about
                                :: 47,000 of these 122,880 - 38%% of the window. Past that,
                                :: xhigh truncates and returns NOTHING (section 09). medium
                                :: when you are waiting at the keyboard, and whenever the
                                :: context is deep
:: FLAGS DELIBERATELY OMITTED HERE:
::   --parallel 2 — serving one person, one slot is faster: a second slot costs 35.4%
::     of per-slot throughput (85.79 -> 55.41 t/s) while buying 22.0% of aggregate
::     (82.98 -> 101.25 t/s), measured 2026-08-25 on the matched drafter-on pair.
::     Raise it only to serve two users at once (section 03's axis table).
::   -ctk q4_0 — measured at +0.693% perplexity against fp16 (q8_0 costs +0.309%).
::     A knowing trade for a bigger window, not a default (section 08).
::   nvidia-smi -pl — MEASURED 2026-08-25, re-measured 2026-08-28 in a saturating
::     regime (SwPowerCap residency 99-100%). Capping the board is a real
::     efficiency win: 300 W costs 4.1% of throughput for 13.1% less board power
::     and 9.4% less energy per token; 250 W costs 14.6% for 27.1% less power
::     and 14.7% less energy. The non-saturating sweep (2026-08-25) read
::     5.0%/6.5% and 12.8%/13.7% — understated the deal at 300 W because its
::     stock arm sat 32 W under its own cap. Left out of the recipe because it is a persistent
::     hardware setting and needs an elevated shell, not because it does not
::     work (section 11).
:: text-only variants — drop the --mmproj/--image-* line and the window grows:
::   2026-10-08: text-only, keep the n3/p0 drafter above: it holds about
::     7 x 150 MiB less than n10 at the same window (derived, not filled).
::     The ceilings below were measured with the drafters they name.
::   -c 180224 + n-max 10 / p-min 0.5: the measured text-only ceiling WITH that
::     drafter - 23,729 MiB of board VRAM, 847 MiB of slack left for a desktop.
::     (196,608 leaves 415 MiB and 212,992 leaves 280, with decode already
::     sagging 56.6 -> 51.9 -> 49.2 on a
::     SHORT probe. The 847 MiB at 180,224 does not clear the 1,796 MiB reserve.)
::   -c 262144 THE FULL NATIVE CONTEXT - but ONLY with --spec-type none AND with no
::     graphical session using this card ("headless", section 02: closing everything
::     that draws, a second card driving the screen, or a remote login with nobody
::     logged in locally - NOT just unplugging the monitor).
::     Drafter off it derives to 23,216 MiB (22.7 GiB) at load, leaving 1,360 MiB of
::     board VRAM - 309 MiB SHORT of the 1,669 MiB desktop worst case, so it does not
::     fit with a screen attached. WITH the n-max 4 / p-min 0.75
::     drafter on it does not fit at all: 2,364 MiB measured living in system RAM
::     (text-only). With the projector loaded as well, 3,502 MiB spilled (same
::     drafter) and a 91k-token document then decodes at 8.0 t/s (section 05).
::   The rule: a ceiling belongs to the whole configuration — file + drafter flags +
::   projector + the desktop you run. Quote all four or the ceiling is not portable.

llama-server.exe -m Qwen3.8-27B-UD-IQ4_XS.gguf --alias qwen/qwen3.8-27b --mmproj mmproj-Qwen3.8-27B-BF16.gguf --image-min-tokens 1024 --image-max-tokens 10580 -c 122880 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"

The file: unsloth/Qwen3.8-27B-GGUF, UD-IQ4_XS — 14.25 GB on the download page, 13.3 GiB resident (download pages count in decimal GB, memory budgets in binary GiB; the ~7% gap is real and matters whenever memory is tight). Plus the BF16 mmproj projector from either repository if you want vision. The window: -c 122880. The flags: all layers on the GPU, one slot, an 8-bit KV cache, and the n-max 3 / p-min 0 drafter, the best score for this file at this card’s sampler (§06). The effort levels it offers: all three — low, medium and xhigh. Nothing is excluded here: xhigh's measured thinking appetite is 61,500–75,800 tokens and a 122,880-token window holds its upper tail with room for a prompt and an answer. Expected speed, measured before 2026-10-08 with the drafters named (n3/p0 was measured only as ratios, §06): 76–79 t/s of answer tokens on short-context code in most sessions (20 of 23 sessions fell in this band on 2026-08-28, UD-IQ4_XS, n10/p0.5, -c 32768, -ctk/-ctv q8_0, -ngl 99, --parallel 1, -fa on, greedy, 700 tokens, fresh server per session, one warmup discarded, RTX 3090 at 350 W, build 10502). Three of those 23 sessions reached 88–89 t/s, about 13% higher, with nothing in between and nothing predicting which level a session will land on. The published 86.91 t/s in §08 sits inside the high band; the figure a reader will usually see is in the low band. Seven hypotheses for the gap have been tested and ruled out; the cause is unmeasured (§15, C27). Then 64.8 at 91k of depth (n4/p0.75), and 37–39 t/s on the xhigh reasoning stream at 91k (measured at n4/p0.75 and n10/p0.5) — which is where a long agent run actually spends its wall clock. VRAM at the top of this window: 20,658 MiB of server process by the measured budget model, leaving 2.7–3.7 GiB of board VRAM depending on what your desktop is holding. Room left for your desktop: 2.7–3.7 GiB — this is the row chosen so that a browser and an agent's web interface do not push it over.

NVIDIA 24–32 GB · UD-IQ4_XS at 98,304 with vision · the former speed pick (n3/p0 since 2026-10-08)

:: 2026-10-08: n3/p0 now, the n/p sweep's best score for this file at the cards'
:: temperature-1.0 sampler with thinking on (section 06), so this is the default
:: recipe at a smaller window. The record that named it, measured 2026-08-25:
:: THE SPEED PICK: the default recipe with the WIDE drafter, bought with 24,576
:: tokens of window. 87.04 t/s against the default's 65.67 (+32.5%), both with the
:: projector loaded, 700-token novel-JS probes at short context, three settled
:: probes after a discarded warmup. Peaks 22,258 MiB with a 1440p shot on 77,194
:: tokens and keeps 2,318 - clearing this page's 1,796 MiB desktop reserve by 522.
:: The SAME drafter at -c 122880 keeps only 1,126 and is 670 SHORT, which is why
:: the window moved. The August serve-qwen.bat shipped this as pick [0].
:: file: unsloth/Qwen3.8-27B-GGUF (UD-IQ4_XS, 14.25 GB = 13.3 GiB) + BF16 mmproj
llama-server.exe -m Qwen3.8-27B-UD-IQ4_XS.gguf  --alias qwen/qwen3.8-27b
  --mmproj mmproj-Qwen3.8-27B-BF16.gguf  --image-min-tokens 1024  --image-max-tokens 10580
  -c 98304                      :: 24,576 fewer than the default. It paid for the n10/p0.5
                                :: drafter until 2026-10-08; it is not itself a speed setting.
  -ngl 99                       :: all 65 layers - the output layer counts as one (section 05)
  --parallel 1                  :: one slot: the whole window goes to one conversation
  -fa on  -ctk q8_0  -ctv q8_0  :: q8_0 KV REQUIRES flash attention; -fa off refuses to load
  --spec-type draft-mtp  --spec-draft-n-max 3  --spec-draft-p-min 0.0
                                :: n3/p0 since 2026-10-08: shallow reasoning 1.111x and
                                :: answers 1.120x n4/p0.75 on this file at the cards'
                                :: sampler, against n10/p0.5's 0.961x and 1.049x (section 06).
                                :: With a greedy client (temperature 0) the sweep's greedy
                                :: probe ranked n4/p0 and n5/p0 above n3/p0.
                                :: About 7 x 150 MiB lighter than the n10/p0.5 this
                                :: window was measured with (derived), so it only gains
                                :: margin. Until 2026-10-08: n10/p0.5, 898 MiB over
                                :: n4/p0.75, which is why the window moved.
  --jinja                       :: required for the chat template and for reasoning_content
  --host 127.0.0.1  --port 1234

llama-server.exe -m Qwen3.8-27B-UD-IQ4_XS.gguf --alias qwen/qwen3.8-27b --mmproj mmproj-Qwen3.8-27B-BF16.gguf --image-min-tokens 1024 --image-max-tokens 10580 -c 98304 -ngl 99 --parallel 1 -fa on -ctk q8_0 -ctv q8_0 --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234

NVIDIA 24–32 GB — THE BIG-WINDOW PICK · UD-Q2_K_XL (vision and the drafter and 196,608 tokens)

The file: unsloth UD-Q2_K_XL — 9.83 GB, 9.154 GiB resident. The window: -c 196608. The flags: the universal set, the n-max 4 / p-min 0 drafter der, and the full image budget. This file was not in the 2026-10-08 n/p sweep, and its window was measured with n4/p0.75; p-min costs no memory, so the window still fits. On the four files that sweep ran, p-min 0.75 at n-max 4 decoded shallow reasoning at 0.87–0.97 of p-min 0’s speed with one slot and read 0.90–1.03 of it after a 57,540-token prefix; on UD-IQ4_XS deep answers it was slightly faster (n4/p0 read 0.973 of it, interval 0.969–0.996; §06). n-max 3 scored best with one slot on the three files run at xhigh and is about 150 MiB lighter, so n3/p0 here is a candidate to measure, not a measured setting. VRAM at depth: 22,014 MiB meas — peak, with a 1440p screenshot in flight on 163,124 tokens, sampled every 0.5 s. Room left for your desktop: 2,562 MiB, clearing this page’s 1,796 MiB reserve by 766. Effort: medium is what this pick ships, and the reason is the window rather than the clock. The window is one pot — prompt, thinking and answer share it. An ordinary xhigh answer only wants about 2,217 tokens of thinking, which fits here easily; but this pick exists to fill 196,608, and at the 163,124 tokens it was measured at only 33,484 remain. A complete-build task — the one thing measured wanting 61,500–75,800, four August samples of one task (§09) — would truncate there and return nothing. So: xhigh for questions of a full window, medium when you are asking for a whole program. Effort changes how much the model writes, not how long it takes to read your prompt.^C3

Why this pick exists, added 2026-08-25. A reader wanted the largest window they could get while keeping both vision and the drafter, and this page had no answer for them — the 4-bit file cannot do it, and the full 262,144 fails in every drafter state once an image is in flight. The 2-bit file can, and three separate measurements say it costs less than its name suggests: it ties the 4-bit reference on accuracy (one of 75 paired items different, p=1.00), it reads images identically (7/7 against 7/7 on a generated detail target), and its window genuinely retrieves — 5 of 5 needles found at every depth out to 241,655 tokens, with a clean control (§08). What you actually trade is decode speed at short windows, where the 4-bit file drafts better.

196,608 is the ceiling, measured. -c 229376 peaks at 23,529 MiB and keeps only 1,047 — it is under the reserve at load, before an image is even sent. So this window is not a round number chosen for looks; it is the largest that survived the test. One honest note on that failing run: its answer to the image question came back as prose rather than the bare number, which the scoring here cannot grade either way. It is reported as a memory failure only, because that is what was measured.

The cost that matters more than VRAM: prefill. Filling this window took 278 seconds meas, against 210 s at -c 163840 and roughly 8 minutes at the full 262,144. Prefill is compute-bound and does not get cheaper. If your agent appends to its context you pay this once and the prefix cache carries you; if it rebuilds context every turn you pay it every turn, and -c 163840 saves you a minute for 32,768 fewer tokens. Choose on that, not on the bigger number.

:: THE BIG-WINDOW PICK: vision + drafter + 196,608 tokens, which no 4-bit
:: configuration on this page can hold. MEASURED 2026-08-25 with a 1440p image
:: in flight at 163,124 tokens: peak 22,014 MiB, 2,562 MiB left for a desktop.
:: 196,608 IS THE CEILING, and it is measured rather than assumed:
::   -c 196608   peak 22,014 MiB   2,562 MiB slack   CLEARS by 766
::   -c 229376   peak 23,529 MiB   1,047 MiB slack   FAILS by 749
::   -c 262144   peak 22,877 MiB   1,699 MiB slack   FAILS by 97 (drafter OFF!)
:: 229,376 is already under the reserve AT LOAD (1,516) before an image lands.
:: file: unsloth/Qwen3.8-27B-GGUF (UD-Q2_K_XL, 9.83 GB = 9.154 GiB)
llama-server.exe -m Qwen3.8-27B-UD-Q2_K_XL.gguf  --alias qwen/qwen3.8-27b
  --mmproj mmproj-Qwen3.8-27B-BF16.gguf  --image-min-tokens 1024  --image-max-tokens 10580
                                :: full image budget - the 1024 cap makes the model
                                :: confidently MISREAD fine detail (section 05)
  -c 196608  -ngl 99  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0
                                :: 22,014 MiB peak MEASURED at depth with an image, not
                                :: at load. Drop to -c 163840 for 3,852 MiB of slack and
                                :: a minute less prefill
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0  :: derived:
                                :: not in the 2026-10-08 n/p sweep. This window was
                                :: measured at n4/p0.75, and p-min costs no memory, so
                                :: it still fits. On the four files swept, p-min 0.75
                                :: ran 0.87-0.97x p-min 0 at n-max 4 on shallow reasoning
                                :: and 0.90-1.03x after a 57.5k prefix (one slot;
                                :: section 06). n3/p0, ~150 MiB lighter, is a
                                :: candidate to measure here, not a measured
                                :: setting. n10/p0.5 costs ~900 MiB more and is not
                                :: measured at this window - do not assume it fits
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"  :: the window is one POT
                                :: an ordinary xhigh answer averages 2,217 tokens (175 benchmark
                                :: prompts), which the 33,484 left at the 163,124-token depth above
                                :: holds 15 times over. medium is shipped because a COMPLETE BUILD
                                :: is the one thing measured wanting 61,500-75,800 (four August samples of
                                :: one task, section 09) and that would truncate here. Ask questions
                                :: of a full window at xhigh; ask for a whole program at medium.
                                :: This has nothing to do with the 278 s prefill: effort changes how
                                :: much the model WRITES, not how long it takes to READ your prompt.

llama-server.exe -m Qwen3.8-27B-UD-Q2_K_XL.gguf --alias qwen/qwen3.8-27b --mmproj mmproj-Qwen3.8-27B-BF16.gguf --image-min-tokens 1024 --image-max-tokens 10580 -c 196608 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"

NVIDIA 24–32 GB — THE ONE YOU HAVE HEARD OF · Q4_K_M (no measurable quality advantage, 2.1 GiB larger, ~0.3 GiB left for your desktop)

The file: lmstudio-community/Qwen3.8-27B-GGUF, Q4_K_M — 16.5 GB on the download page, 15.4 GiB resident. The window: -c 122880. The flags: the same universal set as the default recipe, with the n-max 4 / p-min 0 drafter der. Q4_K_M was not in the 2026-10-08 n/p sweep and this row was measured at n4/p0.75, but p-min costs no memory, so the row still fits; on the four files the sweep ran, p-min 0.75 at n-max 4 decoded shallow reasoning at 0.87–0.97 of p-min 0’s speed with one slot and read 0.90–1.03 of it after a 57,540-token prefix, and on UD-IQ4_XS deep answers it was slightly faster (n4/p0 read 0.973 of it; §06). n10/p0.5 does not fit here, and at the cards’ sampler it was slower than n3/p0 on UD-IQ4_XS on shallow reasoning and answer tokens anyway. The effort levels it offers: all three, for the same reason as the default recipe. An ordinary xhigh answer averages 2,217 tokens of thinking across 175 benchmark prompts; the 61,500–75,800 figure quoted elsewhere on this page is a ceiling from four August samples of one task — a complete build — and a 122,880-token window holds even that, leaving about 47,000 for your prompt. Expected speed, measured with these flags but at n4/p0.75 (n4/p0 is not measured on this file): 69.8 t/s of answer tokens on short-context code and 57.9 t/s on the reasoning stream; the 81.7 t/s figure this file is famous for on this page belongs to the wider drafter, which this row cannot afford — it is a cross-reference, not this recipe's speed. VRAM at the top of this window: about 23.7 GiB of board VRAM with a desktop up, measured, leaving roughly 0.3 GiB. Room left for your desktop: about 0.3 GiB — less than a browser typically holds. §11 measured this very configuration spilling about 1.9 GB with two browsers open and falling to 20–35 t/s.

:: TAKE THIS ONE only if you believe an unresolvable 0.9% perplexity edge is
:: worth 2.1 GiB AND you can free the VRAM your desktop uses - this row leaves
:: about 0.3 GiB, less than a browser typically holds. The edge is 0.061 of
:: perplexity against this page's own +/-0.063 comparison error: INSIDE THE BAR.
:: The GSM8K half of the old argument is unaudited. THE DEFAULT ABOVE is better
:: on context, speed, vision and desktop headroom, and no worse on any
:: measurement this page can resolve.
:: file: lmstudio-community/Qwen3.8-27B-GGUF (Q4_K_M, 16.5 GB = 15.4 GiB)
:: before first run: set "Prefer No Sysmem Fallback" (NVIDIA Control Panel) and
:: never write -ngl 64 — both explained in section 11
llama-server.exe -m Qwen3.8-27B-Q4_K_M.gguf  --alias qwen/qwen3.8-27b
  -c 122880                     :: 24 GB: the largest window whose every byte still lives on the
                                :: card is ~131k, so this leaves slack (section 05's two
                                :: ceilings). 5090 32 GB: -c 262144
                                :: fits — budget ~11 GiB of KV at the MEASURED slope, not the
                                :: 8.5 GiB the q8 arithmetic suggests
  -ngl 99  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0  :: derived:
                                :: this row's ~23.7 GiB was measured at n4/p0.75; p-min costs
                                :: no memory, so it still fits. The 2026-08-23 matched sweep
                                :: ranked n10/p0.5 first on BOTH files (81.7 vs 69.8 here at
                                :: n4/p0.75), but that was greedy code with thinking off. Q4_K_M
                                :: was not in the 2026-10-08 n/p sweep; on the four files it ran,
                                :: p-min 0.75 ran 0.87-0.97x p-min 0 at n-max 4 on shallow
                                :: reasoning, one slot, and 0.90-1.03x after a 57.5k prefix (section
                                :: 06). n3/p0 is a candidate to measure; it would free ~150 MiB.
                                :: BUDGET the drafter: 1,008 MiB fixed + 11.4% of your window
                                :: (section 05). --spec-type none frees all of it; on prose it is
                                :: worth only ~1.16x, so that is the easiest 1.8 GiB to reclaim
:: vision: add  --mmproj mmproj-Qwen3.8-27B-BF16.gguf --image-min-tokens 1024
:: --image-max-tokens 10580  when you need it. The projector costs 1,138 MiB
:: once loaded - not the 0.867 GiB its file weighs - and with this file at -c 122880
:: that is the measured ~23.7 GiB reference configuration: it fits, but leaves only
:: ~0.3 GiB of the card for your desktop. With a desktop running, drop to -c 65536,
:: or -c 49152 if you keep browsers open.
:: Budget from the MEASURED slope: each 32k of window costs ~1.375 GiB
:: with the drafter on and ~1.22 GiB with it off; the 1.06 GiB q8 arithmetic is a
:: FLOOR. Vision + a desktop + a BIG window = the default recipe above.
:: On the 1 GiB-larger UD-Q4_K_XL, projector + full context needs more than 24 GB -
:: use -c 98304
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"  :: quality-first default;
                                :: medium when the wait matters (section 09's price list)
:: FLAGS DELIBERATELY OMITTED: the same three as the default recipe — --parallel 2 (+22.0%
:: aggregate but -35.4% per slot with the drafter on, measured 2026-08-25 — slower
:: for one user), -ctk q4_0 (+0.693% perplexity, a window trade), and nvidia-smi -pl
:: (measured: 300 W costs 4.1% of throughput for 9.4% less energy, 250 W costs 14.6%
:: for 14.7%; left out because it is a persistent hardware setting needing an elevated
:: shell, not because it does not work).

llama-server.exe -m Qwen3.8-27B-Q4_K_M.gguf --alias qwen/qwen3.8-27b -c 122880 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"

Swift-1.5 recipes — no detectable quality cost at IQ4_XS, wider windows at IQ3_XXS

Take the default card unless you have a reason to choose a Swift file. Swift IQ4_XS scored indistinguishably from the anchor on this page’s 175-prompt suite at smoke-test scale (25 questions per set, greedy, drafter off) and generated fewer tokens there (§09). Swift IQ3_XXS holds the longer window of the two Swift files and, with vision at two slots, more tokens per slot (§05, §08), at the cost of a perplexity gap against Swift IQ4_XS on their shared tokenizer (§08). Five of the six Swift cards carry the window that is fully resident at this page’s 1,796 MiB desktop threshold, measured where a deep fill shows it and derived from the per-card arithmetic otherwise; the per-card arithmetic has no two-slot text configuration, so the Swift IQ3_XXS two-slot text card carries the October launcher’s window, fully resident with a desktop of up to 976 MiB rather than 1,796. Each card’s speed band was measured on the lower of this machine’s two speed levels (the two-levels note); its derived higher-level line divides each figure by the anchor’s 0.917 (at 1.5k) or 0.928 (at 91k) of its August speed. The card for bytkim’s Qwen3.8-27B-pi, a fine-tune whose bartowski IQ4_XS shares Swift IQ4_XS’s tensor layout and template, is in Appendix B.

NVIDIA 24 GB · 3090 — Swift IQ4_XS (text, 1 slot, 159,744-token window)

:: Swift IQ4_XS, text, 1 slot, 159,744 tokens per slot.
:: One sweep 2026-10-04; not comparable with the August
:: tables in section 06.
:: Choose this for text-only on the Swift fine-tune;
:: for vision on the same file choose
:: the Swift IQ4_XS vision card, for the base model choose
:: the default card; for a longer window choose
:: the Swift IQ3_XXS text card.
:: file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF
::   (IQ4_XS, 15,475,951,936 bytes)
llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ4_XS.gguf
  --alias qwen/qwen3.8-27b
  -c 159744                     :: measured: deep fill to 156,261 tokens
                                :: left 1,880 MiB, fully resident at this
                                :: page's 1,796 MiB threshold. This config
                                :: is not in the October launcher.
                                :: Server footprint at depth (dedicated +
                                :: shared): 22,212 + 484 MiB (fill to
                                :: 156,261 tokens, 44.57 t/s at depth)
  -ngl 99  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --reasoning-preserve
  --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0  :: the
                                :: n3/p0 drafter since 2026-10-08: the n/p
                                :: sweep's best score for this file at this
                                :: sampler and xhigh (-c 81920, text only, two
                                :: sessions): shallow reasoning 1.189x n4/p0.75
                                :: (paired 95% 1.158-1.224), 1.052-1.057x after
                                :: a 57,540-token prefix, answers 1.062-1.064x;
                                :: n2/p0 faster shallow but 0.932x after the
                                :: prefix (section 06). ~150 MiB LESS than n4,
                                :: so the window, filled at n4/p0.75, only
                                :: gains margin.
                                :: Earlier, one prompt (-c 32768, greedy,
                                :: reasoning off): n4/p0.75 acceptance 0.917,
                                :: 3.348 accepted tokens per verify pass;
                                :: n10/p0.5 acceptance 0.641, 5.667 per pass.
                                :: Board VRAM at -c 32768: n4 costs
                                :: 1,092-1,124 MiB; n10 adds 898 MiB (one pass
                                :: read 851 on a board counter that includes the
                                :: desktop, section 05).
                                :: Greedy thinking-off text is byte-identical
                                :: across none/n4/n10 for this file on that
                                :: one prompt: evidence, not proof, that
                                :: drafter-off scores transfer (section 08)
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"
                                :: low, medium, xhigh offered; high is not a
                                :: level of this template (section 14).
                                :: Turns per window: section 09
:: speed (greedy-only; not the card's temperature-1.0 sampler),
:: 2 passes, -c 131072, one slot, n4/p0.75, 2026-10-04:
::   floor (drafter off, reasoning off, 1.5k): 42.88 t/s (first
::     pass inside the downloads; passes agree within 3%)
::   answer regime, 1.5k: 87.2 t/s (higher level, derived: 95.1)
::   answer regime, 91k: 61.14 t/s (higher level, derived: 65.9)
::   reasoning regime, 1.5k and 91k: void (its group failed
::     3% pass agreement in the sweep and in a 2026-10-05 re-run)
::   ceiling (n10/p0.5, reasoning off, -c 32768, one prompt):
::     98.1 t/s
::   temperature-1.0 decode (n=2, n4/p0.75): 61.11-65.9 t/s
::     (section 09)
:: FLAGS DELIBERATELY OMITTED:
::   --parallel 2 — unmeasured for this file (section 03)
::   -ctk q4_0 — unmeasured for this file (the q8_0 cache
::     measured 6.7708 perplexity against f16's 6.7742,
::     section 08)
::   nvidia-smi -pl — not measured for this file; a persistent hardware
::     setting that needs an elevated shell.
:: invalidation trigger: build 10502 / 0adcc3bb5,
::   driver 596.36, a new upload of the file
:: fields that differ from the measured command: port (no
::   effect on inference); sampler and effort
::   (the speed probes ran greedy; the temperature-1.0 line
::   above ran the card's sampler at xhigh); the deep fill
::   ran --reasoning off at temperature 0 (KV memory per
::   token is the same for prompt and generated tokens);
::   every measurement loaded the file memory-mapped and the
::   window fills turned the host-RAM prompt cache off
::   (--cache-ram 0); the card loads with --load-mode none
::   (system RAM and load time differ, not throughput: 43.0
::   against 43.4 t/s on the anchor, section 15) and keeps the prompt cache
::   on (its RAM use at depth is unmeasured).
::   Quality at depth: not verified (the needle test did
::   not run)

llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ4_XS.gguf --alias qwen/qwen3.8-27b -c 159744 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --reasoning-preserve --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"

The file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF, IQ4_XS — 15.48 GB on the download page, 14.41 GiB (15,475,951,936 bytes). The window: -c 159744 meas — deep-filled to 156,261 tokens at an 846 MiB desktop and fully resident; the fill left 1,880 MiB for a desktop der, which clears this page’s 1,796 MiB threshold. This configuration is not in the October launcher. The flags: all layers on the GPU, one slot, an 8-bit KV cache, the n-max 3 / p-min 0 drafter, and --reasoning-preserve. The effort levels it offers: low, medium and xhigh; high is not a level of this template (§14); turns per window in §09. Speed (greedy-only; not the card’s temperature-1.0 sampler) n=2, n4/p0.75, one sweep 2026-10-04: 87.2 t/s answer tokens at 1.5k of depth (higher speed level, derived: 95.1), 61.14 t/s at 91k (higher level: 65.9); floor 42.88 t/s with no drafter (its first pass also ran inside the downloads; the passes agree within 3%), ceiling 98.1 t/s at n10/p0.5 on one prompt at -c 32768; on 2026-10-07, n-max 3 / p-min 0 decoded reasoning tokens ×1.153 faster than this card’s n4/p0.75 under its own sampler (4 prompts × 2 seeds, -c 32768, text only), with answer tokens level on one prompt and 152 MiB less VRAM (§08); since 2026-10-08 the card runs n3/p0, the n/p sweep’s best score for this file (shallow reasoning tokens 1.189 of n4/p0.75, §06); its window was deep-filled at n4/p0.75, and n-max 3 needs about 150 MiB less; the reasoning-regime probes are void (their passes disagreed beyond 3%, in the sweep and in a re-run on 2026-10-05). Not comparable with the August tables in §06. Temperature-1.0 decode n=2, n4/p0.75: 61.11–65.9 t/s (§09). VRAM at the top of this window: server footprint at depth 22,212 MiB dedicated + 484 MiB shared meas (deep fill to 156,261 tokens, 2026-10-04). Room left for your desktop: 1,880 MiB — clears this page’s 1,796 MiB threshold.

NVIDIA 24 GB · 3090 — Swift IQ4_XS (text + vision, 1 slot, 128,000-token window, derived)

UNVERIFIED — DERIVED CONFIG
:: Swift IQ4_XS, text + vision, 1 slot, 128,000 tokens
:: per slot.
:: One sweep 2026-10-04; not comparable with the August
:: tables in section 06.
:: Choose this for vision on the Swift fine-tune; for
:: text-only on the same file (wider window) choose
:: the Swift IQ4_XS text card, for the base model choose
:: the default card; for a longer window choose
:: the Swift IQ3_XXS vision card.
:: file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF
::   (IQ4_XS, 15,475,951,936 bytes) + f16 mmproj
::   (927,607,552 bytes)
llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ4_XS.gguf
  --alias qwen/qwen3.8-27b
  --mmproj mmproj-ukisai_Swift-1.5-Qwen3.8-27b-f16.gguf
  --image-min-tokens 1024  --image-max-tokens 10580
  -c 128000                     :: DERIVED from per-card arithmetic to
                                :: clear this page's 1,796 MiB desktop
                                :: threshold. The October launcher ships
                                :: -c 139264: fully resident with an
                                :: 841 MiB desktop (deep fill to 126,838
                                :: tokens, 2026-10-04); it left 1,330 MiB,
                                :: below the 1,796 MiB threshold.
                                :: Server footprint at depth from the
                                :: -c 139264 fill with vision (dedicated
                                :: + shared): 22,802 + 444 MiB; at the
                                :: card's -c 128000 the footprint derives
                                :: lower
  -ngl 99  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --reasoning-preserve
  --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0  :: the
                                :: n3/p0 drafter since 2026-10-08: the n/p
                                :: sweep's best score for this file at this
                                :: sampler and xhigh (-c 81920, text only, two
                                :: sessions): shallow reasoning 1.189x n4/p0.75
                                :: (paired 95% 1.158-1.224), 1.052-1.057x after
                                :: a 57,540-token prefix, answers 1.062-1.064x;
                                :: n2/p0 faster shallow but 0.932x after the
                                :: prefix (section 06). ~150 MiB LESS than n4,
                                :: so the window, filled at n4/p0.75, only
                                :: gains margin.
                                :: The sweep ran with no projector and no
                                :: image; how image tokens move the best
                                :: n-max was not measured.
                                :: Earlier, one prompt (-c 32768, greedy,
                                :: reasoning off): n4/p0.75 acceptance 0.917,
                                :: 3.348 accepted tokens per verify pass;
                                :: n10/p0.5 acceptance 0.641, 5.667 per pass.
                                :: Board VRAM at -c 32768: n4 costs
                                :: 1,092-1,124 MiB; n10 adds 898 MiB (one pass
                                :: read 851 on a board counter that includes the
                                :: desktop, section 05).
                                :: Greedy thinking-off text is byte-identical
                                :: across none/n4/n10 for this file on that
                                :: one prompt: evidence, not proof, that
                                :: drafter-off scores transfer (section 08)
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"
                                :: low, medium, xhigh offered; high is not a
                                :: level of this template (section 14).
                                :: Turns per window: section 09
:: speed (greedy-only; not the card's temperature-1.0 sampler),
:: 2 passes, -c 131072, one slot, no projector, n4/p0.75,
:: 2026-10-04:
::   floor (drafter off, reasoning off, 1.5k): 42.88 t/s (first
::     pass inside the downloads; passes agree within 3%)
::   answer regime, 1.5k: 87.2 t/s (higher level, derived: 95.1)
::   answer regime, 91k: 61.14 t/s (higher level, derived: 65.9)
::   reasoning regime, 1.5k and 91k: void (its group failed
::     3% pass agreement in the sweep and in a 2026-10-05 re-run)
::   ceiling (n10/p0.5, reasoning off, -c 32768, one prompt):
::     98.1 t/s
::   temperature-1.0 decode (n=2, n4/p0.75): 61.11-65.9 t/s
::     (section 09)
::   deep fill decode at -c 139264 with vision: 41.03 t/s
::     on 126,838 tokens (n=1, 2026-10-04); the text-only
::     fill at the same window: 47.42 t/s (n=1)
::   re-filled 2026-10-07 (Pi fine-tune study; another
::     sweep, not comparable with 41.03): 45.14 / 45.55
::     t/s (n=2), 1,217 MiB desktop, the same footprint
::     of 22,802 + 444 MiB
::   n-max 3 / p-min 0 (2026-10-07, -c 32768, no projector):
::     reasoning tokens 1.153x n4/p0.75 at this card's
::     sampler, 1.087x greedy; answer tokens level on one
::     prompt; 152 MiB less VRAM. Since 2026-10-08 this
::     card runs n3/p0 (n/p sweep: shallow reasoning
::     1.189x n4/p0.75 at xhigh, -c 81920, with no
::     projector and no image)
::     (section 08)
:: FLAGS DELIBERATELY OMITTED:
::   --parallel 2 — unmeasured for this file (section 03)
::   -ctk q4_0 — unmeasured for this file (the q8_0 cache
::     measured 6.7708 perplexity against f16's 6.7742,
::     section 08)
::   nvidia-smi -pl — not measured for this file; a persistent hardware
::     setting that needs an elevated shell.
:: invalidation trigger: build 10502 / 0adcc3bb5,
::   driver 596.36, a new upload of the file
:: fields that differ from the measured command: port (no
::   effect on inference); sampler and effort
::   (the speed probes ran greedy; the temperature-1.0 line
::   above ran the card's sampler at xhigh); the deep fill
::   ran --reasoning off at temperature 0 (KV memory per
::   token is the same for prompt and generated tokens);
::   every measurement loaded the file memory-mapped and the
::   window fills turned the host-RAM prompt cache off
::   (--cache-ram 0); the card loads with --load-mode none
::   (system RAM and load time differ, not throughput: 43.0
::   against 43.4 t/s on the anchor, section 15) and keeps the prompt cache
::   on (its RAM use at depth is unmeasured).
::   Quality at depth: not verified (the needle test did
::   not run)

llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ4_XS.gguf --alias qwen/qwen3.8-27b --mmproj mmproj-ukisai_Swift-1.5-Qwen3.8-27b-f16.gguf --image-min-tokens 1024 --image-max-tokens 10580 -c 128000 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --reasoning-preserve --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"

The file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF, IQ4_XS — 15.48 GB on the download page, 14.41 GiB (15,475,951,936 bytes), plus the f16 mmproj projector (927,607,552 bytes). The window: -c 128000 der — derived from the per-card arithmetic to clear this page’s 1,796 MiB desktop threshold; the October launcher ships -c 139264, which is fully resident with an 841 MiB desktop (deep fill to 126,838 tokens, 2026-10-04) but leaves only 1,330 MiB for the desktop, below the 1,796 MiB threshold. The flags: all layers on the GPU, one slot, an 8-bit KV cache, the n-max 3 / p-min 0 drafter, the full image budget (1,024–10,580 tokens), and --reasoning-preserve. The effort levels it offers: low, medium and xhigh; high is not a level of this template (§14); turns per window in §09. Speed (greedy-only; not the card’s temperature-1.0 sampler) n=2, one slot, no projector, n4/p0.75, one sweep 2026-10-04: the band of the text card, because the probes ran without the projector — 87.2 t/s answer tokens at 1.5k (higher speed level, derived: 95.1), 61.14 t/s at 91k (higher level: 65.9); floor 42.88 (its first pass also ran inside the downloads; the passes agree within 3%), ceiling 98.1 at n10/p0.5; n-max 3 / p-min 0 (2026-10-07) decoded reasoning tokens ×1.153 faster than n4/p0.75 under the card’s sampler, with answer tokens level on one prompt and 152 MiB less VRAM (§08); since 2026-10-08 the card runs n3/p0 (§06), which needs about 150 MiB less than the n4/p0.75 its window was filled at; the reasoning-regime probes are void (their passes disagreed beyond 3%, in the sweep and in a re-run on 2026-10-05). Deep fill decode at -c 139264 with vision, n4/p0.75: 41.03 t/s on 126,838 tokens n=1, against 47.42 t/s for the text-only fill at the same window n=1 — not a paired test of the projector: the vision fill also held a 1440p image and drew lower drafter acceptance (0.83 against 0.88, the mean of two probes each), and the paired probe in §06 found the projector itself decode-free (0.04–0.09% at a 91k fill). A second fill on 2026-10-07, in the Pi fine-tune study (window-knee.py, in the order Pi, Swift, Swift, Pi, the same 126,838 tokens, projector loaded), held at 45.14 and 45.55 t/s meas n=2, with a 1,217 MiB desktop before load, a board VRAM peak of 24,019 MiB and 444 MiB in shared memory at depth, which leaves the same 22,802 MiB dedicated footprint as the 2026-10-04 fill der; its absolute t/s are not comparable with the 2026-10-04 figure (Appendix B). Not comparable with the August tables in §06. Temperature-1.0 decode n=2, n4/p0.75, no projector: 61.11–65.9 t/s (§09). VRAM at the top of this window: the fill at -c 139264 with vision measured 22,802 MiB dedicated + 444 MiB shared server footprint at depth meas (fill to 126,838 tokens, 2026-10-04); at the card’s -c 128000 the footprint derives lower. Room left for your desktop: at the launcher’s -c 139264 the fill left 1,330 MiB (below this page’s 1,796 MiB threshold); the derived -c 128000 targets 1,796 MiB by construction.

NVIDIA 24 GB · 3090 — Swift IQ3_XXS (text, 1 slot, 216,064-token window, derived)

UNVERIFIED — DERIVED CONFIG
:: Swift IQ3_XXS, text, 1 slot, 216,064 tokens (derived from the per-card
:: arithmetic at this page's 1,796 MiB desktop threshold).
:: Choose this for one user and the longest window of the Swift cards;
:: choose the Swift IQ3_XXS two-slot text card for two users,
:: the Swift IQ3_XXS vision card for images,
:: the Swift IQ4_XS text card for the lower-perplexity file,
:: the default card for the base model.
:: file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF
::   (IQ3_XXS, 12,320,168,256 bytes)
llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ3_XXS.gguf
  --alias qwen/qwen3.8-27b
  -c 216064                     :: DERIVED, not measured: the per-card
                                :: arithmetic's fully resident window at
                                :: 1,796 MiB desktop (Appendix A). The
                                :: October launcher carries 262,144,
                                :: decode-verified at 253,952 (38.46 t/s)
                                :: and 262,144 (36.23 t/s) with a 415 MiB
                                :: desktop, and fully resident with a
                                :: desktop of up to 976 MiB (35 to spare,
                                :: dedicated memory only); on the server
                                :: footprint the 262,144 fill left 327 MiB
  -ngl 99  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --reasoning-preserve
  --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"
:: VRAM at the top of this window: not measured at 216,064. The knee
::   rows above it measured a server footprint (dedicated + shared) of
::   23,213 + 668 MiB at 253,952 and 23,565 + 684 at 262,144, both
::   fully resident with a desktop of up to 976 MiB; 216,064 is the
::   per-card arithmetic's window at a 1,796 MiB desktop (derived,
::   Appendix A).
:: Speed band (greedy-only; not the card's temperature-1.0 sampler):
::   one slot, n4/p0.75, -c 131072, two passes, one sweep 2026-10-04; not
::   comparable with the August tables in section 06.
::   Floor (drafter off, reasoning off, 1.5k): 44.12 t/s (43.74, 44.49);
::     first pass inside the downloads, passes within 3%
::   Short context, answer regime, 1.5k: 79.31 t/s (79.17, 79.45);
::     higher speed level, derived: 86.5 t/s
::   Short context, reasoning regime, 1.5k: void (its group failed
::     3% pass agreement in the sweep and in a 2026-10-05 re-run)
::   At depth, answer regime, 91k: 56.34 t/s (56.35, 56.33);
::     higher speed level, derived: 60.7 t/s
::   At depth, reasoning regime, 91k: void (its group failed
::     3% pass agreement in the sweep and in a 2026-10-05 re-run)
::   Ceiling (n10/p0.5, reasoning off, -c 32768, one prompt):
::     99.53 t/s
::   Temperature-1.0 decode (the card's sampler, n4/p0.75, -c 131072,
::     xhigh): 63.86-63.98 t/s
:: Effort levels offered: low, medium, xhigh (high is not a level of
::   this template, section 14). Turns per window:
::   section 09.
:: Drafter: on the one prompt (JavaScript, -c 32768, greedy, reasoning off):
::   n4/p0.75 acceptance 0.927, 3.142 accepted tokens per verify pass
::   n10/p0.5 acceptance 0.686, 6.071 accepted tokens per verify pass
::   Drafter board VRAM at -c 32768: n4 costs 1,092-1,098 MiB, n10 adds 898 MiB
::   2026-10-08 n/p sweep (this file, one slot, -c 81920, text only, xhigh):
::   n3/p0 shallow reasoning 1.097x n4/p0.75 (paired 95% 1.065-1.130), answers
::   1.054x; n4/p0 1.034x, n5/p0 0.940x. n3 keeps one fewer recurrent row
::   than n4 (149.625 MiB predicted; not resolved on this file, section 06).
:: FLAGS DELIBERATELY OMITTED:
::   --parallel 2 — the two-slot text card.
::   -ctk q4_0 — not measured for this file.
::   nvidia-smi -pl — not measured for this file; a persistent hardware
::     setting that needs an elevated shell.
:: Invalidation trigger: build 10502 / 0adcc3bb5, driver 596.36,
::   a new upload of the bartowski Swift-1.5 IQ3_XXS file.
:: Fields that differ from the measured command:
::   Port (no effect on inference). Sampler and effort
::   (the speed probes ran greedy; the temperature-1.0 line ran the
::   card's sampler at xhigh). The knee rows ran --reasoning off at
::   temperature 0 (KV memory per token is the same for prompt and
::   generated tokens); every measurement loaded the file
::   memory-mapped and the window fills turned the host-RAM prompt
::   cache off (--cache-ram 0); the card loads with --load-mode none
::   (system RAM and load time differ, not throughput: 43.0 against
::   43.4 t/s on the anchor, section 15) and keeps the prompt cache on (its RAM
::   use at depth is unmeasured). Quality at depth: not verified (the
::   needle test did not run).

llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ3_XXS.gguf --alias qwen/qwen3.8-27b -c 216064 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --reasoning-preserve --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"

The file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF, IQ3_XXS — 12.32 GB on the download page, 11.47 GiB (12,320,168,256 bytes). The window: -c 216064 der — derived from the per-card arithmetic at this page’s 1,796 MiB desktop threshold (Appendix A); not measured at this window. The October launcher carries -c 262144, decode-verified at 253,952 (38.46 t/s) and 262,144 (36.23 t/s) with a 415 MiB desktop meas, and fully resident with a desktop of up to 976 MiB, 35 MiB to spare, by the fully resident test, which counts dedicated memory only; on the server footprint (dedicated + shared) the 262,144 fill left 327 MiB der (§05). The flags: all layers on the GPU, one slot, an 8-bit KV cache, the n-max 3 / p-min 0 drafter (§06), and --reasoning-preserve. The effort levels it offers: low, medium and xhigh; high is not a level of this template (§14); turns per window in §09. Speed (greedy-only; not the card’s temperature-1.0 sampler) n=2, one slot, n4/p0.75, one sweep 2026-10-04: 79.31 t/s answer tokens at 1.5k of depth (two pass means 79.17–79.45; higher speed level, derived: 86.5), 56.34 t/s at 91k (two pass means 56.33–56.35; higher level: 60.7); floor 44.12 t/s with no drafter (43.74–44.49; its first pass also ran inside the downloads), ceiling 99.53 t/s at n10/p0.5 on one prompt at -c 32768; the reasoning-regime probes are void (their passes disagreed beyond 3%, in the sweep and in a re-run on 2026-10-05). Not comparable with the August tables in §06. Temperature-1.0 decode n=2, n4/p0.75: 63.86–63.98 t/s (§09). VRAM at the top of this window: not measured at -c 216064; the knee rows measured a server footprint of 23,213 MiB dedicated + 668 MiB shared at 253,952 tokens and 23,565 + 684 at 262,144 meas (2026-10-04). Room left for your desktop: 1,796 MiB — the per-card arithmetic’s target at this page’s desktop threshold der (Appendix A).

NVIDIA 24 GB · 3090 — Swift IQ3_XXS (text, 2 slots, 123,904 tokens per slot, desktop up to 976 MiB)

:: Swift IQ3_XXS, text, 2 slots, 123,904 tokens per slot: the October
:: launcher's window. The per-card arithmetic has no two-slot text
:: configuration, so this page derives no window at its 1,796 MiB
:: desktop threshold for this card; this one is fully resident with a
:: desktop of up to 976 MiB, not at 1,796 MiB.
:: Choose this to serve two users on text; choose
:: the Swift IQ3_XXS text card for one user and a longer
:: window, the Swift IQ3_XXS two-slot vision card for images,
:: the default card for the base model.
:: file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF
::   (IQ3_XXS, 12,320,168,256 bytes)
llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ3_XXS.gguf
  --alias qwen/qwen3.8-27b
  -c 247808                     :: 123,904 per slot x 2, derived: the
                                :: launcher's knee rule, between loaded
                                :: steps at 116,736 and 124,928 per slot
                                :: that both passed the fully resident
                                :: test with a desktop up to 976 MiB
                                :: (section 05). Decode held to 133,120 per
                                :: slot (29.2 t/s per slot) and collapsed
                                :: at 135,168 (14.04 t/s per slot). The
                                :: knee loads ran at a 356-479 MiB desktop.
  -ngl 99  --parallel 2  --load-mode none  -ctk q8_0 -ctv q8_0
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --reasoning-preserve
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"
:: VRAM at the top of this window: the knee row at 124,928 per slot,
::   just above the card's window, measured a server footprint
::   (dedicated + shared) of 23,546 + 424 MiB, deep-filled with both
::   slots decoding (2026-10-04).
:: Speed band (greedy-only; not the card's temperature-1.0 sampler):
::   one slot, n4/p0.75, -c 131072, two passes, one sweep 2026-10-04; not
::   comparable with the August tables in section 06.
::   Floor (drafter off, reasoning off, 1.5k): 44.12 t/s (43.74, 44.49);
::     first pass inside the downloads, passes within 3%
::   Short context, answer regime, 1.5k: 79.31 t/s (79.17, 79.45);
::     higher speed level, derived: 86.5 t/s
::   Short context, reasoning regime, 1.5k: void (its group failed
::     3% pass agreement in the sweep and in a 2026-10-05 re-run)
::   At depth, answer regime, 91k: 56.34 t/s (56.35, 56.33);
::     higher speed level, derived: 60.7 t/s
::   At depth, reasoning regime, 91k: void (its group failed
::     3% pass agreement in the sweep and in a 2026-10-05 re-run)
::   Ceiling (n10/p0.5, reasoning off, -c 32768, one prompt):
::     99.53 t/s
::   Two slots decoding at once (knee row, 124,928 per slot, answer
::     regime, n4/p0.75): 28.97 t/s per slot
::   Temperature-1.0 decode (the card's sampler, n4/p0.75, -c 131072,
::     xhigh, one slot): 63.86-63.98 t/s
:: Effort levels offered: low, medium, xhigh (high is not a level of
::   this template, section 14). Turns per window:
::   section 09.
:: Drafter: on the one prompt (JavaScript, -c 32768, greedy, reasoning off):
::   n4/p0.75 acceptance 0.927, 3.142 accepted tokens per verify pass
::   n10/p0.5 acceptance 0.686, 6.071 accepted tokens per verify pass
::   Drafter board VRAM at -c 32768: n4 costs 1,092-1,098 MiB, n10 adds 898 MiB
::   2026-10-08 n/p sweep at the October launcher's two-slot vision shape
::     (123,904 x 2, projector loaded, both slots busy, xhigh; second pass,
::     one load per setting, no interval): per slot, n4/p0 decoded
::     reasoning tokens 1.337x n4/p0.75 and answers 1.484x (n3/p0:
::     1.185x and 1.233x). Depth with two slots: not resolved. Same
::     n-max, same memory (section 06). The text-only two-slot shape was not run.
:: FLAGS DELIBERATELY OMITTED:
::   -ctk q4_0 — not measured for this file.
::   nvidia-smi -pl — not measured for this file; a persistent hardware
::     setting that needs an elevated shell.
:: Invalidation trigger: build 10502 / 0adcc3bb5, driver 596.36,
::   a new upload of the bartowski Swift-1.5 IQ3_XXS file.
:: Fields that differ from the measured command:
::   Port (no effect on inference). Sampler and effort
::   (the speed probes ran greedy; the temperature-1.0 line ran the
::   card's sampler at xhigh). The knee rows ran --reasoning off at
::   temperature 0 (KV memory per token is the same for prompt and
::   generated tokens); every measurement loaded the file
::   memory-mapped and the window fills turned the host-RAM prompt
::   cache off (--cache-ram 0); the card loads with --load-mode none
::   (system RAM and load time differ, not throughput: 43.0 against
::   43.4 t/s on the anchor, section 15) and keeps the prompt cache on (its RAM
::   use at depth is unmeasured). Quality at depth: not verified (the
::   needle test did not run).

llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ3_XXS.gguf --alias qwen/qwen3.8-27b -c 247808 -ngl 99 --parallel 2 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --reasoning-preserve --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"

The file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF, IQ3_XXS — 12.32 GB on the download page, 11.47 GiB (12,320,168,256 bytes). The window: -c 247808, 123,904 per slot × 2 der — the October launcher’s window, set by its knee rule between two loaded steps, 116,736 and 124,928 per slot, that both passed this page’s fully resident test with a desktop of up to 976 MiB meas (§05); decode held to 133,120 per slot (29.2 t/s per slot) and collapsed at 135,168 (14.04 t/s per slot), both at n4/p0.75. This page’s per-card arithmetic has no two-slot text configuration, so the page derives no window at its 1,796 MiB desktop threshold for this card. The flags: all layers on the GPU, two slots, an 8-bit KV cache, the n-max 4 / p-min 0 drafter (§06), and --reasoning-preserve. The effort levels it offers: low, medium and xhigh; high is not a level of this template (§14); turns per window in §09. Speed (greedy-only; not the card’s temperature-1.0 sampler) n=2, one slot, n4/p0.75, one sweep 2026-10-04: 79.31 t/s answer tokens at 1.5k of depth (two pass means 79.17–79.45; higher speed level, derived: 86.5), 56.34 t/s at 91k (two pass means 56.33–56.35; higher level: 60.7); floor 44.12 t/s with no drafter (43.74–44.49; its first pass also ran inside the downloads), ceiling 99.53 t/s at n10/p0.5 on one prompt at -c 32768; the reasoning-regime probes are void (their passes disagreed beyond 3%, in the sweep and in a re-run on 2026-10-05). With both slots decoding at 124,928 per slot, the knee row measured 28.97 t/s per slot at n4/p0.75 meas. Not comparable with the August tables in §06. Temperature-1.0 decode n=2, one slot, n4/p0.75: 63.86–63.98 t/s (§09). VRAM at the top of this window: the knee row at 124,928 per slot measured a server footprint of 23,546 MiB dedicated + 424 MiB shared meas (2026-10-04). Room left for your desktop: up to 976 MiB by the fully resident test, which counts dedicated memory only; on the server footprint (dedicated + shared) the knee row at 124,928 per slot left 606 MiB der — both below this page’s 1,796 MiB threshold.

NVIDIA 24 GB · 3090 — Swift IQ3_XXS (text + vision, 1 slot, 195,584-token window, derived)

UNVERIFIED — DERIVED CONFIG
:: Swift IQ3_XXS, text + vision, 1 slot, 195,584 per slot (derived from the
:: per-card arithmetic at this page's 1,796 MiB desktop threshold).
:: Choose this for vision on Swift IQ3_XXS with one slot; choose
:: the Swift IQ3_XXS two-slot vision card for two users at a
:: shorter window, the Swift IQ4_XS vision card for the
:: lower-perplexity file, the Swift IQ3_XXS text card for
:: text only, the default card for the base model.
:: file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF
::   (IQ3_XXS, 12,320,168,256 bytes) + F16 mmproj
::   (mmproj-ukisai_Swift-1.5-Qwen3.8-27b-f16.gguf, 927,607,552 bytes)
llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ3_XXS.gguf
  --alias qwen/qwen3.8-27b
  --mmproj mmproj-ukisai_Swift-1.5-Qwen3.8-27b-f16.gguf
  --image-min-tokens 1024  --image-max-tokens 10580
  -c 195584                     :: DERIVED, not measured: the per-card
                                :: arithmetic's fully resident window at
                                :: 1,796 MiB desktop (Appendix A). The
                                :: October launcher carries 262,144,
                                :: decode-verified at 253,952 (31.14 t/s)
                                :: and 262,144 (30.84 t/s) with a 366-415
                                :: MiB desktop; not fully resident: it
                                :: spills at load from its first step
  -ngl 99  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --reasoning-preserve
  --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"
:: VRAM at the top of this window: not measured at 195,584. The knee
::   rows above it measured a server footprint (dedicated + shared) of
::   23,837 + 1,516 MiB at 253,952 and 23,847 + 1,874 at 262,144, both
::   spilling; 195,584 is the per-card arithmetic's fully resident
::   window at a 1,796 MiB desktop (derived, Appendix A).
:: Speed band (greedy-only; not the card's temperature-1.0 sampler):
::   one slot, no projector, n4/p0.75, -c 131072, two passes, one sweep 2026-10-04;
::   not comparable with the August tables in section 06.
::   Floor (drafter off, reasoning off, 1.5k): 44.12 t/s (43.74, 44.49);
::     first pass inside the downloads, passes within 3%
::   Short context, answer regime, 1.5k: 79.31 t/s (79.17, 79.45);
::     higher speed level, derived: 86.5 t/s
::   Short context, reasoning regime, 1.5k: void (its group failed
::     3% pass agreement in the sweep and in a 2026-10-05 re-run)
::   At depth, answer regime, 91k: 56.34 t/s (56.35, 56.33);
::     higher speed level, derived: 60.7 t/s
::   At depth, reasoning regime, 91k: void (its group failed
::     3% pass agreement in the sweep and in a 2026-10-05 re-run)
::   Ceiling (n10/p0.5, reasoning off, -c 32768, one prompt):
::     99.53 t/s
::   Temperature-1.0 decode (the card's sampler, n4/p0.75, -c 131072,
::     xhigh): 63.86-63.98 t/s
:: Effort levels offered: low, medium, xhigh (high is not a level of
::   this template, section 14). Turns per window:
::   section 09.
:: Drafter: on the one prompt (JavaScript, -c 32768, greedy, reasoning off):
::   n4/p0.75 acceptance 0.927, 3.142 accepted tokens per verify pass
::   n10/p0.5 acceptance 0.686, 6.071 accepted tokens per verify pass
::   Drafter board VRAM at -c 32768: n4 costs 1,092-1,098 MiB, n10 adds 898 MiB
::   2026-10-08 n/p sweep (this file, one slot, -c 81920, text only, xhigh):
::   n3/p0 shallow reasoning 1.097x n4/p0.75 (paired 95% 1.065-1.130), answers
::   1.054x; n4/p0 1.034x, n5/p0 0.940x. n3 keeps one fewer recurrent row
::   than n4 (149.625 MiB predicted; not resolved on this file, section 06).
::   No projector or image in the sweep; how images move the best n-max: not measured.
:: FLAGS DELIBERATELY OMITTED:
::   --parallel 2 — the two-slot vision card.
::   -ctk q4_0 — not measured for this file.
::   nvidia-smi -pl — not measured for this file; a persistent hardware
::     setting that needs an elevated shell.
:: Invalidation trigger: build 10502 / 0adcc3bb5, driver 596.36,
::   a new upload of the bartowski Swift-1.5 IQ3_XXS file.
:: Fields that differ from the measured command:
::   Port (no effect on inference). Sampler and effort
::   (the speed probes ran greedy; the temperature-1.0 line ran the
::   card's sampler at xhigh). The knee rows ran --reasoning off at
::   temperature 0 (KV memory per token is the same for prompt and
::   generated tokens); every measurement loaded the file
::   memory-mapped and the window fills turned the host-RAM prompt
::   cache off (--cache-ram 0); the card loads with --load-mode none
::   (system RAM and load time differ, not throughput: 43.0 against
::   43.4 t/s on the anchor, section 15) and keeps the prompt cache on (its RAM
::   use at depth is unmeasured). Quality at depth: not verified (the
::   needle test did not run).

llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ3_XXS.gguf --alias qwen/qwen3.8-27b --mmproj mmproj-ukisai_Swift-1.5-Qwen3.8-27b-f16.gguf --image-min-tokens 1024 --image-max-tokens 10580 -c 195584 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --reasoning-preserve --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"

The file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF, IQ3_XXS — 12.32 GB on the download page, 11.47 GiB (12,320,168,256 bytes), plus the F16 mmproj (927,607,552 bytes). The window: -c 195584 der — derived from the per-card arithmetic at this page’s 1,796 MiB desktop threshold (Appendix A); not measured at this window. The October launcher carries -c 262144, decode-verified at 253,952 (31.14 t/s) and 262,144 (30.84 t/s) with a 366–415 MiB desktop; not fully resident: it spills at load from its first step. The flags: all layers on the GPU, one slot, an 8-bit KV cache, the vision projector, the n-max 3 / p-min 0 drafter, and --reasoning-preserve. The effort levels it offers: low, medium and xhigh; high is not a level of this template (§14); turns per window in §09. Speed (greedy-only; not the card’s temperature-1.0 sampler) n=2, one slot, no projector, n4/p0.75, one sweep 2026-10-04: 79.31 t/s answer tokens at 1.5k of depth (two pass means 79.17–79.45; higher speed level, derived: 86.5), 56.34 t/s at 91k (two pass means 56.33–56.35; higher level: 60.7); floor 44.12 t/s with no drafter (43.74–44.49; its first pass also ran inside the downloads), ceiling 99.53 t/s at n10/p0.5 on one prompt at -c 32768; the reasoning-regime probes are void (their passes disagreed beyond 3%, in the sweep and in a re-run on 2026-10-05). Not comparable with the August tables in §06. Temperature-1.0 decode n=2, n4/p0.75, no projector: 63.86–63.98 t/s (§09). VRAM at the top of this window: not measured at -c 195584; the knee rows measured a server footprint of 23,837 MiB dedicated + 1,516 MiB shared at 253,952 tokens and 23,847 + 1,874 at 262,144, both spilling meas (2026-10-04). Room left for your desktop: 1,796 MiB — the per-card arithmetic’s target at this page’s desktop threshold der (Appendix A).

NVIDIA 24 GB · 3090 — Swift IQ3_XXS (text + vision, 2 slots, 93,184 tokens per slot, derived)

UNVERIFIED — DERIVED CONFIG
:: Swift IQ3_XXS, text + vision, 2 slots, 93,184 per slot (derived from
:: the per-card arithmetic at this page's 1,796 MiB desktop threshold).
:: Choose this to serve two concurrent users with vision on Swift
:: IQ3_XXS; choose the Swift IQ3_XXS one-slot vision card for a
:: longer window when serving one user,
:: the Swift IQ3_XXS two-slot text card for text only,
:: the default card for the base model.
:: file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF
::   (IQ3_XXS, 12,320,168,256 bytes) + F16 mmproj
::   (mmproj-ukisai_Swift-1.5-Qwen3.8-27b-f16.gguf, 927,607,552 bytes)
llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ3_XXS.gguf
  --alias qwen/qwen3.8-27b
  --mmproj mmproj-ukisai_Swift-1.5-Qwen3.8-27b-f16.gguf
  --image-min-tokens 1024  --image-max-tokens 10580
  -c 186368                     :: 93,184 per slot x 2. DERIVED, not
                                :: measured: the per-card arithmetic's
                                :: fully resident window at 1,796 MiB
                                :: desktop (Appendix A). The October
                                :: launcher carries 123,904 per slot
                                :: (247,808 total), decode-verified at
                                :: 122,880/slot (24.9 t/s per slot) and
                                :: 133,120 (23.12 t/s per slot) with a
                                :: 338-539 MiB desktop; not fully resident
                                :: (Appendix A's knee table puts the spill
                                :: onset at 122,880 per slot)
  -ngl 99  --parallel 2  --load-mode none  -ctk q8_0 -ctv q8_0
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --reasoning-preserve
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"
:: VRAM at the top of this window: not measured at 93,184 per slot.
::   The knee rows above it measured a server footprint (dedicated +
::   shared) of 23,691 + 870 MiB at 114,688/slot and 23,737 + 1,530 at
::   122,880/slot; 93,184 is the per-card arithmetic's fully resident
::   window at a 1,796 MiB desktop (derived, Appendix A).
:: Speed band (greedy-only; not the card's temperature-1.0 sampler):
::   one slot, no projector, n4/p0.75, -c 131072, two passes, one sweep 2026-10-04;
::   not comparable with the August tables in section 06.
::   Floor (drafter off, reasoning off, 1.5k): 44.12 t/s (43.74, 44.49);
::     first pass inside the downloads, passes within 3%
::   Short context, answer regime, 1.5k: 79.31 t/s (79.17, 79.45);
::     higher speed level, derived: 86.5 t/s
::   Short context, reasoning regime, 1.5k: void (its group failed
::     3% pass agreement in the sweep and in a 2026-10-05 re-run)
::   At depth, answer regime, 91k: 56.34 t/s (56.35, 56.33);
::     higher speed level, derived: 60.7 t/s
::   At depth, reasoning regime, 91k: void (its group failed
::     3% pass agreement in the sweep and in a 2026-10-05 re-run)
::   Ceiling (n10/p0.5, reasoning off, -c 32768, one prompt):
::     99.53 t/s
::   Two slots decoding at once (knee row, 114,688 per slot, answer
::     regime, n4/p0.75): 25.2 t/s per slot
::   Temperature-1.0 decode (the card's sampler, n4/p0.75, -c 131072,
::     xhigh, one slot): 63.86-63.98 t/s
:: Effort levels offered: low, medium, xhigh (high is not a level of
::   this template, section 14). Turns per window:
::   section 09.
:: Drafter: on the one prompt (JavaScript, -c 32768, greedy, reasoning off):
::   n4/p0.75 acceptance 0.927, 3.142 accepted tokens per verify pass
::   n10/p0.5 acceptance 0.686, 6.071 accepted tokens per verify pass
::   Drafter board VRAM at -c 32768: n4 costs 1,092-1,098 MiB, n10 adds 898 MiB
::   2026-10-08 n/p sweep at the October launcher's two-slot vision shape
::     (123,904 x 2, projector loaded, both slots busy, xhigh; second pass,
::     one load per setting, no interval): per slot, n4/p0 decoded
::     reasoning tokens 1.337x n4/p0.75 and answers 1.484x (n3/p0:
::     1.185x and 1.233x). Depth with two slots: not resolved. Same
::     n-max, same memory (section 06). This card's own 93,184 x 2 window is derived.
:: FLAGS DELIBERATELY OMITTED:
::   -ctk q4_0 — not measured for this file.
::   nvidia-smi -pl — not measured for this file; a persistent hardware
::     setting that needs an elevated shell.
:: Invalidation trigger: build 10502 / 0adcc3bb5, driver 596.36,
::   a new upload of the bartowski Swift-1.5 IQ3_XXS file.
:: Fields that differ from the measured command:
::   Port (no effect on inference). Sampler and effort
::   (the speed probes ran greedy; the temperature-1.0 line ran the
::   card's sampler at xhigh). The knee rows ran --reasoning off at
::   temperature 0 (KV memory per token is the same for prompt and
::   generated tokens); every measurement loaded the file
::   memory-mapped and the window fills turned the host-RAM prompt
::   cache off (--cache-ram 0); the card loads with --load-mode none
::   (system RAM and load time differ, not throughput: 43.0 against
::   43.4 t/s on the anchor, section 15) and keeps the prompt cache on (its RAM
::   use at depth is unmeasured). Quality at depth: not verified (the
::   needle test did not run).

llama-server.exe -m ukisai_Swift-1.5-Qwen3.8-27b-IQ3_XXS.gguf --alias qwen/qwen3.8-27b --mmproj mmproj-ukisai_Swift-1.5-Qwen3.8-27b-f16.gguf --image-min-tokens 1024 --image-max-tokens 10580 -c 186368 -ngl 99 --parallel 2 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --reasoning-preserve --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"

The file: bartowski/ukisai_Swift-1.5-Qwen3.8-27b-GGUF, IQ3_XXS — 12.32 GB on the download page, 11.47 GiB (12,320,168,256 bytes), plus the F16 mmproj (927,607,552 bytes). The window: -c 186368 (93,184 per slot × 2) der — derived from the per-card arithmetic at this page’s 1,796 MiB desktop threshold (Appendix A); not measured at this window. The October launcher carries 123,904 per slot (247,808 total), decode-verified at 122,880 per slot (24.9 t/s per slot) and 133,120 (23.12 t/s per slot), both at n4/p0.75, with a 338–539 MiB desktop; not fully resident (Appendix A’s knee table puts the spill onset at 122,880 per slot). A fill at the launcher’s own window on 2026-10-07 (123,904 × 2, projector loaded, 112,652 and 112,647 tokens in the two slots, both decoding) held at 24.48 and 24.81 t/s per slot at n4/p0.75 meas n=2, with 572–634 MiB desktops and 1,620–1,722 MiB in shared memory at depth; that is another sweep, so its absolute t/s are not comparable with the 2026-10-04 figures (Appendix B). The flags: all layers on the GPU, two slots, an 8-bit KV cache, the vision projector, the n-max 4 / p-min 0 drafter, and --reasoning-preserve. The effort levels it offers: low, medium and xhigh; high is not a level of this template (§14); turns per window in §09. Speed (greedy-only; not the card’s temperature-1.0 sampler) n=2, one slot, no projector, n4/p0.75, one sweep 2026-10-04: 79.31 t/s answer tokens at 1.5k of depth (two pass means 79.17–79.45; higher speed level, derived: 86.5), 56.34 t/s at 91k (two pass means 56.33–56.35; higher level: 60.7); floor 44.12 t/s with no drafter (43.74–44.49; its first pass also ran inside the downloads), ceiling 99.53 t/s at n10/p0.5 on one prompt at -c 32768; the reasoning-regime probes are void (their passes disagreed beyond 3%, in the sweep and in a re-run on 2026-10-05). With both slots decoding at 114,688 per slot, the knee row measured 25.2 t/s per slot at n4/p0.75 meas. Not comparable with the August tables in §06. Temperature-1.0 decode n=2, one slot, n4/p0.75, no projector: 63.86–63.98 t/s (§09). VRAM at the top of this window: not measured at 93,184 per slot; the knee rows measured a server footprint of 23,691 MiB dedicated + 870 MiB shared at 114,688 per slot and 23,737 + 1,530 at 122,880 per slot meas (2026-10-04). Room left for your desktop: 1,796 MiB — the per-card arithmetic’s target at this page’s desktop threshold der (Appendix A).

The full menu for a 24 GB card, losers included

The reference machine’s August launcher offered eight configurations (rows 1–8), and this is all of them — including the ones that lost, each with the number that beat them, so that an option you have already heard of is answered rather than absent. Rows 9–14 are the Swift cards (§03): their speeds come from one sweep on 2026-10-04 and are not comparable with rows 1–8, and five of their six windows are derived. Hover or tap a row to see the flags that produce it. Since 2026-10-08 rows 1, 2 and 9–14 carry the n/p sweep’s drafters (n3/p0 with one slot, n4/p0 with two); rows 4–6 run files the sweep did not measure and carry n4/p0 since 2026-10-09 der (p-min costs no memory, so their windows still fit; on the four files the sweep ran, p-min 0.75 at n-max 4 decoded shallow reasoning at 0.87–0.97 of p-min 0’s speed with one slot and read 0.90–1.03 of it after a 57,540-token prefix); rows 3, 7 and 8 keep the August flags (drafter off, an external drafter, and the August comparison run that row 8 reproduces); every Expect figure keeps the drafter it was measured at (§06).

Every "Expect" band carries its token regime, because answer tokens run about 1.7× the speed of reasoning tokens on code and a band without its regime is not a number you can plan with. Speed also falls with depth, not only with spill: at n4/p0.75 (the flags shipped before 2026-10-08), answer tokens measure 86.3 → 80.2 → 64.8 t/s at 1.5k / 28k / 91k of fill under the cooled protocol, while the reasoning stream on the same file and flags runs 51.2 → 47.1 → 36.6. An agent session holding 10–30k of context therefore lives near 80 t/s of deliverable; the same session left on the xhigh default over a 91k document spends most of its wall clock at 37–39 t/s, at either measured drafter setting (n4/p0.75, n10/p0.5; n3/p0 was not measured at 91k). Slack figures are the board VRAM measured left over with little on screen; the desktop's own share swung 1,179–1,669 MiB in direct no-server readings on one machine (§05), so a row keeping under about 1 GiB has room for a near-idle desktop and nothing beyond it.

@echo off
rem ============================================================================
rem  serve-qwen.bat - llama.cpp server for Qwen3.8-27B on RTX 3090 24GB
rem ============================================================================
rem  Measured 2026-08-22, corrected 2026-08-23. Every pick carries the
rem  measurement that justifies it, because this file travels to other
rem  machines without the guide.
rem  - -ngl 99 always (llama.cpp counts the output layer as layer 65; -ngl 64
rem    leaves the output layer on CPU: 25.7 vs 39.7 t/s on Q4_K_M,
rem    29.8 vs 42.3 on UD-IQ4_XS, with NO memory signature)
rem  - MTP flags are PER PICK, and the split is NOT per file. A matched sweep
rem    (2026-08-23, both quants, novel code, thinking OFF) peaks at n-max 10 /
rem    p-min 0.5 on BOTH files: IQ4_XS 93.9 t/s at -c 32768 (2.18x over its
rem    43.0 no-drafter floor; at -c 180224 the same drafter measures -3.1%%),
rem    Q4_K_M 81.7 (2.04x over 40.0). The shipped n-max 4 / p-min 0.75
rem    reads 83.5 and 69.8 on the same probe - 11%% / 15%% off the peak, and
rem    898 MiB cheaper. So, in August: n10/p0.5 on the text-only picks where
rem    that VRAM is spare, n4/p0.75 wherever the projector is loaded or the
rem    slack is thin.
rem    2026-10-08: SUPERSEDED for this launcher's own sampler (temp 1.0,
rem    thinking on). The n/p sweep found n3/p0 best on UD-IQ4_XS with one
rem    slot: shallow reasoning 1.11x and answers 1.12x n4/p0.75, against
rem    n10/p0.5's 0.96x and 1.05x; n-max 3 is ~150 MiB lighter than 4 and
rem    ~1 GB lighter than 10. So picks 1 and 2 now take SPEC_N3. The figures
rem    above are greedy code with thinking off, where longer drafts still
rem    lead. Picks 4-6 run files the sweep did not measure (guide, section 06).
rem    Acceptance is a property of the MTP head, not of the quant - the two
rem    files land within 1.6 points at six of seven configs (3.7 at the seventh).
rem  - the drafter is also the cheapest ENERGY setting here: 3.21 J per decode
rem    token at n10/p0.5 against 8.10 with --spec-type none (2.52x), and
rem    the board draws the SAME power while decoding either way (344.6 / 341.7 /
rem    341.0 W): the entire win is throughput, not wattage.
rem  - TOKEN REGIME decides the number. Answer tokens (thinking off) run ~1.7x
rem    reasoning tokens, and mean DRAFT LENGTH is what predicts speed, not
rem    acceptance: at 91k depth the reasoning stream drafts 2.99 tokens per
rem    pass (36.6 t/s) and the answer stream 4.31 (62.0) - at the same ~0.90
rem    acceptance. Highest-acceptance config (n2/p0.75, 96.5%%) is the slowest.
rem  - Depth, answer tokens, n4/p0.75, cooled probes: 86.3 t/s at 1.5k, 80.2 at
rem    28k, 64.8 at 91k. On the xhigh REASONING stream at 91k expect 37-39 t/s
rem    (n10/p0.5 buys only 5.6%% there - not worth 898 MiB).
rem  - UD-IQ4_XS (13.3 GiB): quality tied with Q4_K_M (perplexity 6.596 vs 6.535;
rem    the GSM8K half of that comparison is unaudited - see the guide),
rem    FASTER everywhere measured, and 8%% cheaper per token in energy.
rem    Measured ceilings for a window whose every byte still lives on the card:
rem    -c 180224 text-only WITH n10/p0.5; the full native -c 262144 ONLY with
rem    --spec-type none (1,360 MiB of board VRAM left at load; filled to
rem    218,233 tokens it keeps 755 MiB, 1,041 short of the 1,796 reserve -
rem    needs no graphical session on the card at all); -c 163840 with vision
rem    at n4/p0.75.
rem    122880 + vision is the daily default, chosen to leave a desktop room.
rem    Filling a window is the only test that counts - a short
rem    probe at 262144 with the drafter on looks fine and delivers 8 t/s on a
rem    91k document.
rem  - Q4_K_M (15.4 GiB): the quality leader, by one metric and narrowly; +mmproj
rem    at -c 122880 (~0.3 GiB of slack - room for a near-idle desktop, no more).
rem  - the BF16 mmproj costs 1,138 MiB of VRAM and 0%% of decode: measured at
rem    90,862 tokens of fill, projector on vs off, 0.04-0.09%% apart.
rem  - KV cache: q8_0 costs +0.309%% perplexity, q4_0 +0.693%% (measured
rem    2026-08-23, super-linear in bits). q8_0 is what these recipes ship.
rem  - reasoning_effort is THE wall-clock knob: ~4x medium on one long
rem    authoring task, 1.8x across a 175-prompt benchmark suite where the
rem    three levels scored within 1.6 points of each other. llama-server
rem    ignores per-request effort, so it must be set here.
rem ----------------------------------------------------------------------------
rem  USAGE:  serve-qwen.bat [low^|medium^|xhigh] [context] [1-8]
rem          arg1 = reasoning effort, default xhigh (quality-first)
rem          arg2 = context override (otherwise each choice's safe default)
rem          arg3 = model choice 1-8, skips the menu (for scripts)
rem          No args: menu below, auto-picks [1] after 8 seconds, so an
rem          unattended restart never blocks on a human.
rem ----------------------------------------------------------------------------
set EFFORT=%1
if "%EFFORT%"=="" set EFFORT=xhigh
if /i "%EFFORT%"=="low" goto effort_ok
if /i "%EFFORT%"=="medium" goto effort_ok
if /i "%EFFORT%"=="xhigh" goto effort_ok
echo Unknown reasoning effort "%EFFORT%". Usage: serve-qwen.bat [low^|medium^|xhigh] [context] [1-8]
pause
exit /b 1
:effort_ok
set CTXARG=%2
set PICK=%3

set MODELS=C:\path\to\your\models
set MMPROJ=%MODELS%\mmproj-Qwen3.8-27B-BF16.gguf

rem  the drafter is PER PICK (it is a load-time flag; llama-server ignores the
rem  per-request speculative fields). 2026-10-08: SPEC_N3 is the n/p sweep's
rem  best for UD-IQ4_XS with one slot and serves picks 1 and 2 (header note).
set SPEC_N3=--spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0
rem  August: n10/p0.5 is the greedy-code peak on both files but costs 898 MiB,
rem  so it rode the text-only pick. Superseded 2026-10-08, kept for comparison,
rem  used by no pick. SPEC_SAFE (n4/p0.75) likewise since 2026-10-09: picks 4-6 take SPEC_N4.
set SPEC_FAST=--spec-type draft-mtp --spec-draft-n-max 10 --spec-draft-p-min 0.5
set SPEC_N4=--spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0
set SPEC_SAFE=--spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

if not "%PICK%"=="" goto pick_%PICK%

echo.
echo  Qwen3.8-27B - pick a model (effort: %EFFORT%):
echo    [1] UD-IQ4_XS   + HI-RES vision  -c 122880  n3/p0  ~3 GiB slack (DEFAULT)
echo                                          ~83-86 t/s answer tokens on code,
echo                                          ~65 at a 91k fill, 37-39 on xhigh
echo                                          reasoning at 91k (n4/p0.75 / n10/p0.5);
echo                                          screenshots to ~4K detail (1080p=2.0k /
echo                                          1440p=3.6k / 4K=8.2k ctx tokens each);
echo                                          full xhigh cycle + ~10 shots fit
echo    [2] UD-IQ4_XS   text-only  -c 180224  n3/p0  at n10/p0.5 (August): 93.9 t/s
echo                                          at -c 32768, -3.1%% at -c 180224 (73.94,
echo                                          582 MiB slack), the measured text-only
echo                                          ceiling WITH n10/p0.5: little on screen
echo    [3] UD-IQ4_XS   text-only  -c 262144  DRAFTER OFF - required, not optional
echo                                          full native - needs the card free of any
echo                                          graphical session;
echo                                          no drafter = ~43 t/s short-context floor
echo    [4] Q4_K_M      + vision   -c 122880  n4/p0  highest measured quality,
echo                                          reference config
echo                                          ~0.3 GiB slack - a near-idle desktop only
echo    [5] UD-Q4_K_XL  text-only  -c 122880  n4/p0  quality LOSES to Q4_K_M
echo                                          (perplexity +2.3%%) and is 1 GB bigger:
echo                                          gains nothing
echo    [6] UD-Q4_K_M   + vision   -c 122880  n4/p0  measured worse than plain Q4_K_M
echo                                          (perplexity +1.8%%) - skip
echo    [7] DFlash2 build                     ~46 t/s on real code (loses to built-in MTP)
echo    [8] NVFP4 HIGH                        ~48-54 t/s  dequant fallback, lower quality,
echo                                          +13%% J/token vs IQ4_XS
echo.
echo    t/s above are ANSWER tokens (thinking off) at short context unless said
echo    otherwise. Reasoning tokens run ~1.7x slower on the same server, and
echo    decode falls with DEPTH: 86 shallow / 80 at 28k / 65 at 91k (answer),
echo    51 / 47 / 37 (reasoning). Acceptance actually RISES with depth
echo    (0.80 -^> 0.92): the cost is KV reads, not the drafter failing.
echo.
choice /C 12345678 /T 8 /D 1 /M "Choice"
goto pick_%ERRORLEVEL%

:pick_1
set MODEL=%MODELS%\Qwen3.8-27B-UD-IQ4_XS.gguf
set CTX=122880
set MM=--mmproj "%MMPROJ%" --image-min-tokens 1024 --image-max-tokens 10580
rem  August: n4/p0.75 on purpose - this pick already spends 1,138 MiB on the
rem  projector, and the n10 flags would take another 898 MiB out of the slack
rem  that absorbs the desktop (measured 1,179-1,669 MiB). They also buy least
rem  here - 5.6%% on the xhigh reasoning stream this default decodes at depth.
rem  2026-10-08: n3/p0 (n/p sweep, this file at xhigh): shallow reasoning 1.11x
rem  and answers 1.12x n4/p0.75; ~150 MiB lighter than n4, so the slack grows.
set SPEC=%SPEC_N3%
goto launch
:pick_2
set MODEL=%MODELS%\Qwen3.8-27B-UD-IQ4_XS.gguf
set CTX=180224
set MM=
rem  August: text-only, the 898 MiB was spare here, so it took the greedy-code
rem  peak n10/p0.5 (+12%%). 2026-10-08: n3/p0 instead - at this launcher's
rem  sampler it beat n10/p0.5 on shallow reasoning (1.11x against 0.96x of
rem  n4/p0.75) and on answers (1.12x against 1.05x), and needs ~1 GB less.
set SPEC=%SPEC_N3%
goto launch
:pick_3
set MODEL=%MODELS%\Qwen3.8-27B-UD-IQ4_XS.gguf
set CTX=262144
set MM=
rem  drafter OFF is required, not a preference: with the n-max 4 / p-min 0.75
rem  drafter on this window needs ~24.9 GiB and 2,364 MiB was measured living
rem  in system RAM at that setting.
set SPEC=--spec-type none
echo.
echo  WARNING: -c 262144 fits ONLY with the drafter off. Filled to 218,233
echo  tokens it keeps 755 MiB, 1,041 short of the 1,796 reserve. At load:
echo  23,216 MiB derived, 1,360 MiB of board VRAM left - 309 MiB SHORT once a
echo  worst-case desktop takes its measured 1,669 MiB. WITH the n-max 4 /
echo  p-min 0.75 drafter on it does not fit at all: 2,364 MiB spilled text-only,
echo  3,502 MiB with the projector loaded and 8.0 t/s on a 91k-token fill.
echo  A browser or agent
echo  web UI will spill it either way. Run this with NO graphical session using
echo  the card: close everything that draws, drive the screen from a second card,
echo  or log in from another machine. Unplugging the monitor does not do it.
echo.
goto launch
:pick_4
set MODEL=%MODELS%\Qwen3.8-27B-Q4_K_M.gguf
set CTX=122880
set MM=--mmproj "%MMPROJ%" --image-min-tokens 1024
rem  ~0.3 GiB slack with the projector loaded: no room for the n10 flags here.
set SPEC=%SPEC_N4%
goto launch
:pick_5
set MODEL=%MODELS%\Qwen3.8-27B-UD-Q4_K_XL.gguf
set CTX=122880
set MM=
rem  kept in the menu as an answered loser: perplexity 6.682 vs Q4_K_M's 6.535.
set SPEC=%SPEC_N4%
goto launch
:pick_6
set MODEL=%MODELS%\Qwen3.8-27B-UD-Q4_K_M.gguf
set CTX=122880
set MM=--mmproj "%MMPROJ%" --image-min-tokens 1024
rem  answered loser: perplexity 6.654, +1.8%% over plain Q4_K_M at the same size.
set SPEC=%SPEC_N4%
goto launch
:pick_7
rem  DFlash2 needs its own build (llama.cpp PR 27342) and a separate 1.14 GB
rem  drafter; best on real code measured 46.5 t/s at n-max 2, below MTP's 57.9.
call serve-qwen-dflash2.bat %EFFORT%
exit /b
:pick_8
rem  NVFP4 on Ampere runs through a software fallback: slower, +4.4%% perplexity,
rem  +13%% J/token. Kept for Blackwell users and for the record.
call serve-qwen-nvfp4.bat HIGH %EFFORT%
exit /b

:launch
if not "%CTXARG%"=="" set CTX=%CTXARG%
if not exist "%MODEL%" (
    echo Model not found: %MODEL%
    pause
    exit /b 1
)
echo Serving %MODEL%
echo   effort=%EFFORT%  ctx=%CTX%
echo   spec=%SPEC%
llama-server.exe ^
    -m "%MODEL%" ^
    %MM% ^
    --alias qwen/qwen3.8-27b ^
    -c %CTX% ^
    -ngl 99 ^
    --parallel 1 ^
    --load-mode none ^
    --api-key dummy ^
    -ctk q8_0 -ctv q8_0 ^
    %SPEC% ^
    --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 ^
    --reasoning-preserve ^
    --chat-template-kwargs "{\"reasoning_effort\":\"%EFFORT%\"}" ^
    --jinja ^
    --host 127.0.0.1 ^
    --port 1234

NVIDIA 16 GB · 5080 / 4080 / 4070 Ti S / 5060 Ti

The file: unsloth UD-Q2_K_XL — 9.83 GB, 9.154 GiB resident. The window: -c 65536. The flags: the universal set plus the n4/p0 drafter der: its 13,982 MiB was measured at n4/p0.75, and p-min costs no memory, so the window still fits; on the four files the 2026-10-08 n/p sweep ran, p-min 0.75 at n-max 4 decoded shallow reasoning at 0.87–0.97 of p-min 0’s speed with one slot and read 0.90–1.03 of it after a 57,540-token prefix, and n3/p0, about 150 MiB lighter, scored best with one slot at xhigh; neither is measured on this file or card (§06). VRAM at the top of this window: 13,982 MiB meas. Room left for your desktop: 606 MiB against the 14,588 MiB this page budgets for a 16 GB card — enough for a light desktop, not for a browser full of tabs; drop to -c 49152 if you need more. The effort levels it offers: low and medium. xhigh is not offered: its measured thinking appetite is 61,500–75,800 tokens against a 65,536 window, so it would fit on its shortest runs and truncate on its longest — which is worse than a clean no, because you would not know which you got. Expected speed: 25–50 t/s drafter-off, derived from the bandwidth formula in §04 der; the drafter adds to that. No speed on a 16 GB card is measured on this page and none can be, from a machine that owns one 24 GB card. The memory figure is measured and transfers to any card.

Why not UD-Q3_K_XL: it needs 16,906 MiB at -c 65536 — more than the whole card — and ~16,218 MiB at -c 49152, still 1,142 MiB over what a 16 GB card has to give (§08). UD-Q2_K_XL at the same -c 65536 gives twice the window inside less memory and ties the 4-bit reference on accuracy (§08).^C15

:: best daily experience on 16 GB, MEASURED 2026-08-25: the 2-bit file, not the
:: 3-bit one. UD-Q2_K_XL at -c 65536 and n4/p0.75 needs 13,982 MiB; UD-Q3_K_XL
:: at the same window needs 16,906 and does not fit. See section 08's requirement table.
:: Q4-with-CPU-offload preserves more fidelity but crawls at ~8-12 t/s (section 07).
:: file: unsloth/Qwen3.8-27B-GGUF (UD-Q2_K_XL, 9.83 GB = 9.154 GiB)
llama-server.exe -m Qwen3.8-27B-UD-Q2_K_XL.gguf  --alias qwen/qwen3.8-27b
  -c 65536  -ngl 99  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0
                                :: 13,982 MiB MEASURED on the reference 3090 at this exact
                                :: window at n4/p0.75 (p-min costs no memory) - not arithmetic. Against
                                :: the 14,588 MiB this page budgets for a 16 GB card that
                                :: leaves 606 MiB for your desktop. Drop to -c 49152 if
                                :: you run a browser, or --spec-type none to buy back ~1.3 GiB.
                                :: A >131k window cannot fit resident on 16 GB - that needs
                                :: the Q4 offload path (section 07)
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0  :: p-min 0 derived; start at 4 and
                                :: sweep on your own card (section 06): the optimum moved with
                                :: content and hardware in every run measured here
                                :: On the 3090 (2026-10-08) p-min 0 read higher than 0.75
                                :: at n-max 4 on shallow reasoning on every file swept, and
                                :: n3/p0 scored best at one slot at xhigh: include both
                                :: (unmeasured on this card and file)
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"  :: xhigh NOT OFFERED:
                                :: measured appetite 61.5-75.8k tokens against a 65,536 window,
                                :: so it would fit on short runs and truncate on long ones. You
                                :: would not know which you got - see section 09's ceiling table

:: FLAGS DELIBERATELY OMITTED: --parallel 2 (settled 2026-08-25: +22.0% aggregate,
:: -35.4% per slot with the drafter on - slower for one user),
:: -ctk q4_0 (+0.693% perplexity; UD-Q2_K_XL at -c 65536 fits with 606 MiB
:: spare, so the trade buys no window this recipe needs).

llama-server.exe -m Qwen3.8-27B-UD-Q2_K_XL.gguf --alias qwen/qwen3.8-27b -c 65536 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"

12 GB class · RTX 3060 / RTX 5070 / Arc B580

The honest framing first: this class runs a 27B model poorly in every configuration measured or derived here, and if the exact model is not a requirement, a 14B-class Qwen at 4-bit fits resident and is several times faster. The file, if it must be this model: Q4_K_M with most layers on the processor. The window: -c 112640. The effort levels it offers: all three fit the window — but the recipe ships medium, and here the binding constraint is wall clock, not the window. How much wall clock depends entirely on what you ask for. At 6–8 t/s an ordinary xhigh answer — 2,217 tokens of thinking, averaged over 175 benchmark prompts — takes about 5 minutes. A complete build, the one task measured wanting 61,500–75,800, takes about three hours against one for medium. The three-hour figure is the cost of a complete build and overstates ordinary work by roughly thirty times.^C4 Ask questions at xhigh freely here; save the overnight run for whole deliverables. Expected speed: 6–8 t/s, derived, and set by your system memory bandwidth rather than by the graphics card. Room left for your desktop: nearly all of the card — almost none of the model is on it. For the Swift-1.5 files' 12 GB windows — which files fit, and how large a window each holds — see Appendix A.

:: honest framing: this class runs the 27B poorly in every configuration. If the
:: exact model is not required, a 14B-class Qwen at Q4 fits resident and flies.
:: If it must be this model: Q4 with most layers on CPU (~6-8 t/s; speed is set by
:: your system RAM. The ~6-8 t/s assumes TWO sticks of DDR5 in dual channel
:: (~90 GB/s); one stick = single channel = half the bandwidth = half the t/s).
:: file: lmstudio-community/Qwen3.8-27B-GGUF (Q4_K_M — on a bandwidth-bound
:: offload path the smaller Q4 file means fewer GB re-read per token, and the
:: measured perplexity in section 08 favours it too)
llama-server.exe -m Qwen3.8-27B-Q4_K_M.gguf  --alias qwen/qwen3.8-27b
  -c 112640  -ngl 28            :: ~28 of the 65 offloadable layers (64 + the output
                                :: layer — section 11); raise until <500 MB VRAM is free
  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0   :: CPU layers need ~10 GB of real RAM
:: no MTP flags here on purpose: speculation is UNMEASURED on the CPU-offload path.
:: The measurement that would justify adding them is one paired probe with and
:: without --spec-type draft-mtp at this -ngl, on your machine. It costs ten minutes.
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"  :: unlike the 16 GB card
:: this is NOT a window limit — the 112k window (mostly in system RAM) holds xhigh's
:: 61.5-75.8k appetite fine. It is a PATIENCE limit: at ~6-8 t/s an xhigh answer runs
:: ~3 HOURS against ~1 h for medium (section 09). Switch to xhigh for unattended runs.
:: Arc B580: use the Vulkan build (same SYCL caveat as the Arc Pro cards - they are
:: all Battlemage, Intel's Arc B series)

llama-server.exe -m Qwen3.8-27B-Q4_K_M.gguf --alias qwen/qwen3.8-27b -c 112640 -ngl 28 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"

Intel Arc Pro B70 · 32 GB

The file: Q6_K — 22.4 GB, 20.9 GiB resident, near-8-bit quality, which the extra VRAM affords. The window: -c 229376 with the drafter, or the full 262144 with --spec-type none. The effort levels it offers: all three; the window holds xhigh's appetite several times over. Expected speed: 18–26 t/s, derived from bandwidth and unmeasured on this card's chip generation (Battlemage — Intel's Arc B series). VRAM at the top of the window: about 30.5 GiB derived with the drafter off. Room left for your desktop: not measured — every figure for this card is derived, so treat the last gigabyte as unverified.

:: VULKAN llama.cpp build - currently beats SYCL on Battlemage, Intel's Arc B series,
:: which this card belongs to (SYCL has known perf
:: bugs there; IPEX-LLM is archived — do not use it). Re-benchmark after driver updates.
:: file: lmstudio-community/Qwen3.8-27B-GGUF (Q6_K, 22.4 GB = 20.9 GiB)
llama-server.exe -m Qwen3.8-27B-Q6_K.gguf  --alias qwen/qwen3.8-27b
  -c 229376  -ngl 99            :: NOT the full 262144 with a drafter loaded: at the
                                :: MEASURED 45,056 B/token the native window costs ~11 GiB of
                                :: KV, not the 8.5 the q8 arithmetic suggests, and Q6_K + MTP +
                                :: that needs more than 32 GB. For the FULL 262144 here, add --spec-type
                                :: none and drop the MTP line below (~30.5 GiB). Both figures
                                :: are DERIVED from the 3090's constants — unmeasured on Battlemage
  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0   :: verify KV-quant support in your build
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0  :: p-min 0 derived; start at 4, sweep (section 06)
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"  :: all three fit, with room to spare at THIS window (229,376).
                                :: An ordinary xhigh answer averages 2,217 tokens of thinking
                                :: (175 benchmark prompts). Only a COMPLETE BUILD has been
                                :: measured wanting 61,500-75,800 - four August samples of one task,
                                :: section 09 - and even that leaves ~153,000 here for prompt
                                :: and history. Earlier editions of this comment quoted 122,880
                                :: and ~47,000: copied from the 24 GB recipe, never re-done
                                :: against this window. SUPERSEDED 2026-08-25
:: FLAGS DELIBERATELY OMITTED: --parallel 2 (+22.0% aggregate, -35.4% per slot with
:: the drafter on, measured 2026-08-25 — see section 03's axis table).

llama-server.exe -m Qwen3.8-27B-Q6_K.gguf --alias qwen/qwen3.8-27b -c 229376 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"

Intel Arc Pro B50 · 16 GB

The file: UD-Q2_K_XL — 9.83 GB, 9.154 GiB resident. A 4-bit file does not fit, and at 224 GB/s this card is very slow as soon as any layer has to sit on the processor, so every byte must be resident. The window: -c 65536. VRAM: 13,982 MiB meas on the reference 3090 at this window at n4/p0.75 (p-min costs no memory, so n4/p0 needs the same der) — a memory requirement is a property of the model and the flags, so it transfers to this card even though no speed here does. Room left for your desktop: 606 MiB of the 16 GB. The effort levels it offers: low and medium. xhigh is not offered, on the same window constraint as the NVIDIA 16 GB recipe: 61,500–75,800 tokens of appetite against a 65,536-token window, so it fits on short runs and truncates on long ones. Expected speed: 8–12 t/s der from this card’s 224 GB/s and the bandwidth formula in §04 — no Intel card was ever measured for this page.

UD-Q3_K_XL does not fit on this card with the drafter on (§08).^C15

:: Q4 does not fit, and at 224 GB/s any layer left on the CPU is very slow - so run
:: the 2-bit file with every byte on the card, Vulkan build. MEASURED 2026-08-25:
:: UD-Q2_K_XL at -c 65536 needs 13,982 MiB; UD-Q3_K_XL at that window needs 16,906
:: and does not fit at all. Section 08 has the full requirement table.
:: file: unsloth/Qwen3.8-27B-GGUF (UD-Q2_K_XL, 9.83 GB = 9.154 GiB)
llama-server.exe -m Qwen3.8-27B-UD-Q2_K_XL.gguf  --alias qwen/qwen3.8-27b
  -c 65536  -ngl 99  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0
                                :: 13,982 MiB measured, leaving 606 MiB of a 16 GB card for
                                :: your desktop. --spec-type none buys back ~1.3 GiB if you
                                :: would rather have headroom than the drafter. A >131k window
                                :: cannot fit resident on this card
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0  :: p-min 0 derived; start at 4, sweep (section 06)
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"  :: xhigh NOT OFFERED: 61.5-75.8k
                                :: appetite against a 65k window - a WINDOW limit (section 09)

llama-server.exe -m Qwen3.8-27B-UD-Q2_K_XL.gguf --alias qwen/qwen3.8-27b -c 65536 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"

DGX Spark · GB10, 128 GB unified memory

The file: Q4_K_M — 16.5 GB, 15.4 GiB resident. With 128 GB of unified memory, capacity is not the constraint; bandwidth is, at 273 GB/s, so the smallest strong file wins. The window: the full -c 262144. The effort levels it offers: all three — but note the wall clock: the reference 100k-token run derives to about 2–2.8 hours here, so xhigh is an unattended-run setting. Expected speed: 10–15 t/s, derived from bandwidth. Room left for your desktop: not applicable — this machine has 128 GB of memory shared between the chip and the system.

:: llama-server (ARM64 Linux build — no .exe here), same stack as every other card.
:: Bandwidth (273 GB/s) is the ceiling, so the smallest high-quality file wins:
:: Q4_K_M (16.5 GB) over UD-Q4_K_XL (17.6 GB) is ~6% more t/s on a bandwidth-bound
:: box — and the measured perplexity favours it too (section 08).
:: file: lmstudio-community/Qwen3.8-27B-GGUF (Q4_K_M, 16.5 GB = 15.4 GiB)
llama-server -m Qwen3.8-27B-Q4_K_M.gguf  --alias qwen/qwen3.8-27b
  -c 262144  -ngl 99            :: 128 GB unified: full native context is trivial (~11 GiB of
                                :: KV at the MEASURED slope with the drafter on, not the 8.5
                                :: the arithmetic suggests)
  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0  :: p-min 0 derived; start at 4, sweep
                                :: (section 06) — UNMEASURED on GB10; the 3090 gained +45% on
                                :: real code at n4/p0.75; there, p-min 0 read higher than
                                :: 0.75 at n-max 4 on shallow reasoning (2026-10-08)
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"  :: window holds it; wall clock
                                :: is the cost — ~2-2.8 h for the 100k reference run (section 07)
:: faster vendor alternative: Blackwell-native NVFP4 on vLLM (~1.5x BF16 speed, FP8 KV,
:: 92-97% accuracy) — vllm serve unsloth/Qwen3.8-27B-NVFP4 --max-model-len 131072.
:: llama.cpp cannot load that release (FP8 lm_head); switching stacks buys the FP4 path

llama-server -m Qwen3.8-27B-Q4_K_M.gguf --alias qwen/qwen3.8-27b -c 262144 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}"

Intel Core Ultra iGPU · Arc B390-class

The file: Q4_K_M — 16.5 GB, 15.4 GiB resident; every gigabyte re-read per token hurts at 153.6 GB/s. The window: -c 65536 as an interactive default. This is not a memory limit — an integrated graphics chip borrows system memory — but a practical one. The effort levels it offers: low and medium at this window; xhigh is not offered here for two independent reasons, and both need naming because they have different fixes: the 65,536-token window cannot hold a 61,500–75,800-token thought (raise -c to fix that), and at 5–7 t/s an xhigh answer takes three to four hours (only patience fixes that). Expected speed: 5–7 t/s, derived, and it scales with your memory bandwidth. Room left for your desktop: ample — an integrated graphics chip borrows system memory — but leave the processor and operating system some working memory.

:: llama-server.exe, VULKAN build (SYCL has known perf bugs on current Intel GPUs;
:: IPEX-LLM is archived). RAM is the spec that matters: the B390 tier (Core Ultra X9
:: 388H) ships dual-channel LPDDR5X-9600 = 153.6 GB/s, up to 96 GB shared.
:: Intel's formal Arc-branding floor for B390/B370 is 7,467 MT/s — slower RAM relabels
:: the iGPU as generic "Intel Graphics" — and other Core Ultra configs may ship slower
:: RAM or ONE module (single channel = half the bandwidth = half the t/s). The ~5-7 t/s
:: scales with YOUR bandwidth via section 05's formula; the model fits easily.
:: file: lmstudio-community/Qwen3.8-27B-GGUF (Q4_K_M, 16.5 GB = 15.4 GiB)
llama-server.exe -m Qwen3.8-27B-Q4_K_M.gguf  --alias qwen/qwen3.8-27b
  -c 65536  -ngl 99             :: interactive default, NOT a memory limit: an iGPU borrows
                                :: system RAM, ~57% by default. To raise it: Intel Graphics
                                :: Software >=25.26.1602.2 with driver >=32.0.101.6974 -> the
                                :: GPU's General tab -> "Shared GPU Memory Override" slider
                                :: (~87%; up to ~93% on B390/B370 with current Arc Pro
                                :: drivers, RAM-dependent) -> reboot. Needs >=10 GB RAM, select
                                :: Core Ultra systems. With 96 GB RAM even -c 262144 fits
  --parallel 1  --load-mode none  -ctk q8_0 -ctv q8_0   :: verify KV-quant support in the Vulkan build
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0  :: p-min 0 derived; start at 4, sweep
                                :: (section 06); UNMEASURED on iGPUs — drop the spec flags if
                                :: a paired probe shows no gain on your machine; on the 3090,
                                :: p-min 0 read higher than 0.75 at n-max 4 on shallow reasoning (2026-10-08)
  --jinja --host 127.0.0.1 --port 1234
  --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"  :: xhigh NOT OFFERED, for TWO
:: separate reasons with two separate fixes: 61.5-75.8k of appetite against this 65k
:: WINDOW (fix: raise -c, you have the RAM), and 3-4 HOURS per answer at 5-7 t/s
:: (fix: nothing but patience). Raise -c and effort together for overnight runs only.
:: possibly-faster vendor alternative: Intel's OpenVINO Model Server on the iGPU with
:: OpenVINO/Qwen3.8-27B-int4-ov (OVMS LLM quickstart; OpenAI-compatible /v3 endpoint) —
:: Intel tunes that stack for its own silicon; benchmark both if the t/s matters

llama-server.exe -m Qwen3.8-27B-Q4_K_M.gguf --alias qwen/qwen3.8-27b -c 65536 -ngl 99 --parallel 1 --load-mode none -ctk q8_0 -ctv q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 --jinja --host 127.0.0.1 --port 1234 --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"

Standardized industry metrics

What each recipe costs in electricity, in the units the industry compares on. Read the two plain-language conclusions first: turning the drafter on is the single cheapest energy decision on this page — it cut the energy per token by 2.52× at n10/p0.5 and 2.17× at n4/p0.75 (energy at the n3/p0 and n4/p0 the cards carry now is not measured) — and a slower setting is a more expensive setting in exactly the same proportion, because while the model is generating, this card draws essentially the same power whatever it is doing, so the joules per token are just those watts divided by the tokens per second. The whole energy win is throughput. Nothing here saves power; it saves time at the same power, which is the same thing arriving by a less interesting route.

Instrumentation tier for every figure below: in-band GPU board power — read from the card's own sensor rather than from a meter at the wall (NVML, nvidia-smi --query-gpu=power.draw). Inside the number: the graphics chip, its memory, the board's voltage regulators and fans. Excluded and never measured on this machine: power-supply conversion loss, the processor, system memory, drives, chassis fans, the display, and any datacentre overhead. Do not call any of these figures system power or wall power, and do not divide an electricity bill by them.

The idle pair, dated. Settled, with little on screen, first 60 seconds of each idle window discarded (2026-08-23 power matrix): 29.9 W with no server running, 34.1 W with the model resident and answering nothing. A resident model cost +4.2 W in that pair — small, and in the physically sensible direction. The 34.1 W figure is what the idle-subtracted columns below remove.^C5

Sustained board power is a range, not a constant. Across 2.5 hours of drafter-off benchmark work the board drifted 305.5 → 341.1 W (+11.7%) at constant throughput, constant 77–79 °C and constant memory clock, tracking its own SM clock from 1,453 to 1,606 MHz. Decode at one request at a time is limited by memory bandwidth, so the extra clock bought no tokens and cost about 6% more energy per token. Plan with the band — 306–341 W drafter-off, 338–345 W on the long drafter-on authoring runs, 302–339 W across the short power-matrix arms — or with your own run's mean.^C6

Conditions, and the ways each row differs from the recipe it prices. All four energy arms ran on 2026-08-23 at -c 32768 with a 1,458-token prompt and 700 generated tokens, thinking off, no projector, three requests per arm, 500 ms power sampling, 100% coverage. Row [1] and [2]: the recipes serve at 122,880 and 180,224 tokens of window with the projector loaded. Window allocation does not change joules per decode token; prompt depth does, and the depth series is its own set of arms with its own shallow reference point — 3.919 J/token at a 1.5k fill, 4.453 at 28k, 5.598 at 91k. Read that series against its own 3.919, not against this row's 3.743: the two shallow points sit 4.7% apart in energy and 6.4% apart in throughput, both outside the 2.9% floor, because they are different arms on different prompts. The projector is decode-free (0.04–0.09% at a 91k fill), so it changes nothing here. Rows [1] and [2] both decoded faster than the matched sweep did at the same flags — 91.3 against 83.5 for row [1] (9.3% apart) and 106.2 against 93.9 for row [2] (13% apart). This gap is real, still open, and larger than any single session can see. Three twenty-probe runs of the identical configuration (UD-IQ4_XS, n10/p0.5, -c 32768, -ctk/-ctv q8_0, greedy, 700 tokens, one server per run, one warmup discarded, RTX 3090 at 350 W, build 10502, all 2026-08-28) gave session means of 88.76, 77.35 and 77.15 t/s. Runs 1 and 2 do not overlap at all (ranges 86.22–94.00 and 73.44–84.21); their means differ by 14.8%. The spread within any one session — standard deviation (typical spread from the mean) 2.70–3.22 — is much smaller than the gap between sessions. The band is set between sessions, not within one; twenty probes in one sitting cannot measure it. Within a session, throughput correlates (tracks together) with mean draft length — r = +0.683, +0.597, +0.790, +0.861, +0.894, +0.240 across six sessions, every one positive. Between sessions, draft length cannot be the mechanism: six further sessions (six probes each, greedy, same configuration, one server per session torn down between, all 2026-08-28) produced bit-identical acceptance and mean-draft-length sequences probe for probe (acceptance 0.6128, 0.6032, 0.6185, 0.6036, 0.6398, 0.6459; mean draft length 7.792, 7.683, 7.603, 7.754, 8.080, 8.117) while session means spanned 75.71–78.65 t/s. Across all nine sessions measured on 2026-08-28 the means span 75.71 to 88.76 t/s, a 16.6% range, while the drafter does precisely the same work in each. What is ruled out for the between-session gap: the workload (bit-identical), the drafter (bit-identical), the llama.cpp build (every compute library dated 2026-08-19, unchanged), the core clock (run 2 held a mean SM clock 49 MHz higher than run 1 while delivering 12.9% less throughput), and heat (75–83 °C in every session). The between-session cause is unmeasured.^C26 Any single speculative-decoding throughput figure on this page is a point drawn from a band at least this wide; plan accordingly.^C25 Use these rows for energy ratios, which survive because every arm in the matrix shares the same spread. Row [4]: this arm ran with the drafter off while the recipe ships n4/p0 since 2026-10-09 (n4/p0.75 before). On UD-IQ4_XS n4/p0.75 cut joules per token by 2.17× (8.104 → 3.743), so recipe [4]'s own figure at that drafter is expected near 4.1 and is unmeasured; no file was measured for energy at n4/p0 — the printed 8.853 is this file's no-drafter cost and is the right number for comparing files, not for pricing this recipe. "Wh per 700-token answer" means exactly that: every arm generated 700 tokens and stopped, so these are not complete-answer figures. Complete-answer energy, for one real authoring task, is in §09. Net columns subtract the dated 34.1 W loaded idle over the covered seconds and are derived; the campaign's own runner subtracted 31.0 W instead, a uniform shift of about 1% that changes no ranking.

Decode energy for the anchor and both Swift files at n4/p0.75, each in the answer regime (thinking off) and the reasoning regime (thinking on), measured 2026-10-04. Loaded idle for this sweep: 12.97 W (anchor, n4 drafter, -c 32768, first 60 s dropped, 240 s window); no-server idle: 13.17 W. One sweep, 2026-10-04; not comparable with the August rows above.

Conditions, and the ways each row differs from the recipe it prices. All six arms ran on 2026-10-04 at -c 32768 with one prompt (a JavaScript red-black tree) sent three times per load; the first send is discarded, and pass 2 repeated pass 1’s text. Each request generated 700 tokens; each pass’s two kept requests took 15.3–28.4 s. The thinking-on arms set no effort level; the template’s default renders the same prompt as xhigh, the cards’ setting. Power sampling at 500 ms, coverage 1.0, means of 2 passes n=2. The board sat at its software power cap in 97–100% of active samples in every window, so J/token restates throughput. J per prompt token is not measured: the long prefills sit in the discarded rows (§15). In-band GPU board power (NVML); PSU, wall and PUE excluded. Net columns subtract the 12.97 W loaded idle over the covered seconds.

Both Swift files cost more energy per decode token than the anchor in the reasoning regime (n4/p0.75, one prompt, n=2), in step with fewer accepted draft tokens per verify pass while thinking — 1.479 and 0.974 against the anchor’s 2.684 (acceptance 0.837 and 0.772 against 0.901).

Energy per setting: the J/token table

Every knob this page recommends on, priced in energy, so that a setting chosen for speed can be checked for electricity too. The pattern underneath all of it is simple and was verified with no residual left over: joules per decode token equals the watts drawn while decoding divided by the decode tokens per second. That qualifier matters — the mean-watt column in the August table above spans each request's whole window, including a short low-power prefill, so it runs 6–11% below the decode figure and dividing it gives the wrong answer. Anything that raises throughput at constant power lowers energy per token in the same proportion. That is why the drafter is an energy feature and why depth is an energy problem.

Five things about how this model is built explain almost every recommendation in §03. It stores far less memory per token of context than its size suggests, so long contexts are cheap. Its context limit is 262,144 tokens, and what you will actually run out of is card memory, not model capability. It thinks before answering by default, and that thinking comes out of the same pool as your prompt. Vision is optional at start-up and costs memory rather than speed. And it wants a specific sampling setup that some clients quietly override, which causes the model to repeat itself. Everything below is those five facts with their numbers.

One more fact governs speed rather than settings, and it is worth having in plain words before the arithmetic: with speculation off, a bigger file is a slower file. To write each token, the card has to read the whole model out of its memory once. A file that is 2 GiB larger takes proportionally longer to read, every single token, forever.

The drafter breaks that rule: with speculation on, a smaller file is not necessarily a faster file. meas Turn speculation on and the ordering can invert: UD-IQ4_XS goes 42.34 → 86.91 t/s (2.05×) while the smaller UD-Q2_K_XL goes 45.66 → 77.01 (1.69×) — so the file that is faster drafter-off is slower drafter-on, by 12.9%. The draft head is quantized along with the model, so a more damaged file drafts worse and gets less back from speculation. The window moves the ordering too (§08). Since every recipe here ships with the drafter on, “smallest that holds its quality” is the right rule for memory and the wrong one for speed.

• Hybrid attention. Most of its 64 layers use Gated DeltaNet, a linear-attention design whose memory use stays constant no matter how long the context gets; only 16 of the 64 are standard attention layers (and those use grouped-query attention with 4 KV heads). The result: the KV cache costs just 64 KiB per token at fp16 — 4.0 GiB at 122k context with an 8-bit cache, 7.5 GiB unquantized — where a dense 27B storing KV in every layer would need exactly 4× as much. Those are the arithmetic, and arithmetic is a floor: what a real server allocates measured 15–29% steeper (§05). Huge contexts are cheap in memory here, and speed falls off gently as they fill rather than collapsing.
• Native 262,144 context. No rope or YaRN tricks are needed below that; the limit you will actually hit is VRAM. The model card lists extension to 1M tokens via YaRN, which is out of scope here: the KV cache alone would run about 34 GiB even at q8_0, by the same arithmetic.
• Thinking is always on, and defaults to xhigh. Reasoning tokens share the context window with your prompt and your output, so a 100k-token thinking run needs a window bigger than 100k (§09). This is also why every speed number on this page states its token regime.
• Vision is optional at serve time — and it costs context, not speed. The BF16 mmproj projector occupies 1,138 MiB of VRAM once loaded — not the 0.867 GiB its file weighs. Two independent measurements (2026-08-23, same machine, UD-IQ4_XS) of the resident cost agreed to the megabyte, and a follow-up pair at 91k of depth reproduced the same 1,138 twice more. At the measured window slope that is about 26,500 tokens of 8-bit window with the coding drafter loaded, 29,900 without it (§05). What it does not cost is speed: at 90,862 tokens of fill, decode with and without the projector differed by 0.04–0.09% (§06) — the only question the projector asks is about window. Serving text-only converts vision you are not using directly into context, but not all the way to the native maximum: with the drafter on, UD-IQ4_XS's measured text-only ceiling is 180,224, and 262,144 fits only with the drafter off, and then only just (§03 prints both variants).
• Official thinking-mode sampling: temperature 1.0 · top-p 0.95 · top-k 20 · min-p 0. Clients that silently send temperature 0 — aider does — cause repetition loops. Override them; §14 shows how, per agent.

The arithmetic that predicts decode speed

Generation speed comes down to one ratio: how fast your card can read its memory, divided by how much it must read per token. Every token requires reading the entire model — about 16.5 GB for a 4-bit K-quant, 14.25 GB for UD-IQ4_XS. So take your card's bandwidth in GB/s, divide by your file's GB, and multiply by an efficiency constant, and that is your speed ceiling in tokens per second. That constant is format-specific: on CUDA it measures about 0.70 for K-quants and about 0.65 for IQ-quants (not a single ~0.7 for both). You can tell which family a file belongs to from its name: a name containing IQ — UD-IQ4_XS, UD-IQ3_XXS, UD-IQ2_S — is an IQ-quant, so use 0.65; a name containing K without IQ — Q4_K_M, UD-Q3_K_XL, UD-Q2_K_XL, Q6_K — is a K-quant, so use 0.70. The independent re-measurement measured 0.649 on UD-IQ4_XS — 936 GB/s ÷ 14.25 GB = 65.7 t/s theoretical against 42.6 measured — and Q4_K_M lands at 0.700: 936 ÷ 16.5 = 56.7 against 39.7 measured. Those are one file of each family: on this card with the drafter off, bartowski’s Swift IQ4_XS and IQ3_XXS read 0.71 and 0.58 of the same arithmetic der (42.9 and 44.1 t/s, 2026-10-04, beside the anchor’s 42.7 in the same sweep, which is 0.65 of its own arithmetic; §06), so the constant belongs to the file as much as to its family — §08’s 2.9-bit K-quant, UD-Q2_K_XL, reads 0.48 against the K-quant 0.70 (45.66 t/s at 9.83 GB; C39). This one calculation predicts every row of the matrix in §07, and it doubles as a fault detector: land far below your format's constant at short context and something is wrong (§11).

A 24 GB card has two ceilings, not one, and the difference between them is the most expensive misunderstanding on this page. The first ceiling is how large a window you can set and still have every byte of it live on the card — fully resident, in the words of §02: about 131,000 tokens for the reference configuration. The second is how large a window you can set before the server refuses to start at all: about 213,000. Between those two numbers the server starts happily, answers a short prompt at full speed, and then collapses to a fifth of that speed the first time somebody actually fills the window. Set your window under the first ceiling, not the second, and never trust a ceiling that was checked with a short prompt. The rest of this section is the memory arithmetic that tells you where your own first ceiling is.

The second law of this section: speed against context is a cliff, not a curve. It is flat while the weights and the cache fit in VRAM, and it collapses once anything spills across the PCIe bus — with the magnitude set by how much spilled: about 10× when the weights file itself lands in system memory; 1.1–2× from a 1.9 GB spill of weights answering a short prompt (§11); and 2.5–3.8× once the spilled pages are cache that a deep prompt actually reads (the table below). What decides the size is not how much spilled but how often the spilled pages are touched. Short probes on the reference 3090, showing where an unfilled window stops being fast — the table after the chart shows where a filled one collapses:^C33

That corollary now has a measured face. Probed 2026-08-22 with Q4_K_M, a reconstruction (the record does not name the file; q8_0 KV, drafter on, projector loaded — the 1 GiB-larger UD-Q4_K_XL shifts every ceiling about 30k tokens lower, while dropping --mmproj frees 1,138 MiB and raises them about 26,500) by stepping -c upward with short temperature-0 wall-clock probes (prefill included, first request after each load, completion tokens divided by the stopwatch time of the whole request): every reading from -c 122880 through -c 212992 lands between 45.1 and 55.4 t/s, all above the sweep's 41.6 t/s pass line; 217,088 falls just under that line (41.2 t/s); at 221,184 the short probe reads 19.5 t/s.^C33 But nvidia-smi tells the other half of the story: dedicated VRAM hits its ~24 GB ceiling already at -c 131072 (measured 24,052 MiB). Past that point the KV cache is overcommitted, and the run stays fast only while a short prompt leaves the deep pages untouched. So: about 131k is the practical fully-resident ceiling — fast even when a long run really fills it, though it sits right at the limit, which is why the recipe ships 122,880 — while up to about 213k will load and answer a short prompt. Past that ceiling, a window that is actually filled does not degrade gradually — it collapses, 30.6 t/s on -c 131072 → 8.0 t/s on -c 262144 at 90,886 tokens (UD-IQ4_XS, 2026-08-23), a 3.82× penalty — as the table below measures.

Prefill collapses with it — 819 → 183 t/s at 90.9k, 1,085 → 408 at 29.8k — while draft acceptance is unchanged (0.907 against 0.849 at 90.9k), which rules out any drafting explanation: this is purely the KV cache being read across the PCIe bus. The practical reading: a reader who trusted the shallow ceiling gets 26% of the promised speed the first time they open a 90k-token document — 8 t/s where the fully resident -c 131072 window does 30.6 at the same 90,886-token depth, greedy^C29 — and the one line the server log carries about it is the start-up no-fit warning (§11), easy to miss.^C31 Never label a window resident or safe on the strength of a short probe; fill it once, deeply, before you believe it.

One caveat on the table's own numbers: every cell in it is a single decode probe fired immediately after its prefill, and that is the one moment this card is not at its boost clock — the ±25% band declared in §01 and measured in §11. Re-measured under the cooled protocol, the resident 90.9k row reads 36.6 t/s rather than 30.6, about 20% higher. The spilled column was taken the same way and carries the same uncertainty, so read the penalty ratios as approximate. Nothing about the verdict moves: a 2.5–3.8× collapse is an order of magnitude outside a ±25% band.

The VRAM budget table

The chart above says where a short probe stops being fast; the collapse table says where a filled window fails; this budget table says why.^{C33 C8} Budget from llama-server's own dedicated-VRAM counter, not from arithmetic: a fixed base plus a per-window-token slope, fitted across 17 fully-resident server loads (2026-08-23, same machine, UD-IQ4_XS), reproducing all 17 to within 127 MiB. (Those 17 are the fully-resident subset of 26 loads in total; the other nine were overcommitted configurations, which the model does not describe. Later measurement rounds loaded further configurations that have never been checked against it.)

Conditions: RTX 3090, 24,576 MiB, UD-IQ4_XS, q8_0 KV, -ngl 99, --parallel 1, llama.cpp build 10502. Worked example derived — §03's the default recipe with vision at -c 122880, if you put the n-max 10 coding drafter on it: 13,232 + 1,138 + 1,008 + 898 + (122,880 × 45,056 ÷ 1,048,576 = 5,280) = 21,556 MiB of server-process VRAM, leaving 3,020 MiB (2.95 GiB) of board VRAM for the desktop and slack. Drop the 898 and the same row keeps 3,918 MiB (3.83 GiB) — which is one reason §03’s vision recipe never carried n-max 10: the wider drafter setting fits here, it just spends the slack that absorbs a desktop measured anywhere from 1,179 to 1,669 MiB. Since 2026-10-08 the recipe ships n-max 3 / p-min 0, lighter than the n-max 4 the 1,008 above was measured at (§06).

Two rules follow, and both change how you read every ceiling in this guide. First, budget on the server's own report, not on the board VRAM total — the three counters and what each can see are in §02. The nvidia-smi board VRAM figure includes your window manager and your browsers; the campaign watched one identical configuration read 474 MiB apart on the board VRAM — about 11,000 tokens of window — with nothing changing but what was on screen, while llama-server's own number moved by at most 127 MiB. That spread is the whole explanation for "same command, different ceiling". Second, the drafter is a line item, not a freebie. At -c 163840 with vision, its fixed and per-token costs together book roughly 1.8 GiB — more than the vision projector. On a pure prose workload, where speculation was measured worth only about 1.16× (§06), that is the easiest 1.8 GiB on the card to reclaim: --spec-type none.

With those constants in hand, what fits on a 24 GiB card by file and window — the KV column stated as the q8_0 arithmetic floor, so add 15% (drafter off) or 29% (drafter on) for what a server really allocates:

Two footnotes the table earns. First, the q4_0 KV flag. A q4_0 K/V cache costs +0.693% perplexity against fp16 — 6.6413 against 6.5956 (2026-08-23, same corpus and flags as the q8_0 run) — where q8_0 costs +0.309%. The degradation is super-linear in bits: the second halving of the cache costs slightly more than the first (+0.383% on top of q8_0), which is the shape the mechanism predicts and the reason the cut stops paying. 0.69% is small — smaller than the perplexity gap between the two 4-bit files this page recommends (0.93%). The mechanism concern — that at long context every new token attends over hundreds of thousands of perturbed entries, and retrieval gets unreliable at exactly the 200k-plus lengths a q4_0 cache would enable — has been tested on UD-Q2_K_XL, where q4_0 scored identically to f16 and q8_0 at depths up to 119,435 tokens (§15); the same test on the 4-bit files at the 200k-plus lengths the mechanism targets has not been run. q8_0 remains what every recipe ships because it already buys the context ceilings here; q4_0 is a window-extension trade to take knowingly (§08 carries the row and its caveat). Second, prefill: these budgets make a 200k-plus prompt fit, and the table below measures what filling one costs, across a 60× span of prompt lengths on a fully-resident -c 131072 server:

Conditions: UD-IQ4_XS, -c 131072 fully resident, q8_0 KV, drafter at n-max 4 / p-min 0.75, greedy decoding, no projector, prompts prefixed with a unique identifier so the prefix cache (§02) cannot be reused (2026-08-23). The energy column comes from a separate 2026-08-23 arm at 1.5k / 28k / 91k of fill, not from these five prompts, and its depths are close but not identical. Use about 1,000 t/s as the planning figure: prefill decays only 25% over a 60× longer prompt, because the quadratic term lives on just 16 of the 64 layers (§04) — the hybrid dilutes it. A genuinely full 100k prompt is about 2 minutes; extrapolating the same slope, 200k lands nearer 5 minutes. Overcommit the window and this number collapses with everything else: 183 t/s at 91k in the correction above.

The Swift files use the same memory per token of context; only their weights differ

Swift IQ4_XS and Swift IQ3_XXS are bartowski’s quantizations of ukisai’s Swift-1.5, a fine-tune of this model with the same architecture (Appendix A). They share the same memory per token of context as the anchor (unsloth UD-IQ4_XS of the base model). For Swift IQ4_XS it is measured across its two text-only deep fills at 139,264 and 159,744 tokens (one sweep, 2026-10-04), where dedicated memory grows 44.0 KiB per token meas — the anchor’s 45,056 B in the budget table above — and the server footprint (dedicated + shared) grows 46.0 KiB per token meas, 0.3% der above the 45.87 KiB one-slot footprint slope of the per-card arithmetic and inside the ±2% tolerance set before the fills ran. For Swift IQ3_XXS the 45.87 KiB slope is itself an August-to-October transplant from the anchor onto knee pick 5 (Appendix A). Only the weights term changes. By file arithmetic (file size less the token embedding, which llama.cpp keeps in host RAM, and the header), Swift IQ4_XS (15,475,951,936 B meas) puts 1,043.3 MiB more weight on the card than the anchor der — 23,291 tokens of window at 45.87 KiB per token der — and Swift IQ3_XXS (12,320,168,256 B meas) puts 2,004.2 MiB less der. Swift IQ4_XS held decode, fully resident, at -c 159744, the largest window filled n=1 (one deep fill, text-only, n4/p0.75, one slot, 2026-10-04; the fill left 1,880 MiB for a desktop between the card’s 24,576 MiB and the server footprint at depth, dedicated + shared der, above this page’s 1,796 MiB threshold (§02); no larger window was tried). Swift IQ3_XXS held decode, fully resident, at -c 262144 in text at one slot n=1 (knee pick 5, 2026-10-04, 257,415 tokens filled at a 415 MiB desktop); that fill left 327 MiB for a desktop on the same basis der, so at this page’s 1,796 MiB threshold its text window is 216,064 der (Appendix A).

The deep fills on Swift IQ4_XS, one sweep, 2026-10-04; not comparable with the August rows above in this section:

Conditions: 2026-10-04, NVIDIA GeForce RTX 3090, q8_0 KV, -ngl 99, --parallel 1, thinking off, greedy decoding, decode the mean of two 400-token probes after the fill; the vision row loads bartowski’s F16 projector and has a 1440p image in flight. Each row is one load n=1. Server footprint is dedicated + shared; board VRAM peak is memory.used with the desktop inside. Fully resident test: fit (dedicated memory at depth + the 855 MiB desktop maximum + the 121.0 MiB spread ≤ 24,576 MiB), spill and knee, as defined under the next table; the vision row is judged against its text twin at the same window. The 139,264 text row is tested for fit only, because no lower text step within 32,768 tokens was loaded.

Ceilings by kind, per Swift configuration, from two October runs that are not comparable with each other or with the August rows above in this section: the Swift IQ4_XS rows are the deep fills of 2026-10-04 (desktops 841–855 MiB), the Swift IQ3_XXS rows the knee sweep of 2026-10-03/04 (desktops 338–539 MiB), so decode is not compared between them. Each row names its scope — file, drafter, projector, slots — and the desktop the fit test assumed (73 desktop readings, 72 loads and one direct reading: max 855 MiB, sd 121.0 MiB, range 258–539 MiB from the knee sweep plus 841–855 from the deep fills plus a 264 MiB direct reading, 2026-10-04). The knee chart is in Appendix A.

Fully resident: the highest loaded window at which every loaded step up to it passes three tests (dedicated memory at depth + the 855 MiB desktop maximum of the deep fills + the 121.0 MiB spread of 73 desktop readings (72 loads and one direct reading) ≤ 24,576 MiB; shared memory at load up by no more than 100 MiB over the next-lower step; decode within 15% of the next-lower step’s); a step with no loaded step of its configuration within 32,768 tokens below it is tested for fit only, unless it is a vision step with a text-only twin at the same window, which it is then judged against for spill and decode; steps between loaded steps were not tested. That allows 976 MiB der for the desktop, not this page’s 1,796 MiB threshold (§02). Judged against 1,796 MiB on the basis of the per-card tables in Appendix A — what each fill left between the card’s 24,576 MiB and the server footprint at depth, dedicated + shared — the Swift IQ4_XS text fill left 1,880 MiB, so its window stays at 159,744, and the vision fill left 1,330 MiB, so its window is 128,000 der; on dedicated memory alone, the counter of the fit test above, the vision fill misses 1,796 MiB by 22. For Swift IQ3_XXS the per-card tables give 216,064 (text) and 195,584 (vision) at one slot and 93,184 per slot for vision at two der. The anchor’s counterpart of knee pick 7 in the same sweep is knee pick 1 (UD-IQ4_XS, vision, n4/p0.75, one slot: fully resident 180,224 meas, decode-verified 219,136 der; Appendix A); the per-card tables have no two-slot text configuration, so the two-slot text row has no 1,796 MiB figure. Decode-verified: the last step whose decode held with the window filled (to about 91%, 98% for knee picks 5 and 7), minus 0.7 GiB at the configuration’s per-token slope, floored to 1,024 — or the native 262,144 where decode held to it, as for knee picks 5 and 7 — at the knee loads’ 258–539 MiB desktops; it is the October launcher’s window, it is fully resident only where the table says so, and it can sit above the fully resident window (Appendix A). Collapse point: the knee, where decode falls 15% below the next-lower step. Where a ceiling kind has no backing measurement, the cell says “not swept”: no knee sweep was run on Swift IQ4_XS. Speed bands are greedy-only (temperature 0); speed at the temperature-1.0 sampler the cards ship is not measured as a band (§15).

The two Swift IQ4_XS windows were filled again on 2026-10-07 in the Pi fine-tune study (window-knee.py, in the order Pi, Swift, Swift, Pi, n=2 per file; Appendix B). Vision at 139,264 held at 45.14 and 45.55 t/s on the same 126,838 tokens (desktop 1,217 MiB, board peak 24,019 MiB, 444 MiB shared: the 2026-10-04 footprint of 22,802 MiB dedicated again der), and text at 159,744 held at 48.36 and 48.51 t/s at a 146,423-token fill (board peak 23,429 MiB, 484 MiB shared) meas. Their absolute t/s are not comparable with the 2026-10-04 rows, and they are still not knee sweeps.

The drafter bill per file (board VRAM at -c 32768, one slot, desktop inside, 2026-10-04): the anchor’s n4 step costs 1,138–1,198 MiB over drafter-off meas n=2, Swift IQ4_XS costs 1,092–1,124 MiB meas n=2, and Swift IQ3_XXS costs 1,092–1,098 MiB meas n=2. The n10 step over n4 costs 898–900 MiB in five of the six loads across the three files meas n=2 (the sixth, Swift IQ4_XS’s second pass, reads 851 on a board counter that includes the desktop), matching the 897.75 MiB expected from six recurrent-state rows at 149.625 MiB each der. The projector bill on Swift IQ4_XS is 1,136 MiB of server footprint at load and 1,470 MiB at depth meas n=1 (one load each; the at-depth figure has a 1440p image in flight; bartowski’s F16 file) — the anchor’s published projector cost, 1,138 MiB meas of llama-server’s dedicated memory for the BF16 projector on UD-IQ4_XS (§05, August 2026), is a separate measurement on another file, another projector file and another day.

The per-token arithmetic for the Swift files (server footprint = weights + a constant per configuration + slope × window, Appendix A): Swift IQ4_XS loads 14,104.4 MiB of GPU-resident weights with the drafter on der and Swift IQ3_XXS loads 11,056.9 MiB der, against the anchor’s 13,061.1 MiB der; the per-token slope of server footprint (dedicated + shared) is shared at 45.87 KiB per token at one slot der (confirmed at 46.0 KiB meas on the Swift IQ4_XS fills, 0.3% above; for Swift IQ3_XXS it is the transplant value itself). See Appendix A for every other card and file.

Six flags belong on every card, and each one is here because a measurement put it here. Put every layer on the graphics card with -ngl 99: writing -ngl 64 looks complete and is not, and it costs about 30% of your speed with no warning anywhere. Serve one request at a time with --parallel 1. Store the memory of the conversation in 8-bit with -ctk q8_0 -ctv q8_0: it halves that memory and costs 0.3% of perplexity. Turn the built-in drafter on with --spec-type draft-mtp: it roughly doubles your speed, changes nothing about what the model writes, and cuts the electricity per token by two and a half times (n-max 10 / p-min 0.5 against no drafter, 2026-08-23; 2.17× at n4/p0.75; not measured at the pairs the cards carry now). Choose --load-mode none if system memory is tight and --load-mode mmap if you restart the server often. And whatever number you end up quoting, say which kind of token you counted, because on this model that alone is worth a factor of 1.7.

• All layers on the GPU, or accept the cliff — -ngl 99. llama.cpp counts the output layer as one more layer than the model has transformer layers, so on this 64-layer model -ngl 64 reads as complete and leaves that output layer — a 5120 × ~151k-vocabulary matrix multiplication — on the processor, running for every generated token. Measured cost: 25.7 against 39.7 t/s on Q4_K_M, and independently 29.84 against 42.31 on UD-IQ4_XS — −29.5% — with no memory signature at all (§11). Partial offload is set by system memory bandwidth, not by the graphics card, so a 5080 and a 4060 Ti are predicted to offload at nearly the same 8–12 t/s der — that band is §04's formula run at system-memory bandwidth, not a measurement: neither of those cards has ever been in this machine, and the only offload figure measured here is the 25.7 t/s above, on this 3090. And this is a dense model — every part of it runs on every token — so the trick of parking a model's rarely-used parts on the processor has nothing to target here.
• One slot — --parallel 1: serving one person, one slot is the faster setting. Some interfaces default to 2, which halves the window each request gets — the KV cache stays the size -c sets and is split between the slots, not allocated twice; a second slot at -c 32768 added 134 MiB of board VRAM meas n=1, not a second 1,248 MiB der cache (2026-08-23, UD-IQ4_XS, q8_0 KV, drafter off, kv_unified off as logged).^C28 Measured 2026-08-25 on UD-IQ4_XS with the drafter on at n-max 10 / p-min 0.5 — then the faster shipped setting — three reps each, first post-prefill probe discarded: --parallel 1 gave 82.98 t/s aggregate and 85.79 per slot; --parallel 2 gave 101.25 t/s aggregate and 55.41 per slot — +22.0% aggregate, −35.4% per slot. Draft acceptance is unchanged across the pair (0.618 → 0.620), so the second slot does not disturb the drafter at all; the whole cost is that each user waits about 55% longer for their own tokens — 12.6 s against 8.2 s for a 700-token answer. Once the drafter is already saving most of the repeated reading of the model, batching has far less left to save (§03).
• 8-bit KV cache — -ctk q8_0 -ctv q8_0, verified twice on two files. Measured wikitext perplexity with the 8-bit cache: 6.5498 against 6.5348 at fp16 on Q4_K_M — +0.23%, inside the error bars — and the re-measurement replicated the effect independently on UD-IQ4_XS at 6.6160 against 6.5956, +0.31%, likewise about one standard error. Half the KV memory for statistically nothing, on both files. It also costs nothing measurable in energy: 8.341 against 8.284 J per decode token, 0.7% apart and well inside the 2.9% floor. The one price that is not nothing is speed: fp16 KV measured 43.19 against q8_0's 42.31 t/s at -c 32768, so 8-bit costs about 2% of decode — when VRAM is genuinely spare, fp16 is marginally the faster cache. The step below is measured too: q4_0 K/V at 6.6413, +0.693% over fp16 (§08).
• The drafter on — --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0 with one slot on the files the 2026-10-08 sweep ran; n-max 4 / p-min 0 with two slots and for the Pi fine-tune at medium. Those pairs scored best per file at the cards’ own sampler (UD-IQ4_XS, Swift IQ4_XS and Swift IQ3_XXS at xhigh; two slots on Swift IQ3_XXS only), and n3/p0 is also this build’s own default. Files, cards and slot counts the sweep did not run carry n4/p0 der, at the n-max 4 their windows were fitted at: p-min costs no memory, and on the four files the sweep ran, p-min 0.75 at n-max 4 decoded shallow reasoning at 0.87–0.97 of p-min 0’s speed with one slot, reasoning and answers at 0.67–0.75 per slot with two (Swift IQ3_XXS), and 0.90–1.03 after a 57,540-token prefix, where on UD-IQ4_XS deep answers it was slightly faster (n4/p0 read 0.973 of it, interval 0.969–0.996). The 262,144-token UD-IQ4_XS window runs with the drafter off; what n-max and p-min mean, and how the pairs were chosen, is in the n and p subsection. Every drafted token is checked by the full model before it counts; on one prompt (build 10502, greedy, thinking off, 2026-10-04) greedy text was not always identical token for token with the drafter on (§08).^C38 Measured here it doubles decode on code and cuts energy per token by 2.52× at n10/p0.5 and 2.17× at n4/p0.75 (2026-08-23; energy at n3/p0 and n4/p0 is not measured). It is not free in memory — 1,008 MiB fixed plus 11.4% of your window, measured at n-max 4 (n-max 3 is about 150 MiB less; §05) — and on the August greedy prose probe it was worth only about 1.16×, the one workload measured where --spec-type none and the memory back was the better answer; the 2026-10-08 sweep did not time prose on its own. The rest of this section is how far it goes and why.
• Load mode — none for tight RAM, mmap for restart-heavy work. Without --load-mode none (which replaces the deprecated --no-mmap) the whole GGUF stays cached in system memory even after the weights are in VRAM — a measured 13,480 MiB for the 4-bit file meas. The cost is a slower model load: measured 10–30 s to first health check across dozens of restarts in the campaign proper, and it is storage-dependent. The independent re-measurement ran about 30 restarts on --load-mode mmap instead and measured warm reloads at 6–10 s with decode unaffected (repeat probes agreed to 0.7%) — because at -ngl 99 every weight lands in VRAM either way, so load mode cannot touch decode speed. So: if you are sweeping flags or restarting dozens of times a day, mmap is the better choice. Every recipe in §03 nonetheless ships none, and that is a deliberate default rather than a contradiction: none is the safe assumption about a machine this page cannot see, because the failure it prevents (system memory exhausted by a cached model file) is worse than the one it causes (a load 20 seconds slower). Change it if you know your machine has the memory to spare. Measured 2026-08-26: none holds a 1,275 MiB working set against mmap’s 13,480, a saving of 12,205 MiB meas, confirmed by two measures agreeing to 3 MiB (§15).
• Say which tokens you counted. This model thinks by default, so a probe labelled "code generation" can spend every timed token reasoning about code and return an empty answer field. On code, answer tokens run about 1.7× faster than reasoning tokens on the same server with the same flags. Every speed row below declares its regime.

Flash Attention: you are already using it, and the recipes depend on it

None of the recipes on this page passes -fa, and none of them needs to. This build documents -fa, --flash-attn [on|off|auto] with a default of auto — so a recipe that omits the flag is not running with Flash Attention off, it is letting the server decide. On this machine it decides on meas, and that is not an inference: with -fa unset and -ctk q8_0 -ctv q8_0 the server loads fine, while the same configuration with -fa off refuses to start. Acceptance also reads identically across auto and on — 0.611 shallow and 0.571 deep, to three decimals — and deep decode differs by 0.1 %.

What Flash Attention actually is. It is a different way of computing attention, not a quality setting. Ordinary attention builds the whole attention matrix in memory and then reads it back, so its memory traffic grows with the square of how much context it is attending over. Flash Attention fuses those steps so the large intermediate is never written out at all. The arithmetic is the same; the memory traffic is far smaller, and the saving grows with depth.

llama_init_from_model: quantized V cache requires flash_attn to be enabled
srv    load_model: failed to create_context with model ...
srv    llama_server: exiting due to model loading error

So should you set it? No — leave it alone. Adding -fa on buys nothing over the default on this card (0.1 % at depth, inside noise), and pinning a flag you have not measured on your own backend removes the one mechanism that would have chosen correctly for you. The reason to know about it is diagnostic, not performance: if a recipe here refuses to load and mentions flash_attn, that is this dependency, and the fix is the KV width, not the flag.

One measurement from this run that is not trustworthy, stated so nobody uses it: the prefill figures. Probes reused a cached prefix, so the server reported a full prompt_n against a near-zero prompt_ms and the resulting “prefill t/s” is meaningless. Only the decode, acceptance, VRAM and load-success results from this arm are reported anywhere on this page.

Every speed this page measured, in one table

Read the band, not a number. The floor is set by how fast the card reads its own memory, and everything above that floor is the drafter, so the right row for you is the one whose content and token regime match your work.

All rows measured 2026-08-21 to 2026-08-23 on the reference RTX 3090 at short context, from the server's own timings. The floor row is also the diagnostic threshold, and it is published as a band rather than a point for that reason: it was established across five contents and both token regimes and holds inside 3.6%.

How speed falls as the prompt grows

Speed drops as the conversation gets longer, gently and predictably, and the drop is about 25–30% between an empty window and a 91,000-token one. It is not a cliff — that only happens when the window overflows the card (§05). What matters far more than the depth is which kind of token you are counting: at the same 91k depth, the answers a reader keeps arrive at 64.8 t/s while the thinking that precedes them runs at 36.6. Plan an agent session on the first number and a long xhigh run on the second.

All three series below are UD-IQ4_XS at -c 131072, fully resident, with prompts prefixed by a unique identifier so nothing is served from the prefix cache. The first column is the one to plan against: it was measured 2026-08-23 under the cooled protocol (§02, documented in §11) — the probe that comes back with the prefill discarded, three probes per depth off the cached prefix — while the other two columns are single probes taken right after their prefill and therefore carry the ±25% band from §01.

The last column is not a measurement of the model: it is the same reasoning-series runs divided the wrong way, tokens ÷ total wall-clock, which buries a 26-second or 105-second prefill in the denominator. It is printed here so that a reader who computes 9.2 t/s on a card that decodes at 47.1 recognises their own arithmetic instead of blaming their hardware. Quote the server's own timings, and quote prefill separately (§05). Why the acceptance columns differ so much: the cooled ladder re-used the campaign's red-black-tree prompt, a textbook algorithm the draft head predicts extremely well, while the middle column generated novel code. Acceptance ranks the content, not the protocol — and, as the next part measures, it does not rank speed at all.

Three things to take from it. First, plan an agent session on 65–70 t/s at 91k of depth — those are answer tokens, what an agent actually receives, and they run about 2× faster than the reasoning stream at the same depth. Second, acceptance rises with depth, 0.80 → 0.92 in the reasoning series and 0.60 → 0.65 in the answer series — two independent confirmations. Decode still falls 25–30% across the same span (86.3 → 64.8 on the cooled ladder), which settles the mechanism: the cost is per-token KV reads growing with depth, not the drafter failing. Depth and acceptance are independent axes. Third, the decline is gentle and it is not a cliff — as long as the window is genuinely resident. Overcommit it and the same depths give 18.6 and 8.0 t/s instead (§05).

The mechanism under the two regimes: mean draft length

Line up the cooled ladder against the reasoning column — same file, same flags, different protocols — and the reasoning side sits a near-constant 1.7× below at every depth: 86.3/51.2 = 1.69, 80.2/47.1 = 1.70, 64.8/35.8 = 1.81 — so call it 1.69–1.81×. A ratio that stable across a 60× span of depth is not clock noise, so it was isolated properly (2026-08-23): one server, one 90,894-token prompt, the same n-max 4 / p-min 0.75 flags, cooled probes, the only change being the chat template:

A 1.69× speed difference produced by nothing but the token stream — and acceptance says nothing about it (0.895 against 0.907, essentially tied). The mean-draft-length column is where the difference actually lives. On uncertain reasoning tokens the p-min 0.75 confidence gate keeps cutting the draft tree short — a mean of 2.99 tokens per pass against 4.31 on the answer stream — so acceptance stays high precisely because the gate is discarding the part that would have missed. That makes acceptance blind to the loss. Acceptance tells you whether a draft was right; mean draft length at a fixed n-max tells you how fast you will go. It is the same fact that makes the highest-acceptance configuration the slowest one in the sweeps below. But mean draft length alone does not rank configurations when n-max differs^C24 — a drafter at n-max 4 / p-min 0.00 reaches 82.73 t/s at acceptance 70.5% and mean draft length 2.803, beating n-max 10 / p-min 0.5 at 76.32 t/s with 4.035, because the longer draft speculates up to ten tokens and discards about half, and every discarded token was still computed. The quantity that ranks these is accepted tokens per unit of drafting work, not accepted length alone (2026-08-28, UD-IQ4_XS, -c 32768, q8_0 KV, three settled probes each after a discarded warmup).

The energy column comes from a different pair of arms on a different day's prompt — 2026-08-23's power matrix, thinking on against thinking off at short context — and is printed beside these rows because it measures the same mechanism on a completely independent instrument: 1.62× in joules per token against 1.69× in throughput. Two instruments, one effect. Note also the floor row: 27.01 t/s thinking-on against 26.95 thinking-off — identical, which confirms once more that content dependence on this page is almost entirely a speculation effect: without a drafter, what you are writing moves decode by at most 3.6%.

The flags cannot buy the difference back. At the same depth with thinking on, --spec-type none gives 27.01, the then-shipped n4/p0.75 gives 36.62 (1.36×) and the August greedy-code peak, n10/p0.5, gives 38.67 (1.43×) — a 5.6% gain where the same flag change is worth 12% on code. So the practical number for the configuration this page shipped by default until 2026-10-08 (n4/p0.75; the n3/p0 it ships now was not measured at 91k, §06): anyone running xhigh over a 91,000-token document should plan on 37–39 t/s, not the 62–70 t/s the code probes advertise. The deliverable arrives at code speed; the thinking that precedes it does not.

What the built-in draft head is

Qwen3.8-27B ships with a small helper network inside the same file — a multi-token prediction (MTP) draft head — that guesses several tokens ahead so the full model only has to check them, which is much faster than generating each one. The standard K-quant GGUF files (§08's table) already include it, and the head's precision is set by whoever quantized the file; in UD-IQ4_XS it is held at 8-bit and 6-bit with F32 (Q8_0, Q6_K and F32, 334.75 MiB meas, read from the file header on 2026-10-04), and bartowski's IQ4_XS and IQ3_XXS quantizations of Swift-1.5, a fine-tune of this model with the same architecture, hold it at 4-bit with F32 (Q4_0 and F32, 227.91 MiB meas).^C35 llama.cpp added support in July 2026, and every guess is verified by the full model before it is accepted, so the drafter is built to change only the speed — though on one prompt on this build (greedy, thinking off) its text was not always identical to the drafter-off text (§08).^C38 It works on any GPU llama.cpp supports.

  --spec-type draft-mtp  --spec-draft-n-max 3  --spec-draft-p-min 0.0
  :: the best score with one slot on the three files swept at xhigh at the cards' sampler
  :: (2026-10-08); n-max 4 / p-min 0 with two slots and for the Pi fine-tune at medium;
  :: unswept files take n4/p0, derived, not measured. The best n-max moves with the file, the
  :: slot count and the REGIME: greedy code favours longer drafts (section 06, n and p).
  :: And it is not free: 1,008 MiB + 11.4% of your window at n-max 4, about 150 MiB per
  :: step of n-max, +898 MiB at n-max 10; p-min costs nothing (sections 05 and 06).

How much speed it adds depends heavily on hardware and workload — published reports range from +33% to +145%, and one RTX 5090 tester saw 148 average / 203 peak t/s on the NVFP4 tiers. On the reference 3090, generation went from 40.3 (the 08-21 build's cold-start baseline; the canonical 08-22 baseline is 39.7–39.9) to 58–60 t/s — +45% at the default-ish settings — and on the original benchmark prompt, tuning pushed it to 81.7 t/s, +105%, a figure a 2026-08-22 re-sweep appeared to overturn and a 2026-08-23 matched sweep reproduced to the decimal once the token regime was pinned. The warning all these numbers carry: benchmark on your own machine. That 5090 tester found n-max 4 fastest with 6–8 slower; this 3090 agreed on real code yet kept climbing to n-max 10 on its near-ideal prompt. The optimum moves with hardware and content, so there is no universal best value.

DFlash2, and why it lost

There is a second way to draft, and on this machine it was slower than the one already inside the file. DFlash2 predicts a whole seven-token block in a single pass, then picks the most plausible sequence out of the candidates it considered for each position. Like the built-in head, every draft is verified before it is accepted, so the output is identical to running the model alone. The costs are real: a separate 1.14 GB drafter to download (incoai/Qwen3.8-27B-DFlash2-GGUF, about 1.2 GB of VRAM) and — until llama.cpp PR 27342 is merged — building llama.cpp yourself from that branch. On its best benchmark prompt it reached 69–77 t/s; on realistic code its best was 46.5 t/s at n-max 2, below the built-in head's 57.9. That is the whole verdict.

:: shared flags in every run below:
llama-server -c 32768 -ngl 99 --parallel 1 -ctk q8_0 -ctv q8_0 --jinja --port 1235

:: row 1 — baseline: just the model, no speculative flags
  -m Qwen3.8-27B-Q4_K_M.gguf

:: row 2 — built-in MTP, n-max 4 / p-min 0.75 — the everyday pick in August (the head ships inside the main GGUF):
  -m Qwen3.8-27B-Q4_K_M.gguf --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

:: row 3 — MTP at n-max 10 / p-min 0.5 — the peak on answer tokens († = measured warm-cache):
  -m Qwen3.8-27B-Q4_K_M.gguf --spec-type draft-mtp --spec-draft-n-max 10 --spec-draft-p-min 0.5

:: row 4 — DFlash2 (separate 1.14 GB drafter via -md; needs the PR 27342 build):
  -m Qwen3.8-27B-Q4_K_M.gguf -md Qwen3.8-27B-DFlash2-Q4_K_M.gguf --spec-type draft-dflash --spec-draft-n-max 7

:: rows 5-6 — NVFP4 files with their embedded MTP head (same MTP flags):
  -m Qwen3.8-27B-NVFP4-MTP-HIGH.gguf --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75
  -m Qwen3.8-27B-NVFP4-MTP-VERY-LOW.gguf --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

:: what each speculative flag means:
::   --spec-type            which drafting method: draft-mtp (built-in head) or draft-dflash (external drafter)
::   -md / --model-draft    path to the external drafter GGUF (DFlash2 only)
::   --spec-draft-n-max     how many tokens to draft per step, one after another - the main tuning knob, sweep 2-12 on your machine
::   --spec-draft-p-min     drafter confidence floor 0-1: the draft ends at the first guess whose top-1 probability (renormalised over the head's top 10) is below it; 0 never ends a draft (section 06)

llama-server -c 32768 -ngl 99 --parallel 1 -ctk q8_0 -ctv q8_0 --jinja --port 1235 -m Qwen3.8-27B-Q4_K_M.gguf -m Qwen3.8-27B-Q4_K_M.gguf --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75 -m Qwen3.8-27B-Q4_K_M.gguf --spec-type draft-mtp --spec-draft-n-max 10 --spec-draft-p-min 0.5 -m Qwen3.8-27B-Q4_K_M.gguf -md Qwen3.8-27B-DFlash2-Q4_K_M.gguf --spec-type draft-dflash --spec-draft-n-max 7 -m Qwen3.8-27B-NVFP4-MTP-HIGH.gguf --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75 -m Qwen3.8-27B-NVFP4-MTP-VERY-LOW.gguf --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

That table says two things. First, speculation is real free speed: at the then-shipped everyday flags (n4/p0.75), decode measured 57.9 t/s on realistic code, and the temperature-1.0 xhigh sweep sustained 48.8 t/s across a 73,000-token thinking run — putting the 100k reference run at roughly 34 minutes, down from 45. Second, the NVFP4 rows show that the quantization and the drafter interact: speculative decoding makes the GPU check several drafted tokens at once, and that batch-checking is exactly the work NVFP4's software fallback does slowest. So on cards older than Blackwell, pair speculation with normal K-quants, not NVFP4.

n and p in plain words, and the best pair for each launcher pick (measured 2026-10-07/08)

On the files this sweep ran: n-max 3 / p-min 0 with one slot on UD-IQ4_XS, Swift IQ4_XS and Swift IQ3_XXS at xhigh; n-max 4 / p-min 0 with two slots busy (Swift IQ3_XXS, the one file run that way) and on the Pi fine-tune at medium. Each had the highest mean of shallow and after-prefix reasoning-token ratios for its file at the cards’ own temperature-1.0 sampler, among the settings timed at both depths (two slots: the best per-slot speed of three settings). On shallow reasoning tokens against n4/p0.75 in the same session, n3/p0 read 1.111 on UD-IQ4_XS, 1.189 on Swift IQ4_XS and 1.097 on Swift IQ3_XXS; after a 57,540-token prefix 1.061, 1.052 and 1.007, intervals that include 1. With two slots busy n4/p0 read 1.337 per slot (second pass only, one load per setting). On the Pi file n4/p0 read 1.038 of n3/p0, and 1.073 after the prefix der. The lead is clearest on shallow reasoning tokens: on answer tokens n4/p0 read the same as n3/p0 or higher (UD-IQ4_XS, Swift IQ4_XS), and after the prefix several longer settings read as high as n3/p0 or higher with overlapping intervals, n10/p0.5 among them on UD-IQ4_XS (on the Pi file it matched n3/p0 there, 1.021, but not the pick n4/p0, 1.073). Greedy code is where longer drafts lead most clearly (below). Files, cards and slot counts the sweep did not run take n4/p0 der, keeping the n-max 4 their windows were fitted at: p-min costs no memory, and at n-max 4 p-min 0.75 decoded shallow reasoning at 0.87–0.97 of p-min 0’s speed with one slot on all four files here, reasoning and answers at 0.67–0.75 per slot with two on Swift IQ3_XXS, and 0.90–1.03 after the prefix, where on UD-IQ4_XS deep answers it was slightly faster (n4/p0 read 0.973 of it). Efforts the sweep did not run keep each file’s pick, unmeasured there, except the Pi file at xhigh, where a later follow-up favoured n-max 3 on thinking tokens (Appendix B). No pick needs more memory than the drafter its window was filled at, and p-min costs nothing. Measured 2026-10-07 17:05 to 2026-10-08 10:04, 138 server loads meas.^C42C43C44

To copy: --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0 with one slot on those three files; --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 with two slots and for the Pi fine-tune at medium. The cards in §03 and Appendix B carry them already, and the per-pick table links each one.

n-max is how many tokens the helper may guess before the full model checks them. The built-in head guesses one token at a time, one small pass per guess, and from the second guess on it builds on its own last guess, so each later guess is right less often. It always guesses its own top pick, whatever n-max is: from the same starting point an n3 draft is the first three tokens of the n10 draft. By the source, with p-min 0 every step guesses exactly n-max tokens except near the end of a reply; measured on the Pi file’s reasoning tokens, 2.980 guesses per check at n3/p0 and 9.869 at n10/p0, close to n-max meas.

p-min is a confidence floor that can end a draft early. The confidence is the head’s probability for its top pick, renormalised over its ten best candidates at temperature 1, so your request’s sampler never touches it. The draft ends at the first guess below the floor (that guess is dropped, its pass already spent; the first guess is checked too). The test is strictly “below”, so p-min 0 never ends a draft. p-min moves where a draft is cut, never which tokens are guessed.

Both are load-time flags, and only n-max costs memory. llama-server silently ignores a request’s speculative.n_max and p_min (disabled since PR 22838): asking for n_max 1 on an n10/p0.5 load still drafted 8.65 tokens per check against 8.78 meas n=1, so every pair needs its own server start. Each +1 of n-max keeps one more recurrent-state row per slot, 149.625 MiB here. This build’s defaults are n-max 3 and p-min 0.00. The semantics come from llama.cpp’s source at commit 0adcc3bb5 (§15); the chips mark what was measured.

A wrong guess costs time, never quality. The full model checks every guess in one pass, keeps them up to the first it disagrees with and adds one token of its own. Greedy, a guess survives only if it is the full model’s own top token, so the text is identical in principle, though not bit for bit on this build.^C38 At temperature 1.0 a guess survives with exactly the probability the full model gives it, so the output distribution is preserved in principle. A wrong guess costs its own pass, the passes of every later guess in that step and a row each in the check. From the source one expects sampling to favour shorter drafts than greedy decoding, because even a correct top-pick guess is rejected whenever the full model samples another token; this sweep did not isolate that, because its greedy probe also changed the prompt and turned thinking off. The larger difference it did measure was between reasoning text and answer text at the same sampler: at n10/p0 Swift IQ4_XS kept its third guess 0.258 of the time on reasoning tokens and 0.539 on answers, and on answers n4/p0 read higher than n3/p0 there (1.099 against 1.062). On the sweep’s one greedy probe every setting, drafter off included, gave the same text per file; quality was not scored at the new settings.

How often each guess survives, the curve under every result below: the share of draft steps whose first, second, third… guess was kept, from llama-server’s /metrics counter spec_decode_num_accepted_tokens_per_pos_total, at n10/p0, shallow, both passes pooled meas:

The Pi file ran at medium, the others at xhigh, so effort and file differ together; a later follow-up ran the Pi file at xhigh, where n3/p0 read ahead of n4/p0 on reasoning tokens (1.080, Appendix B). The greedy row is one 256-token prompt. In the Pi file’s n10/p0 reasoning probes about 15% of the text was answer text.

Reading it. On reasoning tokens at xhigh, Swift IQ4_XS and UD-IQ4_XS keep their third guess 0.258 and 0.296 of the time against the Pi file’s 0.475; on answers the files are close. At n4/p0 the fourth guess survived 0.414 of steps on the Pi file’s reasoning and 0.198, 0.218 and 0.211 on Swift IQ4_XS, UD-IQ4_XS and Swift IQ3_XXS meas, consistent with n-max 4 being best on the Pi file and 3 on the one-slot xhigh files — a correlation; the cost of one guess was not measured. Survival barely depends on n-max (the Pi file’s first three positions: 0.806, 0.637, 0.500 at n3/p0). And p-min 0.75 cuts reasoning drafts short: at n-max 4 on UD-IQ4_XS it drafted 1.312 tokens per check and kept 1.048 (acceptance 0.799), against 3.975 and 1.605 at p-min 0 (acceptance 0.404) — higher acceptance, fewer tokens per check (above). After the prefix every position survived more often on reasoning tokens (UD-IQ4_XS fourth guess at n4/p0: 0.344 against 0.218) and less often on answer tokens (0.374 against 0.446), with the content changed too.

“With p-min 0, shouldn’t n-max 4 or 10 be faster than 3?” Under greedy decoding it often was: in the 2026-08-23 greedy re-measurement on novel JavaScript, n10/p0 decoded 87.3 t/s against n3/p0’s 82.0 (below). At the cards’ temperature-1.0 sampler with thinking on it was not, because each guess costs a head pass and a row in the check whether it survives or not, and late guesses rarely survive. On the Pi file’s reasoning tokens n10/p0 kept more tokens per check than n3/p0 (3.133 against 1.932) but paid for about ten guesses every step (9.869 against 2.980), its fifth to tenth guesses surviving only 0.273 down to 0.063 of the time, and it decoded 0.877 as fast meas. p-min 0.75 fails the other way, cutting drafts that would often have survived (the UD-IQ4_XS figures just above). For these files at this sampler the best cap sat between the two, at 3 or 4.

What the flags cost in memory, server VRAM in MiB meas:

Board peak minus the desktop before the load, median of single-slot loads at -c 81920, --cache-ram 0, no projector; not the launcher windows’ footprints. From n2 on each step adds about 150 MiB (n3 to n10 on the Pi file: 1,054 against 1,047 predicted); p-min moved nothing beyond tens of MiB of noise. Turning the drafter on at all cost about 1 GB at this window (drafter off to n2/p0: 1,010 to 1,077 MiB on the three files), part of it the draft cache that grows with -c (§05); the Pi n1 median includes a load taken while the reference machine was in use. On Swift IQ3_XXS the n3 step was not resolved: its two n3/p0 loads read 15,731 and 15,831 MiB against 15,823 at n4 on every n4 load, so the n3 step is not visible there (n4 to n5: +148, as predicted).

The results, file by file. Decode-speed ratios inside one session against the block’s reference (above 1 = faster than the reference), paired 95% interval in grey der. Reasoning: thinking on at the pick’s effort, 6 prompts × 768 tokens × 2 passes; answers: thinking off, 4 × 384 × 2; after the prefix: 3 reasoning and 2 answer questions about 57,540 tokens of source code. The repeat block re-timed leading settings in fresh server starts against its own reference loads; the same setting and seed gave identical text, so it repeats the timing, not the sampling. Highlighted: what the launcher now runs.

bytkim’s Qwen3.8-27B-pi IQ4_XS at medium (pick [P]), against n3/p0. Only n4/p0 is above 1 on both reasoning columns in the main block, and it repeats. n4/p0.25 ties it (1.058 against 1.060 on the reasoning mean, repeat block) at the same n-max, so the pick is n4/p0, about 1.15× the old card drafter (1.038 ÷ 0.900, no interval) der.

Swift-1.5 IQ4_XS at xhigh (pick [9]), against n4/p0.75. n2/p0 leads shallow but drops to 0.932 after the prefix; n3/p0 has the best reasoning mean in both blocks (1.121), with n4/p0.25 in the 3% tie band (1.105). On answers alone n4/p0 reads higher, intervals overlapping. Drafter off read 0.782 on reasoning. One reference spread passed the pre-registered 3% line (main block, pass 2, after the prefix: 3.48%).

Qwen3.8-27B UD-IQ4_XS, the anchor, at xhigh (picks [1] [2] [4]), against n4/p0.75. n3/p0 has the best reasoning mean in both blocks (1.086, 1.087). n10/p0.5 is slower on shallow reasoning and answers and reads higher after the prefix (1.079 against 1.061), intervals overlapping. On deep answers n4/p0.75 was faster than n4/p0 in both blocks (n4/p0 0.973, two prompts). Drafter off read 0.788 on reasoning.

Swift-1.5 IQ3_XXS, one slot, xhigh (picks [5] [7]), against n4/p0.75. n3/p0 leads (reasoning mean 1.052; n4/p0 1.015, n5/p0 0.961); its after-prefix interval includes 1, and n2 was not run.

Two slots, both busy, at pick [8]’s shape (123,904 per slot, projector loaded, no image): per-slot decode, second pass only, one load per setting and pass, no repeated reference, no interval der; the grey figures are ratios to n4/p0.75:

Pass 1 is not quoted: its desktop changed between loads (1,094, 602 and 125 MiB); pass 2 ran at 159 MiB throughout. Both passes, in reversed order, put n4/p0 first (over n3/p0 by 1.135 in pass 1 and 1.128 in pass 2), but with one load per setting that gap is about the size of this rig’s level change between server starts (about 13%, §03); the gap to n4/p0.75 is well beyond it. The pre-registered 3% pass rule voids both challengers, whose ratios move with the pass-1 reference. The deep column is not a clean reading: the two slots’ deep probes decoded at very different speeds in every load and neither reused the cached prefix. Pick [6] (text only) was not run and inherits this result. Why two slots prefer n-max 4 was not measured.

The decision for each pick of the October launcher der. Pick numbers are the October launcher’s, not the menu’s rows; each row links the page’s card for that configuration, some at a different window:

No window was re-filled at the new drafters; the argument is memory only (the same or a smaller n-max everywhere but [P], whose window was filled at n-max 4; p-min free), and no block had an image in flight. The page’s cards follow the same rule: the UD-IQ4_XS, Swift IQ4_XS and one-slot Swift IQ3_XXS cards, menu row 2 and the August listing’s picks 1 and 2 now carry n3/p0, the two-slot Swift IQ3_XXS cards and the Pi card n4/p0; the Q4_K_M, UD-Q2_K_XL, 16 GB and non-3090 cards, the August listing’s picks 4–6 and Appendix A’s other drafter-on cells were not swept and, since 2026-10-09, carry n4/p0 der: the n-max 4 their memory was measured with, and p-min 0, which costs no memory and at n-max 4 read higher than 0.75 on shallow reasoning on all four files swept (0.75 read 0.87–0.97 of it with one slot, 0.90–1.03 after the prefix).

How it was measured. One RTX 3090 (driver 596.36), build 10502. Each block ran its pick’s launcher arguments plus --cache-ram 0 --metrics at -c 81920, one slot, no projector (two-slot block: 123,904 × 2 with the projector), --load-mode none, q8_0 KV, the launcher’s sampler (temperature 1.0, top_p 0.95, top_k 20, min_p 0), medium for the Pi file and xhigh for the rest. Per load: a 20 s settle, a discarded warm-up, the greedy probe, 6 reasoning prompts (an ISO-8601 parser with tests, a time-zone interval merge with tests, a two-train word problem, an S3-sync design in prose, an off-by-one fix as a diff, a red-black tree in JavaScript), 4 of them with thinking off, and, on the depth subset, a 57,540-token prefix of four repository source files, prefilled once, and 3 + 2 questions about it. Rate = Σ generated tokens ÷ Σ decode time. Two passes per block with different seeds (100+i, 200+i), pass 2 in reverse order. In the one-slot blocks, reference loads ran at the start, middle and end of each pass and each load was divided by its own session’s reference (a pass splits into sessions at gaps over 30 min; shallow reasoning reference spread within a session 0.10–0.66%, one Pi session holding a single reference load); the two-slot block had one load per setting per pass and no repeated reference. Paired bootstrap over prompts within a session (B 2,000, seed 42), on 6, 3, 4 and 2 prompts for the four columns: read both after-prefix intervals as ranges. On the Pi file 8–17% of the reasoning probes’ text was answer text, depending on the setting. Never set an absolute t/s from this sweep beside another sweep’s: this rig has two speed levels about 13% apart between server starts (§03, §06). Instruments: §15.

What this does not show. Anything past the 57,540-token prefix, so not the launcher’s 139,264–262,144 windows; an image in flight; other efforts (the Pi file at medium, and at xhigh only n3/p0 and n4/p0 in a later follow-up, Appendix B; 2026-10-07’s ×1.154 was also at xhigh; the rest only at xhigh); n2 on Swift IQ3_XXS, n2 or n5 with two slots, or p-min between 0 and 0.25 (n4/p0.25 tied n4/p0 on the Pi file and sat in Swift IQ4_XS’s tie band: p-min 0 was never beaten outside a tie band, which does not mean every p-min above 0 hurts); quality, agent wall time or energy at the new settings; other files, slot counts, cards or builds (§15). Reading older text: where a table or sentence measured before 2026-10-08 says “shipped default”, “the cards’ drafter” or “the recipe this page ships”, it means n4/p0.75 (n10/p0.5 on the August text-only rows), and those measurements keep their conditions.

Tuning the drafter: what to set n-max and p-min to, and why

2026-10-08: the answer to “what should I set” moved, and it is in the n and p subsection above: on the files that sweep ran, n-max 3 / p-min 0 with one slot, and n-max 4 / p-min 0 with two slots and for the Pi fine-tune at medium, measured at the cards’ own temperature-1.0 sampler, thinking on and off.^C42C43 What follows is the August evidence, measured greedy, mostly on code and mostly with thinking off, and its findings hold for that regime: there n-max 10 / p-min 0.5 led — about +12% over the then-shipped n4/p0.75, 898 MiB extra (~20,900 tokens of window), and the cheapest setting in energy at 3.210 J per decode token against 3.743 at n4/p0.75 and 8.104 with no drafter — and on reasoning tokens at depth the wider drafter recovered only 5.6% (38.7 against 36.6 t/s at 91k). On the August greedy prose probe, --spec-type none was the better answer. One qualification: a cap-only sweep at the recipe §03 shipped then (2026-08-26) measured 74.13 t/s at n-max 4, 73.52 at n-max 6 (−0.8%) and 75.59 at n-max 8 (+2.0%) against a run-to-run band of 0.77% — nothing like the ~12% the matched sweep reports, so treat the wider drafter as a setting to measure on your own content rather than one to assume. The advice is in the “So what should you set?” box at the end of this part; the space between here and there is the evidence, because the measurements went wrong twice on the way and both mistakes are ones any careful person could repeat.

Two knobs control drafting behaviour, and both mattered more than the defaults suggest.

The comparison table above used the settings each method's authors recommend. Then the campaign swept --spec-draft-n-max across both methods, same prompt, temperature 0, warm cache — a different run state from the cold-start table, so absolute numbers shift a few t/s in either direction: this sweep's baseline reads 39.8 against the table's 40.3, and comparisons belong inside one chart, not across them. On a warm cache and a near-ideal benchmark prompt (baseline 39.8 against the cold-start table’s 40.3 — compare within one chart, not across them), the built-in head kept gaining past the recommended n-max 4, peaking at 81.7 t/s at n-max 10 / p-min 0.5, and DFlash2 peaked at n-max 4–5 (77.6 / 77.5) rather than its own recommended 7 (74.0):

Three findings, all measured 2026-08-21:

• On this prompt — and only on prompts that draft like it — the tuned built-in head won outright: n-max 10 with p-min 0.5 gave 81.7 t/s, +105% over baseline, beating DFlash2's best (77.6 at n-max 4–5). No drafter download and no source build — but not free of VRAM. The independent re-measurement (2026-08-23, same machine, UD-IQ4_XS) isolated the built-in head with an on/off pair and measured 1,008 MiB fixed, plus 5,120 B per token of window — 11.4% of whatever -c you set — plus a further 898 MiB for n-max 10 over n-max 4. At -c 163840 that is roughly 1.8 GiB, more than the vision projector costs. The head's own weights are only 334.7 MiB; the rest is draft-path allocation. --spec-type none frees every byte of it, and the server then logs model has unused tensor blk.64.* -- ignoring to prove the head was never loaded.
• --spec-draft-p-min is the hidden second knob. At n-max 10, lowering it from 0.75 to 0.5 was worth +8 t/s (73.5 → 81.7); at n-max 4 the difference was small. A high confidence floor keeps pausing exactly the long drafts that a big n-max exists for.
• DFlash2's recommended n-max 7 was not its own best setting here — 4 and 5 both beat it (77.6 and 77.5 against 74.0). Every recommendation in this space, including this page's, is one machine's measurement.

Conditions: UD-IQ4_XS, -c 32768, -ngl 99, q8_0 KV, --parallel 1, temperature 0 / top-k 1, an 83-token novel-JavaScript prompt, 700 predicted tokens, a fresh server per configuration, reasoning tokens. accepted per target pass = draft_n_accepted ÷ (predicted_n − draft_n_accepted), both fields from the server's own timings.

Rank configurations by accepted tokens per target pass, not by acceptance rate. n-max 4 / p-min 0.75 accepts a near-perfect 89.9% and still loses, because a shallow draft caps how much a single verification pass can deliver. But the metric is a ridge, not a straight line: n16 accepts the most per pass (4.88) and is 8% slower than n10, because drafting deeper costs real time. The same reversal shows up at the extreme: on the verbatim-copy row, n-max 4 accepted a flawless 100.0% (424 of 424) and ran 97.0 t/s while n-max 10 accepted 99.0% and ran 148.7 — 3.96 accepted per pass against 10.54.

Conditions: -c 32768 -ngl 99 --parallel 1 -ctk q8_0 -ctv q8_0 --jinja, enable_thinking:false, temperature 0 / top-k 1, 700 predicted tokens, a fresh server per configuration (the speculative flags are load-time) plus a discarded warm-up, two probes per server, agreeing to about 1%. Answer lengths varied by about 1% between configurations despite temperature 0 — the usual CUDA batch-shape reduction-order non-determinism, with every stream in a file sharing its opening 120 characters. "Identical output regardless of speculative flags" is true in principle and not quite literally true on this build.

Same winner, same loser, same shape on both files — n10/p0.5 is fastest on each, n2/p0.75 is the slowest speculating configuration on each, and the ranking between them matches. There is no per-file optimum. Acceptance is a property of the draft head, not of the quantization: at six of the seven configurations the two files land within 1.6 points of each other (96.5/96.7, 93.3/94.2, 89.7/88.6, 61.4/60.0, 76.8/75.4), and the seventh — n6/p0.5 at 75.8/72.1 — is 3.7 points apart. This page reports that exception rather than inheriting the tidier claim. The quantization does not change how well the head predicts — it changes how fast the target model verifies. UD-IQ4_XS is faster at every configuration, and gains more from a wider drafter (+7.5% with no drafter, +19.6% at n4, +14.9% at n10). And the then-shipped default (n4/p0.75) leaves 11% on the table on UD-IQ4_XS and 15% on Q4_K_M for this workload. The one genuine difference is of degree, not kind: UD-IQ4_XS climbs steadily to n6 while Q4_K_M flattens and dips there (69.8 → 66.4), because its verification step is expensive enough that a wide, low-acceptance tree stops paying sooner.

Swift files against the anchor in the same sweep: speed at depth, the drafter-off floor and the drafter on one prompt

In the answer regime with the n4/p0.75 drafter the cards then shipped, Swift IQ4_XS decodes faster than the anchor at 1.5k and 28k of depth and about level at 91k, and at the same speed with the drafter off; Swift IQ3_XXS matches the anchor at 1.5k and 28k and falls below it at 91k. On one prompt with thinking on, both Swift files decode slower than the anchor with the drafter (below). Measured 2026-10-04 in one alternating sweep, greedy (temperature 0), -c 131072, q8_0 KV, the first probe after each prefill discarded, two passes n=2: with the drafter the cards shipped until 2026-10-08 (n4/p0.75, thinking off), Swift IQ4_XS ran 87.2 t/s at 1.5k depth (×1.102 against the anchor’s 79.1), 81.4 t/s at 28k (×1.116 against 73.0) and 61.1 t/s at 91k (×1.018 against 60.1) meas, ratios der — in step with its higher drafter acceptance (0.928 against 0.892 at 1.5k, 0.943 against 0.927 at 28k, 0.935 against 0.924 at 91k). Swift IQ3_XXS ran 79.3 t/s at 1.5k (×1.002), 73.2 t/s at 28k (×1.003) and 56.3 t/s at 91k (×0.938 against 60.1). With the drafter off at 1.5k depth, Swift IQ4_XS gave 42.9 t/s (×1.005 against the anchor’s 42.7) and Swift IQ3_XXS gave 44.1 t/s (×1.034). The anchor’s own speed in this sweep sat at the lower of this machine’s two levels (§03): 0.917, 0.91 and 0.928 of its August speed at 1.5k, 28k and 91k der; the absolutes behind those ratios are in the provenance chain (§17). The reasoning regime at depth is void: its two passes disagreed by more than 3% in three of its six Swift cells, and its first pass ran inside a download overlap on the test host; a re-run on 2026-10-05 failed the same agreement check in five of its six Swift cells, so no reasoning-regime ratio is published (§15). The same overlap covered the drafter-off group’s first pass and the start of the answer regime’s second pass; both groups’ passes agree within 3%. The one-prompt drafter measurements below, thinking on included, are a separate step and are not void. All speed bands in this sub-section are greedy-only; speed at the temperature-1.0 sampler the cards ship is not measured as a band (§15).

One sweep, 2026-10-04; greedy (temperature 0); -c 131072; q8_0 KV; two passes, the first probe after each prefill discarded; ratios printed only where the two passes agree within 3% of their mean. Not comparable with the August tables above in this section.

The size-derived prediction missed for both Swift files: bytes read per token alone did not predict the drafter-off decode speed of these files.

Predicted ratios are decode-speed ratios against the anchor, derived before the probes ran from file size minus token_embd, header and MTP head (der; token_embd assumed host-resident). Measured ratios are the drafter-off probes at 1.5k depth, 2026-10-04, greedy, -c 131072 n=2. The data show no cause for the discrepancy.

On one prompt at -c 32768, greedy, all three files draft well with thinking off (acceptance 0.917–0.927 at n4/p0.75); with thinking on, the Swift files keep far fewer drafted tokens per verify pass than the anchor (1.479 and 0.974 against 2.684 at n4/p0.75), and for them n10/p0.5 is slower than n4/p0.75 (47.51 against 59.96 and 43.95 against 50.61 t/s). meas n=4 per cell. The board sat at its software power cap in at least 96.8% of the active samples in every window meas.

The full table for this prompt — decode, acceptance and accepted tokens per verify pass for every file, drafter setting and thinking mode — is in §08, with its conditions: one prompt (a JavaScript red-black tree) sent three times per load, the first discarded; two passes, pass 2 repeating pass 1’s text; -c 32768; greedy; 700 tokens per request; 2026-10-04; the power-cap share is over the 30 windows of that step.

With thinking off, greedy output on the one prompt was identical across drafter settings (off, n4/p0.75, n10/p0.5) for Swift IQ4_XS; it differed for the anchor (unsloth UD-IQ4_XS) and for Swift IQ3_XXS, whose drafter-on runs did not always repeat their own text (§08). Two checks on the speed probes hold: prompt_n was equal to the anchor’s at every depth (1,458 at 1.5k, 28,388 at 28k, 90,854 at 91k on the answer arms; 1,498, 28,428 and 90,894 on the reasoning arms; 1,458 on the drafter-off arms), and none of the thinking-off arms produced reasoning tokens.

Only one row of this section was measured: the RTX 3090. Every other card's number is the same arithmetic from §04 — bandwidth divided by file size, times the efficiency constant for that file's format — applied to the card's published bandwidth. That arithmetic predicted the measured 3090 row correctly, which is the only reason to trust it elsewhere, and it is still a prediction. The pattern that matters more than any individual number: how much memory a card has decides which strategy you can use; how fast that memory is decides how long you wait. A card with enormous bandwidth and too little memory loses to a slower card that fits the model, because the moment one layer lands on the processor the whole calculation changes to the processor's memory speed.

The figures below are for the reference run: about 2,000 tokens of prompt and about 100,000 tokens of thinking and output, with each card using the settings that keep quality highest. All of them exclude speculative decoding, which varies per system but helps a lot: on the reference 3090 it moved this run to roughly 34 minutes from 45. Treat the drafter as moving a card up this chart — furthest on predictable content — rather than as a new baseline.

† the 3-bit rows trade quality for that speed — a different model for benchmark purposes (§08). They also trade window: on 16 GiB the resident 3-bit file caps context near 49,152 tokens by §05's budget arithmetic, so a single run longer than 100k tokens must either checkpoint across windows or take the offload path.

System-memory assumption for every offload and shared-memory row: those t/s figures are derived at dual-channel bandwidth — two sticks: desktop DDR5-5600 ≈ 90 GB/s, the integrated-GPU row's LPDDR5X-9600 ≈ 153.6 GB/s. A single stick runs single-channel at half that bandwidth — halve the t/s and double the hours, and slower DDR5 grades scale down proportionally. If your machine reads slower than this table, check that assumption before blaming the table: Task Manager → Performance → Memory shows your speed and slots used, and §04's formula predicts your number — your GB/s ÷ your file's GB × 0.70 for a K-quant or 0.65 for an IQ-quant. Read the family off the filename: a name containing IQ (UD-IQ4_XS, UD-IQ3_XXS, UD-IQ2_S) is an IQ-quant and takes 0.65; a name containing K without IQ (Q4_K_M, UD-Q3_K_XL, UD-Q2_K_XL, Q6_K) is a K-quant and takes 0.70. Rows where the model is resident on a discrete card are unaffected; their bandwidth is the card's own.

On a 24 GB card, download UD-IQ4_XS from unsloth for everyday use, or Q4_K_M from lmstudio-community if you want the highest measured quality and can accept that it leaves only about 0.3 GiB of the card for your desktop. Two files of the same size from different quantizers really can differ in quality, so the choice is not arbitrary — but between these two the measured difference is 0.9% of perplexity, which is smaller than the uncertainty of the measurement itself, while the size difference is 2.1 GiB and buys you either the vision projector or about 50,000 more tokens of window. If you have a 16 GB card, take UD-Q2_K_XL — the memory each file and window needs is measured, and it is the 2.9-bit file, not the 3-bit one, that fits a 65,536-token window with its drafter still switched on (the requirement table below); if you have 32 GB, Q6_K; and if you have a Blackwell card and can run vLLM, the NVFP4 build is the one that runs natively there. If you have 12 GB, the answer is no — not a smaller quantization of this model, but a smaller model. The 24 GB answer depends on how big a window you want, and there are three files rather than one. Up to about 98,000 tokens, stay on UD-IQ4_XS. At about 131,000, UD-Q3_K_XL decodes 20% faster on this page's fill and ties the reference file on accuracy. At the full native 262,144, UD-Q2_K_XL is the faster one and the only one that can still speculate. All three are measured; the ladder that settles the quality half starts at the eight-rung table below, the memory and speed each configuration needs is in the requirement table, and the card-by-card verdicts are in the last part of this section. On the 2-bit question there is a coding-benchmark answer: on 225 programming exercises the 2-bit file finishes with exactly the same score as the 4-bit one and pays 20 to 45% more tokens, 20 to 36% more time and about 32% more energy for every exercise it solves (the coding-benchmark part below). Take the 2-bit file when you need the 4.4 GB of VRAM it saves; otherwise take the 4-bit one.

Units, because they bite: download pages list decimal GB while VRAM budgets are binary GiB, and the gap is about 7%. The 17.6 GB UD-Q4_K_XL is 16.39 GiB resident, which is why "17.6 GB of weights + 4.25 GiB of cache" still fits inside 24 GiB (§05). Every GiB figure in the table above is the GB figure divided by 1.073741824.

Does FP4 beat INT4? A scored smoke test

A common belief says floating-point number formats always beat integer formats at the same bit width. It did not survive the test here: all three 4-bit files scored the same on one benchmark, and on the other the integer-grid file answered two more questions than either floating-point file. Twenty questions cannot rank healthy files — that is what the rest of this section and §10 are about — but it is enough to say the belief is not a rule you can apply blind.

First, what the two formats actually are. A 4-bit quantization stores each weight as one of only 16 allowed values; the formats differ in where those 16 values sit:

The three quantizations above were graded for correctness on GSM8K and MATH-500, 20 problems each, greedy decoding — always take the most likely next token, so the same prompt gives the same answer every time (§02) — identical prompts, a 4,096-token budget, with answers that ran out of budget mid-thinking counted wrong and reported as truncations. One framing rule before the numbers: at 20 problems this is a smoke test, not a ranking — each question is worth 5 points (§02). Its job is to catch a broken file: a bad conversion or a mangled chat template collapses a score by 20–40 points or more, which even 20 questions detects reliably. It cannot rank healthy files, whose true gaps are a point or three (the arithmetic is in §10). Measured on the reference 3090:

What the numbers say:

• "Floating point always wins" did not survive the test. All three tied on GSM8K; on MATH-500 the integer-grid K-quant answered two more problems than either FP4 file. At 20 samples that edge is weak evidence — but it is the opposite direction from the belief. What actually decides 4-bit quality is how cleverly a format spends its bits — block sizes, scale precision, which layers stay at higher precision — and mature K-quants are very good at that. Both families use fine-grained scaling; the float-versus-integer grid underneath matters far less than people assume.
• The two NVFP4 tiers were indistinguishable at this sample size — which is what noise looks like, not evidence of equivalence. The file structure does predict a tie: the family shares one byte-identical NVFP4 backbone, with tiers differing only in the surrounding layers. The surprise is where VERY-LOW's savings actually landed: on these maths prompts at n-max 10 its draft head drafted noticeably worse (accepted length 3.33 against about 4.8) — a speed cost, not an accuracy one. The penalty is configuration-dependent: re-measured 2026-08-22 at n-max 4 / p-min 0.75 on code generation, both tiers accepted identically (0.83) and VERY-LOW was the faster file (54.6 against 48.2 t/s) — smaller weights, less memory traffic per token.
• Sample-size honesty: each problem is worth 5 points. Every gap in this table is one or two questions wide. Treat it as "no meaningful quality difference, mild edge to the K-quant" — not a ranking. The same prompts were also run through a different serving stack and grader version, and individual cells moved by up to 10 points, which means measurement variance is the same size as the differences being measured. A one-run quantization ranking published without error bars is not evidence, whoever published it.

How the files rank on perplexity

Perplexity is the dotted line on the ladder chart.

Accuracy at 20 questions cannot separate healthy files, but a per-token measure can, because it feels quantization damage at every scored token instead of collecting one right-or-wrong bit per question. Perplexity is defined in §02; the run below scores 294,912 token positions — 36 chunks × 8,192, forced by the corpus, the tokenizer and -c 8192. Measured 2026-08-22 on the reference 3090 with llama-perplexity over the full wikitext-2-raw test set:

Read it with three limits attached. One corpus. Wikitext rewards plain-text prediction while unsloth's Dynamic calibration targets chat-style data, so the ordering among the top files may not carry over perfectly to instruction-following work. The scored-position count is tokenizer-bound. A later run on the same corpus files recorded 297,193 tokens under Qwen's tokenizer and 295,216 under Gemma's, both at 36 chunks — so 294,912 is a count of scored windows (36 × 8,192), not of corpus tokens, and it moves with the tokenizer. The practical consequence is a hard rule: perplexity is never comparable across model families. And the reference file reproduces. A separate 2026-08-23 run re-measured the UD-IQ4_XS figure as a gate before further work and returned 6.5956 against an expected 6.5956 — bit-identical, which is what pins the error bars and the chunk count.

Two robust findings survive all three limits: both NVFP4 tiers trail the K-quants — the same direction the scored table's MATH-500 column hinted, reached by an independent measurement — and none of the unsloth Dynamic variants beat the plain lmstudio Q4_K_M on this corpus, including at identical size. Two cheap metrics agreeing beats one expensive one.

The same corpus, the same 36 chunks and the same error bars also price the cache instead of the weights. Measured 2026-08-23 on UD-IQ4_XS, every other flag identical to the runs above (-c 8192 -fa on -ngl 99), on a corpus file byte-verified identical to the one the fp16 and q8_0 baselines used:

Three readings, and the third is the one that keeps this honest. Degradation is super-linear in bits — the second halving costs slightly more than the first (0.383 against 0.309), so the cut stops paying rather than scaling. Both are small next to the model quantization: the Q4_K_M-to-UD-IQ4_XS gap on this same corpus is 0.93%, larger than q4_0's entire cache penalty, so anyone who chose the smaller weights file on quality grounds has already accepted a bigger hit than moving to a 4-bit cache would add. And the error bars overlap: ±0.045 on each of two independent estimates gives ±0.063 on their difference, so the fp16-to-q4_0 gap of 0.0457 is about 0.7 of one standard error and a single 36-chunk run does not resolve it on its own. (That combined figure, not the ±0.045 on a single estimate, is the right yardstick for every file-against-file comparison on this page, including the 0.061 gap between the two 4-bit files above.) the direction is consistent with the q8_0 point and with theory, and that is all it is. Two limits on how far to carry this. It was measured at -c 8192, so it says nothing about the long-context retrieval failure that is the actual argument against a 4-bit cache at 200k-plus (§05's mechanism footnote — still unmeasured). And q4_0 was not faster in any way that matters: 266 s against 298 s of perplexity wall-clock is a bandwidth effect on a prefill-shaped workload, not a decode result. Run -ctk q8_0 -ctv q8_0. It is what every recipe on this page ships and what every context ceiling here is built on.

The quantization ladder: nine files of one model, four readings

One model, squeezed by one vendor to eight sizes from 13.3 GiB down to 5.8 GiB, every one of those eight measured the same way on the same day. Three readings sit beside each file below — perplexity, task accuracy and the empty-answer count — and a fourth runs the code each file wrote (measured 2026-08-25) (further down). It answers the question people actually ask — how small can I go before it stops being the same model? — and it answers it in more than one voice, because the readings disagree about where the damage starts and each is right about something the others cannot see.

Two of those readings are not independent of each other, and this table would mislead if it did not say so. The empty-answer column is counted from the same 75 greedy generations the accuracy column is scored on, and every empty answer, on every file, is also scored wrong — checked across all nine arms, with no exceptions. The empty column is therefore a breakdown of the accuracy column’s failures rather than a second opinion about them. It earns its place for a different reason: counting one named failure is more sensitive than testing a 75-item score for significance, so it moves a full rung earlier, and it costs no GPU time at all (below).

The ninth row is not a rung of this ladder, and it is printed anyway. QAT-Q2_0 is a different vendor’s build of the same model — sdkyuan’s, made by quantisation-aware training, which trains the model with the rounding already in place instead of rounding a finished model afterwards. It is here because it is the one file that does not follow the pattern the other eight make, and a table ordered by bits per weight that quietly dropped its own counterexample would be worth less than one that prints it.

Read the columns like this. Perplexity is a per-token measure of how surprised the model is by ordinary text: it feels a little damage at every rung and is the earliest warning, but a file can lose perplexity and still do your work. Accuracy Mean is the composite of three graded benchmarks — GSM8K, HumanEval and MBPP, 25 questions each, 75 items in total — and it is the only column in the reader's own units. The empty-answer column counts the times the model returned nothing at all, and it is kept separate from truncations because the two are different failures: a truncation hit the token cap, an empty often terminated normally and simply emitted zero characters, which no truncation counter can see. Those last two columns are the functional detectors that §01 counts as the third of this campaign's instruments — the automated checks for what a score cannot see (§02). The two vocabularies name one thing.

Conditions, and they differ between the two scored columns. Perplexity meas: frozen wikitext-2-raw test set, 36 × 8,192 = 294,912 scored positions per file — the same count and corpus as the table above it — -ngl 99 -c 8192 -fa on --load-mode mmap, fp16 KV, llama.cpp build 10502. Accuracy and the two failure columns meas: a frozen suite — fixed once and never edited again, so every file is asked the identical questions — 1cdf54f8eb9d3f8f, GSM8K + HumanEval + MBPP at n=25 each, greedy, seed 42, --max-tokens 16384, -c 32768, q8_0 KV, reasoning_effort=low, and no drafter — which turns out to matter, and is dealt with in the next part. "Silent" empties are the ones that did not hit the cap: they terminated normally and returned zero characters, so no truncation counter ever saw them. Means are printed at the harness's own precision; what they can be read to is the subject of the next two paragraphs. QAT-Q2_0 was measured later and separately, and its row says so: perplexity on 2026-08-25 over the identical corpus, chunk count and flags (7.4996 ± 0.04784), and the accuracy suite and both failure columns on 2026-08-26 over the identical frozen suite. Every condition matches; the day does not.

So what can be said, arm against arm, is whatever a paired McNemar test says. It compares the two files question by question on the same 75 items and counts only the ones they disagree about — b is the number the left file got right and the right file got wrong, c the reverse, and p is how often a gap this large would turn up by chance if the two files were really equal (§02). The exact test, counting a difference in either direction:

The one sentence that reconciles a falling perplexity curve with a flat accuracy curve. Look at the c column: it is zero in seven of the eleven rows, and in every row that compares a file with the 4-bit reference. The smaller file almost never wins a question the larger one lost, and against the reference file the losses run in one direction only. So quality is falling at every step down the ladder; it is simply below what 25 questions per set can resolve until the cliff arrives. The flat accuracy column is not evidence that the files are equal. It is evidence that this instrument cannot see a gap this small, which is what §10 said before the ladder ran.

Where the boundary is, stated as an interval because that is all the data supports. UD-IQ2_S at 2.481 bits per weight ties the reference file. UD-IQ2_XXS at 2.153 does not. The boundary therefore lies between 2.48 and 2.15 bits per weight, and 25 questions per set cannot place it more finely than that. Anyone quoting a single number for where this model breaks — including a future edition of this page — is quoting something the measurement does not contain.

A second opinion on the same ladder: how often it picks the same next word

Agreement with the full-precision model is what separates files the question set cannot; both readings sit together on the ladder chart.

Agreement — how often a shrunken file picks the same next word as the reference — separates every rung of this ladder cleanly, which neither perplexity nor the 75-item accuracy suite manages. Perplexity is one average over a whole corpus, so two files can share a perplexity while behaving differently — this page already carries the proof, in that perplexity cannot separate UD-IQ2_XXS from UD-IQ1_M (they sit 1.67 % apart at 1.7 sigma).

First, a name collision worth clearing up, because the two things are unrelated. --top-p 0.95 in the recipes on this page is nucleus sampling — a knob that controls randomness while the model writes. “Same top p” here is a measurement: an output line of llama-perplexity --kl-divergence. It reports how often a shrunken file picks the same next word as a reference file, over the same text. Word by word, not averaged.