Retiring a model the right way: the Qwen decommission checklist

Published Oct 13, 2026 · Updated Oct 15, 2026

1. PROVE Test the replacement Serving performance and task quality need separate checks.
2. INVENTORY List every dependency Providers, jobs, presets, routes, services and disks.
3. PAUSE Get authority + preserve rollback Back up configuration; pause before irreversible removal.
4. REMOVE Follow the keep-list Remove the retired lane, not a different product with a similar name.
5. READ BACK Re-list + exercise the survivor Check the actual client path; record unverified hosts explicitly.
6. ARCHIVE Keep the why Preserve verdicts, receipts and deliberate exceptions.

This is a dated build log, corrected October 15, 2026. Operational measurements are scoped to their cited audit dates; see the evidence note below for the private-source limit.

Evidence note — October 15, 2026: the operational counts and incident history below come from retained private receipts, not public downloads or a new live audit. Private receipt paths have been removed from the reader copy; the editorial record retains their mapping. Diagrams explain the design and do not prove runtime state.

Why retire at all

Qwen3.8 Flash-Next was our local workhorse for weeks. The replacement had favorable serving measurements, while the adapter program had reached a separate quality dead end. Those are different kinds of evidence.

The head-to-head, same probe, same day (2026-10-01): GLM-5.3-Flash solo 45.44 tok/s vs Qwen single-Spark NVFP4 38.7 tok/s; 3 concurrent long sessions 116.9 tok/s aggregate for GLM. Correction — October 15, 2026: the receipt used a broad “stronger model” verdict. The throughput figures above support only those serving tests, not universal task quality or every serving metric.

The model-quality side had already failed on its own merits. The fine-tune program ran windows with adapter verdicts — candidate 9/12×3 vs base 10/12×2 (“NO deployment case”), then a sealed holdout 10/12×3 at parity, then 9/12×3 = REGRESSION for the last variant. The program stopped at the first honest dead end.

That’s the “why” in one line: the dead config tax. Every day a superseded model stays wired in, you pay in cron cycles, stale policy rules, dead presets, and — the dangerous part — config that looks load-bearing but points at nothing.

The inventory pass

Before touching anything, list every reference. For our fleet that meant: Hermes providers on every profile, cron jobs, MoA presets, the dispatcher’s route table, policy rules, systemd units, and plain disk. The rule: you can’t remove what you haven’t enumerated, and you can’t verify removal without the re-list.

Remove in dependency order

The actual sequence, run over two days and one owner approval:

Day one — the lane itself (2026-10-02). Providers dgx-flash-next and spark2-flash-next removed; the replacement dgx-glm53 became the only local entry in the model picker. Six Qwen/SFT cron jobs were paused first (resumable, in case anything caught fire), the SFT program stopped at the regression verdict, and then the disks: Spark1 lost the qwen systemd units, 341 GB of base weights, 65 GB of expert packs, HF and container caches, adapters — going from 658 GB free to 2.9 TB free. Spark2 lost its twin container, the NVFP4 cache (99 GB), the MLX cache (106 GB), and ~115 GB of qwen docker images. One thing was deliberately kept: ComfyUI’s qwen-image lane, which is a product, not a model server. The day ended with an end-to-end check: GLM answering READY from the Mac after all cleanup.

Day two — the long tail (2026-10-03). The six paused crons were removed outright, with all six IDs recorded and a re-list showing zero remaining. On the mini-host: four dead providers unset via the config CLI (with a timestamped backup taken first), dangling MoA presets re-pointed at the new lane, two dead presets deleted, the dispatcher’s ollama path retired with a live RETIRE-OK through the patched path in 2.7 s, and policy rules rewritten (five dead node overrides gone, fallback re-pointed, hard rules 7→6).

Verify like you mean it

The receipt for “removed” is the re-list, not the command’s exit code. Exit code zero means the command ran. The re-list means the thing is gone. The retained October 13 audit reports the following checks; these are dated readbacks, not a current all-host inventory: systemd units disabled, zero qwen lines in the crontab, zero qwen references in six Mac profile configs, zero dgx-flash-next in the mini-host config.

Two references survived on purpose and are worth naming, because a retirement audit that finds nothing is suspicious: an effort-tuning key for a model id that is no longer plugged in (harmless, noted), and the cloud nous picker list, which was never part of the lane.

One honest gap: the day-one cleanup on the worker Spark was recorded as narrative in the daystate receipt, not as a separate per-host JSON — and it hasn’t been independently re-verified since. The checklist below now makes that a required step.

What stayed and why

Retirement is not a memory-wipe. The skill archive kept the adapter verdicts and four lessons learned; the DAYSTATE receipts kept the exact cron IDs and numbers; and the rollback-tag hygiene pass later deleted old image tags only after verifying zero references. If you torch the history, you can’t answer “why is this config here?” six months later — which is exactly the question retirement is supposed to make easy to answer.

The reusable checklist

1. Prove the replacement first — compare serving workloads under matched conditions and test task quality separately. Retirement is a verdict, not a mood.
2. Inventory everything — providers, crons, presets, dispatcher routes, policy rules, systemd units, disk. Enumerate before you delete.
3. Pause before delete — resumable pauses catch fire early; deletion is for after the pause period.
4. Owner sign-off — a retirement touches live capability. Get it, record it.
5. Backup configs, then unset via CLI — timestamped .bak files, CLI verbs, no hand-editing state.
6. Re-list after every removal — with IDs. “Zero remain” is a measurement, not a feeling.
7. Clean disks with a keep-list — products (like ComfyUI’s image lane) are not the model lane. Name what stays.
8. End-to-end verify the survivor — the replacement must answer from the client after all cleanup.
9. Schedule the re-audit — days later, re-grep everything and write down the two or three references that remain on purpose.
10. Archive the why — verdicts, lessons, and receipts stay. Future-you is the customer.

Source receipts: DAYSTATE-20261001.md and DAYSTATE-20261002.md (flash-next-phases-20260923), repo commit 96610aa, the archived qwen38-flash-next-training skill, and the 2026-10-13 live re-verification.