Every public model, with the result and the failure mode.

OffHours Context-Interference Benchmark

A paired office-work benchmark that separates unresolved personal context from interruption and raw context volume. Devin showed no mental-toll penalty, but reached a reproducible context-volume boundary.

Clean accuracy

99.0%

Mental-toll effect

not detected

Volume boundary

8,000 words/day

Competition: Clean workday vs Resolved family context

Next: Freeze this Devin result and run the same benchmark against local Qwen with exact prompt-token counts and model-file provenance; do not revise the treatments after seeing this null.

Qwen3-0.6B Routed SQL Specialist

A two-adapter routed SQL artifact: public schema-only SQL routes to a b-mc2 adapter; local SQLite execution routes to a synthetic execution adapter.

Public exact

0.531

T5-small baseline

0.484

Synthetic execution

0.860

Competition: posttrainllm routed SQL v1 vs T5-small local baseline

Next: Publish as a report artifact. Two hygiene DPO retries proved reference anchoring cures the SimPO collapse and even lifts execution to 0.920, but output hygiene needs a generation-strength fix (stronger SFT or constrained SELECT-prefix decoding), not more preference tuning. Do not present as a shipped SQL model until a public execution gate and clean-output gate pass.

Report only — no model to use not fully verified Report card →

Pace Intent Router v8

A 49.5M intent classifier that was extremely fast and strong on its source-matched synthetic holdout, then failed the first leakage-checked sealed distribution gate.

Hugging Face model ↗

Sealed accuracy

57.1%

Frontier retained

57.1%

Warm latency

3.8ms

Competition: Pace Intent Router v8 vs Qwen3-4B-Instruct-2507 4-bit

Next: Do not tune on sealed V1. Build a broader public-development corpus from the failure themes, train a successor, and judge it once on a newly generated sealed V2.

Qwen3-4B File-Ops Distilled

A routed 4B file-operation specialist distilled from frontier/gold trajectories, with the breadth regression disclosed in the package.

Hugging Face model ↗

File-ops hard gate

100%

Heldout file-ops

95%

Breadth after tuning

42.3%

Competition: posttrainllm Qwen3-4B file-ops specialist vs Stock Qwen3-4B

Next: Keep routed-only warnings attached and add a consumer pull/load smoke before wiring this into any app.

Shipped — routed only not fully verified Report card →

Qwen3-4B ReST Fused

A teacher-free ReST candidate that preserves the 100% file-ops gate while recovering out-of-domain breadth above the stock 4B baseline.

Hugging Face model ↗

File-ops hard gate

100%

Breadth after ReST

65%

Breadth delta

+5.4pp

Competition: posttrainllm Qwen3-4B ReST vs Stock Qwen3-4B

Next: Keep this package research-only. Freeze a product-specific target before spending compute on another eval or training run.

Shipped — routed only not fully verified Report card →

Qwen3-4B Multibackend Distilled

A public failed attempt that retained 100% file-ops depth but drove the recorded breadth score down to 31%.

Hugging Face model ↗

File-ops depth

100%

Breadth

31%

Breadth delta

-28.6pp

Competition: Multibackend-distilled 4B vs Stock Qwen3-4B

Next: Keep the weights public as a failed comparison artifact. Do not spend compute revalidating them unless a new breadth-preserving recipe explicitly needs this checkpoint as its baseline.

VibeThinker-3B MLX Conversion

A public Apple-Silicon-friendly conversion of WeiboAI/VibeThinker-3B, preserved as a runtime artifact rather than claimed as a PostTrainLLM-trained model.

Hugging Face model ↗

Model class

3B

Local GSM8K screen

40/40

Training delta

None

Competition: PostTrainLLM VibeThinker-3B MLX vs WeiboAI/VibeThinker-3B

Next: Keep this as a conversion and runtime artifact. Add a pinned loader plus parity receipt only when a real Mac-local consumer justifies maintaining it.

VibeThinker-3B Agentic Distilled

A preserved agentic distillation checkpoint whose public weights outlived its evaluation evidence; no current win can be claimed.

Hugging Face model ↗

Public weights

Yes

Current agentic eval

Missing

Reasoning retention

Unknown

Competition: VibeThinker-3B agentic distilled vs VibeThinker-3B MLX base

Next: Leave the weights public but unpromoted. If this lineage becomes active again, start with an evidence-only baseline/candidate evaluation—not more training.

Hugging Face Specialist Model Archive v1

The local specialist model cache was promoted to Hugging Face or deleted when safely re-downloadable from upstream repos.

posttrainllm HF repos

6

Local model cache

cleared

Storage policy

HF first

Competition: Hugging Face artifact storage vs Local Mac cache

Next: Use this as the storage index; use the six dedicated model case studies for quality and decision evidence.

Factory Run Schema v1

The canonical target -> data -> post-training -> eval -> package -> report shape for posttrainllm runs.

Required files

8

Decisions

6

First-class outputs

5

Competition: posttrainllm factory schema vs Ad hoc model card only

Next: Turn the SQL routed result into the first website-native factory report that follows this schema.

Browser WebGPU Training Speedup

The original browser posttrainllm track: hand-written WebGPU kernels beat WASM SIMD more as model width grows.

WebGPU speedup

12.1x

Small-width speedup

2.6x

Browser track

shipped

Competition: posttrainllm WebGPU vs posttrainllm WASM SIMD

Next: Keep as a public performance artifact and cross-link it from factory reports when browser-local training matters.

Memory64 Browser Behemoth Allocation

A WebAssembly Memory64 build lifted the browser model allocation ceiling past the old 4GB tab limit.

Allocated params

473M

Allocation time

3.7s

Train step

82.2s

Competition: posttrainllm Memory64 build vs posttrainllm wasm32 build

Next: Keep this as a public technical artifact; do not make it active factory work unless a browser-run specialist needs it.

Huge Preset Decode Throughput

The native Mac runtime reached high local decode throughput on the Huge preset, showing the serving path is viable for local eval loops.

Huge decode

696 tok/s

Mega pilot

293 tok/s

Warm TTFT p99

5.8ms

Competition: posttrainllm Huge preset vs posttrainllm Mega pilot

Next: Use this as the baseline expectation for future artifact performance tables.