# Mac-local LLM specialist factory | PostTrainLLM > Follow the source-linked loop from target and data through post-training, evaluation, packaging, and an honest ship, retry, or reject report. Canonical: https://posttrainllm.com/learn The factory, end to end # Build the ruler. Then build the model. A small specialist earns its place one artifact at a time. This is the shortest path through the system—from choosing a capability to deciding whether the result should ship, redirect, retry, or stop. [Continue the inference sprint](https://posttrainllm.com/learn/session?route=inference-systems-13w) [13-week overview](https://posttrainllm.com/docs/learn/inference-systems-13w/) [Ground-up foundations](https://posttrainllm.com/learn/session?route=ground-up) [Browse artifacts](https://posttrainllm.com/learn#artifact-journey) [Open the complete docs](https://posttrainllm.com/docs) ## Field guides - [How to cook an LLM — interactive field guide](https://posttrainllm.com/articles/how-to-cook-an-llm) - [CLI lab reference](https://posttrainllm.com/docs/cli-reference) - [How to train your own LLM](https://posttrainllm.com/how-to-train-your-own-llm) - [Fine-tune an LLM on a Mac](https://posttrainllm.com/fine-tune-llm-on-mac) - [MLX LoRA fine-tuning](https://posttrainllm.com/mlx-lora-fine-tuning) - [Competitive landscape](https://posttrainllm.com/docs/learn/competitive-landscape) nine stages / thirteen things you can make ## Learn by leaving artifacts behind. Theory becomes useful when you can change a real system and prove what the change did. Each stage ends in something inspectable: a model, adapter, benchmark, agent, package, runtime decision, or causal probe. **runnable lab:** A bounded local or browser exercise with an existing implementation. **guided replay:** A source-linked reconstruction of retained evidence; it does not claim a fresh run. **recipe contract:** A complete procedure whose expensive execution begins only as a fresh, budgeted experiment. ### Make a language model How does text become a prediction and a gradient? Trace the complete transformer learning loop, then change it without losing track of tensor shapes. [Open the learning path](https://posttrainllm.com/learn#path-foundations) #### [Byte-level TinyGPT](https://posttrainllm.com/learn/artifacts/byte-tinygpt) model checkpoint runnable lab A readable ~0.8M-parameter transformer whose byte tokenizer, architecture, loss, and checkpoint format are all inspectable. **Workload:** light first; training duration is user-controlled - Build: [Train the tiny preset on the bundled corpus.](https://posttrainllm.com/docs/guides/training_guide) - Modify: [Change layers, heads, width, or context in the source-of-truth model config.](https://github.com/PostTrainLLM/posttrainllm/blob/main/configs/model.byte-tinygpt-v0.json) - Tune: [Change learning rate, batch size, seed, or step budget one variable at a time.](https://github.com/PostTrainLLM/posttrainllm/blob/main/configs/training.json) - Prove: [Inspect initial loss, tiny-overfit behavior, samples, and tensor inventory.](https://github.com/PostTrainLLM/posttrainllm/blob/main/tests/README.md) - Package: [Keep a validated .tinygpt checkpoint with its exact config and corpus.](https://posttrainllm.com/docs/guides/model_guide) - Inspect: [model.py](https://github.com/PostTrainLLM/posttrainllm/blob/main/python_ref/model.py) - Inspect: [train.py](https://github.com/PostTrainLLM/posttrainllm/blob/main/python_ref/train.py) - Inspect: [tiny-corpus.txt](https://github.com/PostTrainLLM/posttrainllm/blob/main/data/examples/tiny-corpus.txt) - Run: [`posttrainllm train`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm inspect`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm validate`](https://posttrainllm.com/docs/cli-reference) #### [Browser-trained TinyGPT](https://posttrainllm.com/learn/artifacts/browser-tinygpt) interactive model lab runnable lab The same learning loop made visible through loss, sampling, presets, workers, WASM, and WebGPU-capable browser surfaces. **Workload:** runs in the browser; preset size controls cost - Build: [Open the playground and train a tiny model without a server.](https://posttrainllm.com/playground) - Modify: [Replace the corpus or choose a different model preset.](https://posttrainllm.com/docs/browser_notes) - Tune: [Adjust context, batch, learning rate, and generation controls in the lab.](https://posttrainllm.com/playground) - Prove: [Watch loss and samples together; do not mistake lower loss for useful behavior.](https://posttrainllm.com/docs/learn/session-08-training-mechanics) - Package: [Export the browser artifact and retain the settings that produced it.](https://posttrainllm.com/download) - Inspect: [playground.astro](https://github.com/PostTrainLLM/posttrainllm/blob/main/browser/src/pages/playground.astro) - Inspect: [gpu_model.ts](https://github.com/PostTrainLLM/posttrainllm/blob/main/webgpu/gpu_model.ts) - Inspect: [model.cpp](https://github.com/PostTrainLLM/posttrainllm/blob/main/wasm/src/model.cpp) ### Make training trustworthy Is the learner correct before it is large? Use tiny-overfit, held-out loss, numerical checks, and bounded comparisons to separate implementation bugs from optimization problems. [Open the learning path](https://posttrainllm.com/learn#path-training-mechanics) #### [Tiny-overfit correctness receipt](https://posttrainllm.com/learn/artifacts/tiny-overfit-receipt) training evidence runnable lab A minimal proof that data loading, forward pass, backpropagation, optimizer state, sampling, and checkpoint reload agree. **Workload:** bounded correctness gate - Build: [Run the smallest overfit gate before any ordinary training run.](https://github.com/PostTrainLLM/posttrainllm/blob/main/tests/README.md) - Modify: [Change only one of corpus, seed, precision, or optimizer settings.](https://github.com/PostTrainLLM/posttrainllm/blob/main/configs/training.json) - Tune: [Use learning-rate and clipping changes only after gradients and expected initial loss are verified.](https://posttrainllm.com/docs/techniques/galore_and_stability) - Prove: [Require memorization, checkpoint round-trip, and a sensible held-out comparison.](https://posttrainllm.com/docs/guides/training_guide) - Package: [Retain the config, seed, loss trace, sample, and validated checkpoint as one receipt.](https://posttrainllm.com/docs/factory/run-schema) - Inspect: [evaluate.py](https://github.com/PostTrainLLM/posttrainllm/blob/main/python_ref/evaluate.py) - Inspect: [checkpoint.py](https://github.com/PostTrainLLM/posttrainllm/blob/main/python_ref/checkpoint.py) - Run: [`posttrainllm train`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm eval`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm debug-loss`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm validate`](https://posttrainllm.com/docs/cli-reference) ### Change model behavior Which intervention teaches the missing behavior with the least collateral damage? Turn data and an adaptation method into an adapter whose target gain and regressions are explicit. [Open the learning path](https://posttrainllm.com/learn#path-post-training) #### [LoRA specialist adapter](https://posttrainllm.com/learn/artifacts/lora-specialist) adapter recipe contract A narrow adapter built from a frozen base, traceable data, explicit LoRA geometry, and a held-out task gate. **Workload:** fresh training requires an explicit budget - Build: [Prepare supervised rows and run the retained SFT path against a frozen base.](https://posttrainllm.com/docs/training/sft) - Modify: [Change the target data or modules while keeping the baseline and holdout frozen.](https://posttrainllm.com/docs/recipes/from-traces) - Tune: [Vary rank, alpha, target modules, learning rate, or steps from the checked starting profiles.](https://github.com/PostTrainLLM/posttrainllm/blob/main/configs/lora.json) - Prove: [Compare target gain, breadth, safety, latency, RAM, and throughput to the unchanged base.](https://posttrainllm.com/docs/factory/eval-protocol) - Package: [Bind adapter, base identity, prompt contract, recipe, and evidence into a specialist package.](https://posttrainllm.com/docs/factory/packaging) - Inspect: [SFT.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/SFT.swift) - Inspect: [registry.json](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/recipes/registry.json) - Run: [`posttrainllm prep-data`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm sft`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm eval-gate`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm export-mlx`](https://posttrainllm.com/docs/cli-reference) #### [Distilled tool-calling specialist](https://posttrainllm.com/learn/artifacts/distilled-tool-caller) routed specialist guided replay A real lineage where narrow file-operation depth reached 100% while breadth regressed, proving why routing is part of the artifact. **Workload:** replay is read-only; reproduction is a budgeted experiment - Build: [Reconstruct the student, teacher trajectories, filtering, and training lineage from retained evidence.](https://posttrainllm.com/docs/recipes/distill-specialist) - Modify: [Change teacher mix or selection slices as a new recipe version, never silently.](https://posttrainllm.com/docs/recipes/distillation-fc) - Tune: [Tune accepted rows, sampling mix, and steps against the same narrow and breadth gates.](https://posttrainllm.com/docs/techniques/distillation) - Prove: [Read target and breadth scores together: 100% narrow depth did not make a general upgrade.](https://posttrainllm.com/artifacts/qwen3-4b-file-ops-distilled) - Package: [Package as routed-only with an explicit redirect boundary.](https://posttrainllm.com/docs/factory/packaging) - Inspect: [attempt-ledger.md](https://posttrainllm.com/docs/attempt-ledger) - Inspect: [artifacts.ts](https://github.com/PostTrainLLM/posttrainllm/blob/main/browser/src/artifacts.ts) - Run: [`posttrainllm distill`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm eval-bfcl`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm export-mlx`](https://posttrainllm.com/docs/cli-reference) ### Build the ruler Does the benchmark measure the capability it claims to measure? Freeze a baseline, validate the frontier ceiling, inspect slices, and end with a reproducible decision rather than a score screenshot. [Open the learning path](https://posttrainllm.com/learn#path-evaluation-and-factory) #### [Needle 2 selection ruler](https://posttrainllm.com/learn/artifacts/needle-selection-ruler) benchmark case file guided replay A complete tiny-model journey from catalog ablation through a 2x2 distractor-and-safety training factorial, ending at a measured 45M capacity boundary. **Workload:** public evidence replay; rerunning model evaluation requires approval - Build: [Build the four training arms and verify the tiny-overfit correctness gate.](https://posttrainllm.com/docs/techniques/needle2-successor-factorial) - Modify: [Inspect how distractor-rich and safety examples differ while every other training variable stays fixed.](https://posttrainllm.com/artifacts/needle2-tool-selection) - Tune: [Compare the four LoRA arms, three independent training seeds, and the predeclared unsafe-arm stop rule.](https://posttrainllm.com/experiments?q=needle) - Prove: [Reproduce the public-dev decision and explain why sealed V2 and quantized parity stayed locked.](https://posttrainllm.com/artifacts/needle2-tool-selection) - Package: [Retain fixtures, raw outputs, scorer, catalog policy, and conclusion as a benchmark artifact.](https://posttrainllm.com/docs/factory/reports) - Inspect: [attempts.json](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/attempts.json) - Inspect: [needle2-baseline-review.md](https://posttrainllm.com/docs/techniques/needle2-baseline-review) - Inspect: [needle2-successor-factorial.md](https://posttrainllm.com/docs/techniques/needle2-successor-factorial) - Inspect: [needle-successor-result-v1.json](https://github.com/PostTrainLLM/posttrainllm/blob/main/evals/verified-wins/needle-successor-result-v1.json) - Run: [`posttrainllm eval-bfcl`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm eval-compare`](https://posttrainllm.com/docs/cli-reference) #### [Specialist factory report card](https://posttrainllm.com/learn/artifacts/factory-report-card) decision artifact runnable lab A run directory that binds target, data, candidate, frozen evaluation, packaging evidence, resource measurements, and a terminal ship/retry/reject decision. **Workload:** metadata lifecycle is light; model evaluation may require approval - Build: [Create a factory run and populate its declared evidence contract.](https://posttrainllm.com/docs/factory/run-lifecycle) - Modify: [Version targets, gates, and policies without rewriting completed evidence.](https://posttrainllm.com/docs/factory/run-schema) - Tune: [Change thresholds only before the candidate is scored, with rationale recorded.](https://posttrainllm.com/docs/factory/eval-protocol) - Prove: [Run the same frozen evaluator on baseline and candidate, including regression slices.](https://posttrainllm.com/docs/recipes/eval-gate) - Package: [Compile the complete run into a public before/after report and terminal decision.](https://posttrainllm.com/docs/factory/report-card) - Inspect: [FactoryRunCommand.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/FactoryRunCommand.swift) - Inspect: [build_fine_tune_report_card.py](https://github.com/PostTrainLLM/posttrainllm/blob/main/scripts/factory/build_fine_tune_report_card.py) - Run: [`posttrainllm factory-run`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm eval-gate`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm eval-compare`](https://posttrainllm.com/docs/cli-reference) ### Make the model act What turns model output into a safe, useful local agent? Separate model capability, constrained output, validation, tool policy, state, and serving performance in one trace. [Open the learning path](https://posttrainllm.com/learn#path-runtime-and-agents) #### [Local tool-using agent](https://posttrainllm.com/learn/artifacts/local-tool-agent) agent runtime runnable lab An OpenAI-compatible local model server plus a multi-turn tool loop with schema validation and explicit execution policy. **Workload:** local inference; model size controls RAM and latency - Build: [Serve a local artifact, then run one bounded tool-using conversation.](https://posttrainllm.com/docs/agent_runtime) - Modify: [Change tool schemas, catalog routing, prompts, or execution policy independently.](https://posttrainllm.com/docs/learn/model-vs-agent) - Tune: [Tune decoding, prompt reuse, constraints, and routing before assuming more training is needed.](https://posttrainllm.com/docs/learn/advanced-llm-inference) - Prove: [Score call selection, arguments, legality, task completion, latency, and unsafe actions.](https://posttrainllm.com/docs/factory/eval-protocol) - Package: [Ship the model, tool catalog, policy, and eval gate as one bounded system contract.](https://posttrainllm.com/docs/factory/packaging) - Inspect: [AgentLoop.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/AgentLoop.swift) - Inspect: [Serve.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPTServe/Serve.swift) - Run: [`posttrainllm serve`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm agent`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm eval-bfcl`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm eval-tau-bench`](https://posttrainllm.com/docs/cli-reference) ### Understand the machine Where do the time, memory traffic, and numerical drift actually come from? Use the readable implementation as an oracle while comparing kernels and architectures with end-to-end measurements. [Open the learning path](https://posttrainllm.com/learn#path-architecture-and-kernels) #### [Kernel parity and speed receipt](https://posttrainllm.com/learn/artifacts/kernel-parity-receipt) systems benchmark guided replay A numerical and performance comparison that keeps reference math, optimized kernel, dispatch geometry, and end-to-end effect separate. **Workload:** read-only replay; fresh kernel loops require approval - Build: [Trace reference attention and matmul before inspecting their optimized counterparts.](https://posttrainllm.com/docs/learn/essential-vs-optimization) - Modify: [Change precision, tile size, workgroup shape, or architecture one dimension at a time.](https://posttrainllm.com/docs/learn/webgpu-execution-model) - Tune: [Tune the measured bottleneck rather than optimizing a convenient microbenchmark.](https://posttrainllm.com/docs/performance/fa2_forward_notes) - Prove: [Require numerical parity, warmed timing, memory evidence, and end-to-end impact.](https://posttrainllm.com/speedup) - Package: [Retain source revision, device, shapes, precision, raw timings, and parity tolerance.](https://posttrainllm.com/docs/factory/foundry-evidence) - Inspect: [model.py](https://github.com/PostTrainLLM/posttrainllm/blob/main/python_ref/model.py) - Inspect: [attention_fa2.wgsl](https://github.com/PostTrainLLM/posttrainllm/blob/main/webgpu/attention_fa2.wgsl) - Inspect: [fa2_backward_notes.md](https://posttrainllm.com/docs/performance/fa2_backward_notes) - Run: [`posttrainllm bench`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm bench-train`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm experimental train-heads`](https://posttrainllm.com/docs/cli-reference) ### Make it smaller and portable Did the artifact become cheaper to store, load, and run without losing its job? Distinguish file compression, RAM reduction, fewer operations, runtime kernel support, and retained capability. [Open the learning path](https://posttrainllm.com/learn#path-quantization-and-packaging) #### [Validated MLX specialist package](https://posttrainllm.com/learn/artifacts/mlx-specialist-package) deployable model bundle runnable lab A model or adapter exported with identity, tensors, tokenizer, prompt contract, and evidence that the consumer can load it correctly. **Workload:** conversion is bounded; validation loads the selected model - Build: [Export a .tinygpt model, adapter, or Hugging Face directory into the MLX layout.](https://posttrainllm.com/docs/recipes/mlx-export) - Modify: [Change the packaging boundary or consumer metadata without changing model claims.](https://posttrainllm.com/docs/factory/packaging) - Tune: [Choose quantization or pruning only against frozen quality, RAM, size, and speed gates.](https://posttrainllm.com/docs/techniques/quantization_expansion) - Prove: [Verify round-trip load parity, file size, peak RAM, tok/s, and target quality.](https://posttrainllm.com/docs/techniques/pruning) - Package: [Publish the bundle with model card, hashes, environment, limits, and decision.](https://posttrainllm.com/docs/factory/public-artifacts) - Inspect: [ExportMLX.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/ExportMLX.swift) - Inspect: [TinyGPT.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/TinyGPT.swift) - Run: [`posttrainllm export-mlx`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm validate`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm experimental gptq`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm experimental prune-structured`](https://posttrainllm.com/docs/cli-reference) ### Cross runtime boundaries Which parts belong in Python, Swift and MLX, Core ML, WASM, or WebGPU? Choose a runtime from capability, ownership, memory, latency, energy, distribution, and browser constraints—not platform novelty. [Open the learning path](https://posttrainllm.com/learn#path-browser-and-mac-runtime) #### [Parakeet browser ASR receipt](https://posttrainllm.com/learn/artifacts/browser-asr-case) browser inference artifact guided replay A paired browser/native ASR lab: Parakeet wins bounded WER and latency while a frozen 50x short-clip throughput gate catches fixed dispatch cost. **Workload:** eight reference-scored LibriSpeech clips against native WhisperKit - Build: [Trace model download, cache, WebGPU load, audio preprocessing, and decoded transcript.](https://posttrainllm.com/docs/techniques/parakeet-wgsl-browser-smoke) - Modify: [Change audio input, caching policy, or runtime adapter while preserving the receipt schema.](https://posttrainllm.com/artifacts/parakeet-wgsl-browser-asr) - Tune: [Tune chunking, preprocessing, and execution policy before changing model weights.](https://posttrainllm.com/docs/browser_notes) - Prove: [Recompute paired WER, proper nouns, repetitions, per-clip RTFx, and the frozen decision.](https://posttrainllm.com/artifacts/parakeet-wgsl-browser-asr) - Package: [Retain runtime versions, browser/device facts, cache state, input provenance, output, and limitations.](https://posttrainllm.com/docs/factory/reports) - Inspect: [attempts.json](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/attempts.json) - Inspect: [artifacts.ts](https://github.com/PostTrainLLM/posttrainllm/blob/main/browser/src/artifacts.ts) #### [Mac runtime boundary map](https://posttrainllm.com/learn/artifacts/mac-runtime-boundary-map) architecture decision record guided replay A comparison of owned MLX weights, Core ML deployment, Apple Foundation Models, and browser runtimes grounded in measured capability limits. **Workload:** read-only evidence synthesis - Build: [Map one use case across capability, context, RAM, energy, ownership, and distribution constraints.](https://posttrainllm.com/docs/learn/mac-mastery-map) - Modify: [Add a new runtime only with a directly comparable evidence row.](https://posttrainllm.com/docs/learn/apple-on-device-foundation-models) - Tune: [Tune routing and deployment policy; do not hide a capability gap behind serving changes.](https://posttrainllm.com/docs/learn/model-vs-agent) - Prove: [Use the same task gate plus latency, RAM, tok/s, energy, setup, and ownership constraints.](https://posttrainllm.com/docs/learn/mac-mastery-map) - Package: [Record the chosen runtime, rejected alternatives, evidence, and reversal conditions.](https://posttrainllm.com/docs/factory/reports) - Inspect: [fm_agent_bridge.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/scripts/fm_agent_bridge.swift) - Inspect: [CoreMLServe.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/CoreMLServe.swift) - Run: [`posttrainllm hf-load`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm ane-validate`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm coreml-serve`](https://posttrainllm.com/docs/cli-reference) ### Look inside without fooling yourself Which internal representation causally contributes to a measured behavior? Turn a behavioral failure into a probe, control, intervention, and decision-changing prediction. [Open the learning path](https://posttrainllm.com/learn#path-interpretability) #### [Causal probe dossier](https://posttrainllm.com/learn/artifacts/causal-probe-dossier) interpretability artifact recipe contract A logit lens, linear probe, activation patch, causal trace, or sparse autoencoder investigation with negative controls and an intervention claim. **Workload:** attach to a specific failure before running - Build: [Start from frozen success and failure traces plus a falsifiable representation hypothesis.](https://posttrainllm.com/docs/techniques/interpretability) - Modify: [Change layer, token position, feature, or control while keeping the behavioral gate fixed.](https://posttrainllm.com/docs/learn/advanced-ml-systems-eval) - Tune: [Tune probe or SAE settings on train data; reserve separate examples for the causal claim.](https://posttrainllm.com/docs/techniques/interpretability) - Prove: [Require random controls, replay stability, and a predicted behavior change under intervention.](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/recipes/registry.json) - Package: [Retain activations or hashes, probe weights, controls, intervention results, limitations, and recipe decision.](https://posttrainllm.com/docs/factory/foundry-evidence) - Inspect: [ExperimentalCommands.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/ExperimentalCommands.swift) - Inspect: [session-10-attention.md](https://posttrainllm.com/docs/learn/session-10-attention) - Run: [`posttrainllm experimental tuned-lens`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm experimental linear-probe`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm experimental causal-trace`](https://posttrainllm.com/docs/cli-reference) - Run: [`posttrainllm experimental sae`](https://posttrainllm.com/docs/cli-reference) learning lab / nine ready paths ## Learn it, inspect it, then run it. Each path has prerequisites, a repository lab, and a mastery gate. Start with foundations; use the others when their prerequisites are earned. ### [Foundations to transformer](https://posttrainllm.com/learn/paths/foundations) start here 1. Functions, data, and parameters 2. Loss and gradient descent 3. Vectors, matrices, and tensors 4. Non-linear networks and backpropagation 5. Tokenization and embeddings 6. Attention and transformer blocks **Lab:** Complete the toy calculation or implementation in Modules 1-7 and explain its tensor shapes aloud. **Mastery gate:** Trace text from tokens through embeddings, attention, MLP, logits, loss, and one gradient update without skipping a transformation. - CLI entry point: [`posttrainllm train`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm inspect`](https://posttrainllm.com/docs/cli-reference) Foundation prerequisite; applied recipes begin in later paths. - [learn/curriculum](https://posttrainllm.com/docs/learn/curriculum) - [learn/session-01-neural-net-basics](https://posttrainllm.com/docs/learn/session-01-neural-net-basics) - [learn/session-02-gradient-descent](https://posttrainllm.com/docs/learn/session-02-gradient-descent) - [learn/session-09-tensors](https://posttrainllm.com/docs/learn/session-09-tensors) - [learn/session-03-non-linearities](https://posttrainllm.com/docs/learn/session-03-non-linearities) - [learn/session-06-tokenization-embeddings](https://posttrainllm.com/docs/learn/session-06-tokenization-embeddings) - [learn/session-10-attention](https://posttrainllm.com/docs/learn/session-10-attention) ### [Training mechanics and stability](https://posttrainllm.com/learn/paths/training-mechanics) after foundations 1. Batches and masking 2. Optimizers and schedules 3. Precision and numerical checks 4. Tiny-overfit correctness gate 5. Memory and convergence diagnosis **Lab:** Inspect one successful tiny-overfit and one failed ordinary run, then diagnose data, optimization, capacity, and evaluation separately. **Mastery gate:** Given a loss curve and held-out result, identify what is proven and choose the smallest discriminating next test. - CLI entry point: [`posttrainllm train`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm bench-train`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm eval`](https://posttrainllm.com/docs/cli-reference) - [Recipe: GaLore and training stability](https://posttrainllm.com/recipes/galore-stability) - [Recipe: Optimizer and schedule selection](https://posttrainllm.com/recipes/optimizers) - [learn/session-08-training-mechanics](https://posttrainllm.com/docs/learn/session-08-training-mechanics) - [guides/training_guide](https://posttrainllm.com/docs/guides/training_guide) - [techniques/optimizers](https://posttrainllm.com/docs/techniques/optimizers) - [techniques/galore_and_stability](https://posttrainllm.com/docs/techniques/galore_and_stability) - [techniques/precision](https://posttrainllm.com/docs/techniques/precision) ### [SFT, adapters, preference tuning, and distillation](https://posttrainllm.com/learn/paths/post-training) after training-mechanics 1. SFT behavior learning 2. LoRA geometry 3. DPO and reference anchoring 4. Distillation 5. Routing and breadth retention **Lab:** Reconstruct the SQL and file-ops lineages, including the variable changed, target gain, regression, and decision at each step. **Mastery gate:** Design a recipe with data, target, reward or labels, regression slices, resource budget, stop rule, and ship/retry/reject threshold. - CLI entry point: [`posttrainllm sft`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm dpo`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm distill`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm finetune`](https://posttrainllm.com/docs/cli-reference) - [Recipe: Specialist distillation](https://posttrainllm.com/recipes/distillation) - [Recipe: Evolution strategies](https://posttrainllm.com/recipes/evolution-strategies) - [Recipe: LoRA, DoRA, and adapter geometry](https://posttrainllm.com/recipes/lora) - [Recipe: Parameter-efficient fine-tuning variants](https://posttrainllm.com/recipes/peft-variants) - [learn/session-07-behavior-learning](https://posttrainllm.com/docs/learn/session-07-behavior-learning) - [training/sft](https://posttrainllm.com/docs/training/sft) - [training/dpo](https://posttrainllm.com/docs/training/dpo) - [techniques/lora_guide](https://posttrainllm.com/docs/techniques/lora_guide) - [techniques/distillation](https://posttrainllm.com/docs/techniques/distillation) - [learn/advanced-llm-training](https://posttrainllm.com/docs/learn/advanced-llm-training) - [industry_learning_roadmap](https://posttrainllm.com/docs/industry_learning_roadmap) ### [Evaluation, evidence, and the specialist factory](https://posttrainllm.com/learn/paths/evaluation-and-factory) after post-training 1. Frontier-ceiling validation 2. Frozen baselines and heldouts 3. Slice and safety gates 4. Trace review 5. Packaging and report cards 6. Ship, route, retry, or reject **Lab:** Audit Needle, Chess, and 2048 and explain how the ruler or safety gate changed the decision before integration or training. **Mastery gate:** Reject a superficially good score when its frontier ruler, safety slice, breadth gate, or provenance is invalid. - CLI entry point: [`posttrainllm factory-run`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm eval-gate`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm eval-compare`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm inspect`](https://posttrainllm.com/docs/cli-reference) - [Recipe: SQL specialist lineage](https://posttrainllm.com/recipes/sql-lineage) - [Recipe: Needle 45M successor factorial](https://posttrainllm.com/recipes/needle2-evaluation) - [learn/session-11-evals-rewards](https://posttrainllm.com/docs/learn/session-11-evals-rewards) - [factory/eval-protocol](https://posttrainllm.com/docs/factory/eval-protocol) - [factory/run-schema](https://posttrainllm.com/docs/factory/run-schema) - [factory/reports](https://posttrainllm.com/docs/factory/reports) - [attempt-ledger](https://posttrainllm.com/docs/attempt-ledger) - [recipes/registry.json](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/recipes/registry.json) - [industry_learning_roadmap](https://posttrainllm.com/docs/industry_learning_roadmap) ### [Inference, serving, tools, and long context](https://posttrainllm.com/learn/paths/runtime-and-agents) after foundations + evaluation-and-factory 1. Autoregressive decoding 2. KV cache and prompt reuse 3. Constrained generation 4. Tool dispatch and routing 5. Speculative decoding 6. Long-context cache policies **Lab:** Calculate KV memory, compare cold versus reused-prefill latency, and trace one tool request through model output, validation, and execution policy. **Mastery gate:** Separate model capability, serving optimization, validation, and agent policy in one end-to-end trace. - CLI entry point: [`posttrainllm sample`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm serve`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm agent`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm eval-bfcl`](https://posttrainllm.com/docs/cli-reference) - [Recipe: Constrained generation](https://posttrainllm.com/recipes/constrained-generation) - [Recipe: Multi-token prediction](https://posttrainllm.com/recipes/mtp) - [Recipe: Speculative heads and decoding](https://posttrainllm.com/recipes/speculative-heads) - [Recipe: StreamingLLM and KIVI cache compression](https://posttrainllm.com/recipes/streaming-kivi) - [learn/advanced-llm-inference](https://posttrainllm.com/docs/learn/advanced-llm-inference) - [learn/model-vs-agent](https://posttrainllm.com/docs/learn/model-vs-agent) - [agent_runtime](https://posttrainllm.com/docs/agent_runtime) - [performance/kv_cache_optimization](https://posttrainllm.com/docs/performance/kv_cache_optimization) - [techniques/constrained_generation](https://posttrainllm.com/docs/techniques/constrained_generation) - [techniques/speculative_heads](https://posttrainllm.com/docs/techniques/speculative_heads) - [techniques/streaming_llm_kivi](https://posttrainllm.com/docs/techniques/streaming_llm_kivi) - [industry_learning_roadmap](https://posttrainllm.com/docs/industry_learning_roadmap) ### [Architecture and kernel execution](https://posttrainllm.com/learn/paths/architecture-and-kernels) after foundations + training-mechanics 1. Reference transformer math 2. Attention memory traffic 3. Precision and tiling 4. Sparse versus dense MoE 5. Correctness before performance **Lab:** Match one optimized attention or matmul path against the readable reference and identify the real bottleneck with a measured curve. **Mastery gate:** Explain the numerical contract, memory traffic, and why a microbenchmark may not improve end-to-end training. - CLI entry point: [`posttrainllm bench`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm bench-train`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm experimental train-heads`](https://posttrainllm.com/docs/cli-reference) - [Recipe: Sparse mixture of experts](https://posttrainllm.com/recipes/moe) - [Recipe: Precision and numerical gates](https://posttrainllm.com/recipes/precision) - [learn/llm-mechanics-fundamentals](https://posttrainllm.com/docs/learn/llm-mechanics-fundamentals) - [learn/essential-vs-optimization](https://posttrainllm.com/docs/learn/essential-vs-optimization) - [performance/fa2_forward_notes](https://posttrainllm.com/docs/performance/fa2_forward_notes) - [performance/fa2_backward_notes](https://posttrainllm.com/docs/performance/fa2_backward_notes) - [techniques/moe](https://posttrainllm.com/docs/techniques/moe) - [learn/webgpu-execution-model](https://posttrainllm.com/docs/learn/webgpu-execution-model) - [industry_learning_roadmap](https://posttrainllm.com/docs/industry_learning_roadmap) ### [Quantization, pruning, and artifact packaging](https://posttrainllm.com/learn/paths/quantization-and-packaging) after training-mechanics + evaluation-and-factory 1. Calibration and numeric formats 2. Storage versus runtime kernels 3. Pruning and physical topology 4. Load and serve parity 5. Model cards and public evidence **Lab:** Audit one int4 or pruned artifact from source weights through file size, load parity, quality, RAM, and runtime measurement. **Mastery gate:** Distinguish a smaller file, lower RAM, fewer physical operations, and faster wall-clock execution. - CLI entry point: [`posttrainllm export-mlx`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm validate`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm experimental gptq`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm experimental prune-structured`](https://posttrainllm.com/docs/cli-reference) - [Recipe: Unstructured and structured pruning](https://posttrainllm.com/recipes/pruning) - [Recipe: Quantization and storage formats](https://posttrainllm.com/recipes/quantization) - [techniques/quantization_expansion](https://posttrainllm.com/docs/techniques/quantization_expansion) - [techniques/pruning](https://posttrainllm.com/docs/techniques/pruning) - [factory/packaging](https://posttrainllm.com/docs/factory/packaging) - [recipes/mlx-export](https://posttrainllm.com/docs/recipes/mlx-export) - [factory/public-artifacts](https://posttrainllm.com/docs/factory/public-artifacts) - [industry_learning_roadmap](https://posttrainllm.com/docs/industry_learning_roadmap) ### [Browser, WASM, WebGPU, MLX, and Apple runtime](https://posttrainllm.com/learn/paths/browser-and-mac-runtime) after architecture-and-kernels + runtime-and-agents 1. WASM memory and SIMD 2. WebGPU device and kernel model 3. Browser caching and distribution 4. MLX unified memory 5. Apple Foundation Models and Core ML boundaries 6. Browser ASR case study **Lab:** Compare the historical unqualified WebGPU curve and its missing receipt, Memory64 limit, Apple action-grounding result, and Parakeet browser receipt as four different capability and evidence boundaries. **Mastery gate:** Choose the correct Mac or browser runtime from capability, memory, latency, energy, download, and ownership constraints. - CLI entry point: [`posttrainllm hf-load`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm ane-validate`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm coreml-serve`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm vlm-smoke`](https://posttrainllm.com/docs/cli-reference) - [Recipe: Parakeet WGSL browser ASR](https://posttrainllm.com/recipes/parakeet-browser-asr) - [browser_notes](https://posttrainllm.com/docs/browser_notes) - [learn/webgpu-execution-model](https://posttrainllm.com/docs/learn/webgpu-execution-model) - [learn/mac-mastery-map](https://posttrainllm.com/docs/learn/mac-mastery-map) - [learn/apple-on-device-foundation-models](https://posttrainllm.com/docs/learn/apple-on-device-foundation-models) - [techniques/parakeet-wgsl-browser-smoke](https://posttrainllm.com/docs/techniques/parakeet-wgsl-browser-smoke) - [attempt-ledger](https://posttrainllm.com/docs/attempt-ledger) ### [Interpretability with causal discipline](https://posttrainllm.com/learn/paths/interpretability) after foundations + evaluation-and-factory 1. Attention and logit inspection 2. Linear probes 3. Activation patching and causal tracing 4. Sparse autoencoders 5. Controls and intervention claims **Lab:** Take one behavioral failure and write a probe hypothesis, negative control, intervention, and decision-changing prediction. **Mastery gate:** Refuse to turn an attractive visualization into a causal model claim without an intervention. - CLI entry point: [`posttrainllm experimental tuned-lens`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm experimental linear-probe`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm experimental causal-trace`](https://posttrainllm.com/docs/cli-reference) - CLI entry point: [`posttrainllm experimental sae`](https://posttrainllm.com/docs/cli-reference) - [Recipe: Interpretability probes](https://posttrainllm.com/recipes/interpretability) - [techniques/interpretability](https://posttrainllm.com/docs/techniques/interpretability) - [learn/advanced-ml-systems-eval](https://posttrainllm.com/docs/learn/advanced-ml-systems-eval) - [learn/session-10-attention](https://posttrainllm.com/docs/learn/session-10-attention) 00 / admission rule ## Frontier is the calibration anchor, not the opponent. If a strong frontier model cannot nearly ace a benchmark, that benchmark cannot grade the Mac model. Fix or drop the ruler first. The objective is frontier capability retained per unit of model size, RAM, latency, and cost. 01–06 / the retained lab loop ## Six questions. Six durable artifacts. The narrative is the index. The linked docs and code remain the source of truth. ### Choose a target worth compressing. Can a narrow capability be specified tightly enough to measure? Start with the task, the incumbent, and a frozen success bar—not a model. A frontier model first proves the ruler can represent the capability at all. **Emits:** target.json + frozen baseline - Read: [Factory contract](https://posttrainllm.com/docs/factory/overview) - Read: [Method vs recipe](https://posttrainllm.com/docs/techniques/method-vs-recipe) - Source: [FactoryRunCommand.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/FactoryRunCommand.swift) - Source: [EvalGate.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/EvalGate.swift) ### Build data for the failure, not the benchmark name. Which decisions are missing, and where did each training row come from? Trace review turns misses into slices. Synthetic teachers are useful only when provenance, deduplication, contamination checks, and a held-out gate stay attached to the rows. **Emits:** dataset + manifest + provenance - Read: [Dataset inventory](https://posttrainllm.com/docs/dataset-inventory) - Read: [Distillation recipe](https://posttrainllm.com/docs/recipes/distill-specialist) - Source: [DatasetRegistry.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPTData/DatasetRegistry.swift) - Source: [SFTCorpus.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPTModel/SFTCorpus.swift) ### Change the smallest model that can plausibly close the gap. Does SFT, preference training, or rejection sampling improve the frozen target? The readable Swift + MLX path remains the local reference. Every intervention is a recipe with exact data, adapter, optimizer, steps, and composition—not just a technique name. **Emits:** adapter + recipe + training trace - Read: [Post-training factory](https://posttrainllm.com/docs/factory/post-training-factory) - Read: [Tool-calling case study](https://posttrainllm.com/docs/learn/tool-calling-frontier-parity) - Source: [SFT.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/SFT.swift) - Source: [FactoryRunLifecycle.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPTIO/FactoryRunLifecycle.swift) ### Make the candidate earn the claim. Did capability rise on the frozen target without unacceptable regressions? The same evaluator scores baseline and candidate. Slice metrics, legality, raw outputs, latency, RAM, throughput, and uncertainty stay visible; a benchmark whose frontier ceiling fails is rejected. **Emits:** metrics + traces + regression gate - Read: [Evaluation protocol](https://posttrainllm.com/docs/factory/eval-protocol) - Read: [Benchmark archive](https://posttrainllm.com/benchmarks) - Source: [EvalCompare.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/EvalCompare.swift) - Source: [EvalBFCL.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/EvalBFCL.swift) ### Package the boundary, not just the weights. Where is this specialist allowed to answer, and when must it redirect? A specialist artifact includes identity, base model, adapter, prompt contract, eval evidence, resource measurements, and routing constraints. Narrow depth can ship only with an honest boundary. **Emits:** validated specialist package - Read: [Packaging contract](https://posttrainllm.com/docs/factory/packaging) - Read: [Public specialists](https://posttrainllm.com/artifacts) - Source: [FactoryRun.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPTIO/FactoryRun.swift) - Source: [FineTuneReportCard.swift](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPTIO/FineTuneReportCard.swift) ### End every run with a decision. Should this candidate ship, retry, redirect, or be rejected? The report binds score delta to regressions, cost, latency, RAM, throughput, missing measurements, and source hashes. Failed attempts remain part of the system because they change what gets tried next. **Emits:** report card + ship / retry / reject - Read: [Reporting contract](https://posttrainllm.com/docs/factory/reports) - Read: [Attempt ledger](https://posttrainllm.com/docs/attempt-ledger) - Source: [build_fine_tune_report_card.py](https://github.com/PostTrainLLM/posttrainllm/blob/main/scripts/factory/build_fine_tune_report_card.py) - Source: [factory_run_lifecycle.py](https://github.com/PostTrainLLM/posttrainllm/blob/main/scripts/factory/factory_run_lifecycle.py) case files / decisions, not demos ## What the loop has actually taught us. An improvement, a regression, and a failed ruler are all useful when they force a clear next decision. ### Tool calling: training helps a weak base improved The 1.7B arc moved from stock to SFT to GRPO. SFT did most of the work; GRPO added two points, then the approach plateaued below the stronger 4B. [Read the full arc](https://posttrainllm.com/docs/learn/tool-calling-frontier-parity) 56 → 76 → 78 ### File ops: depth won, breadth regressed routed The distilled 4B aced the narrow hard gate while losing out-of-domain breadth. That is a routed specialist result, not a general model upgrade. [Inspect the package](https://posttrainllm.com/artifacts/qwen3-4b-file-ops-distilled) 58 → 100 / 59.6 → 42.3 ### Character 2048: no intelligence gradient rejected ruler The strongest completed frontier attempt did not clear the frozen 1.10× bar over paired random legal play. Training was stopped and the replay was retained. [Watch the failed attempt](https://posttrainllm.com/benchmarks/game-2048) 1.058× best frontier ratio ### Character Chess: promising, not frozen candidate The first suite showed a real gradient, but the frontier did not approach the required ceiling. The expanded candidate pool remains for human review—not small-model grading. [Inspect candidate evidence](https://posttrainllm.com/benchmarks/chess) 65% frontier development score annotated source / where to begin ## Read the system in the same order it runs. Qwen Chess made its implementation approachable by naming the modules behind each lesson. These are the equivalent entrypoints here; the repository contains the complete implementation and history. - 01 [FactoryRunCommand.swift ↗](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/FactoryRunCommand.swift) CLI composition and the top-level run boundary. - 02 [FactoryRunLifecycle.swift ↗](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPTIO/FactoryRunLifecycle.swift) State transitions, locks, manifests, and durable run identity. - 03 [SFT.swift ↗](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/SFT.swift) The readable Mac-local supervised fine-tuning path. - 04 [EvalCompare.swift ↗](https://github.com/PostTrainLLM/posttrainllm/blob/main/native-mac/Sources/TinyGPT/EvalCompare.swift) Baseline-versus-candidate comparison under one contract. - 05 [build_fine_tune_report_card.py ↗](https://github.com/PostTrainLLM/posttrainllm/blob/main/scripts/factory/build_fine_tune_report_card.py) Offline compiler from validated evidence to the public before/after report. - 06 [attempt-ledger.md →](https://posttrainllm.com/docs/attempt-ledger) The durable memory of shipped, retried, rejected, and incomplete attempts. continue ## Inspect the proof, then reproduce it. [Open the specialists](https://posttrainllm.com/artifacts) [Replay the benchmarks](https://posttrainllm.com/benchmarks) [Study from first principles](https://posttrainllm.com/docs/learn/curriculum)