> Canonical page: https://posttrainllm.com/playground # Playground in-browser playground ⇪ Share [✦ Feedback](https://github.com/PostTrainLLM/posttrainllm/issues) Take the tour → Train a small GPT — the kind of model behind ChatGPT, a million times smaller — from scratch, right here in this tab. No server, no install. Watch the loss curve fall, then ask it to write you a sentence. [Source ↗](https://github.com/PostTrainLLM/posttrainllm) (set synchronously in the head by an inline script that reads localStorage), so repeat visitors see a clean playground from the first paint — no late-injected layout shift. --> × What is this? The learning playground: a tiny GPT — same architecture family as ChatGPT, a million times smaller, trained right here in your browser. Load pretrained to see one we trained earlier, or Start training to teach a fresh one in a few minutes. Every line of math is in the [source](https://github.com/PostTrainLLM/posttrainllm). Want the Mac-side model factory? The CLI packages real trained modules separately: model card, prompt, eval report, lock file, and MLX loader. Start with the [specialist registry](https://github.com/PostTrainLLM/posttrainllm/tree/main/specialists) or the [MLX export recipe](https://posttrainllm.com/docs/recipes/mlx-export/). 1 Setup → 2 Watch & sample GPU active – tok/s ↺ Reset Two ways: browse the gallery for pre-trained models (same architecture, different corpora) and hit Generate, or train your own from scratch using the controls below (~4–15 min). ▶ Load from gallery ## Train ? Corpus ? Fetch 200 KB 1 MB 2 MB 5 MB 10 MB Curated Upload Wiki URL Custom — pick a curated dataset — picks load automatically · or paste your own text below ↑ Choose a .txt file up to 5 MB · UTF-8 Fetch articles searches Wikipedia & concatenates the top ~20 results Fetch URL GitHub raw, jsDelivr, gists — anything that serves CORS-open text Load [Browse Hugging Face ↗](https://huggingface.co/datasets) HF access token optional — unlocks gated datasets & raises rate limit [Create one ↗](https://huggingface.co/docs/hub/security-tokens) Save & retry Clear Model size preset ? Custom Hyperparameters — preset above sets these. Click to edit individually. ▸ Model depth layers ? Model width d_model ? 48 96 128 144 192 256 Bytes the model sees at once context ? Training steps max_steps ? Where it runs backend ? WASM (CPU) WebGPU (GPU) Examples per step batch_size ? Model — Estimated run time — Advanced How fast it learns learning_rate ? Start training Pause Stop ↺ Reset Export ▾ ⟳ Continue training steps ↓ Download .tinygpt portable format with metadata ↓ Download .safetensors Python-readable ↑ Upload model restore a saved run ⌫ Reset everything clear config, corpus, trained model Notify me when done ## Loss curve ? wasm system: nominal Step 0 Loss ? – Time left – Elapsed – More metrics ▸ Perplexity ? – Holdout error ? – Best holdout ? – Bytes / sec ? 0 Train ↔ val gap ? – Live sample step – The model's current output, refreshed every few steps. Watch random characters become letter patterns, then words, then sentences. ## Sample ? live You can sample while training is still running — this is the trick. The model is a single instance in a Web Worker; Generate runs a forward pass against the same weights training is mutating, between steps. Sample twice during a run to watch what the model has picked up. ? Starting text prompt ? Randomness temperature ? How many bytes tokens ? Generate Train a model, then generate text here. ⧉ Inspect & evaluate #### Benchmark Score the loaded model against a registered held-out set. Run #### Logit lens interpretability Per-layer predictions for the current prompt. Reveals when specific knowledge appears in the residual stream. Optionally upload a tuned-lens .lenses sidecar trained via "posttrainllm tuned-lens" for calibrated probes. Run lens 🎯 Upload .lenses #### Ablation interpretability Zero out one layer's attention, MLP, or whole-block contribution, then sample. Reveals how load-bearing that component is. Layer Target attn mlp whole layer Ablate & sample #### Activation patching interpretability Intervene on the residual stream at (layer, position) of the recipient prompt above. Zero : drop that token's representation at that depth — exposes whether it's load-bearing. Swap : substitute a hidden state captured from a separate donor prompt at the same coords — the full Meng et al. (2022) variant that reveals which downstream outputs causally depend on the donor's representation. Layer Position Zero & sample Donor prompt Donor layer Donor pos Swap & sample What to expect: the model has to be sized to the data. The loss curve falling is the real signal — text quality depends entirely on params × bytes seen. ## Watch the model think Click any byte below to see what the model considered at that position — the top-10 next-byte probabilities the last forward pass produced, and the per-head attention weights from the final transformer block (which earlier bytes each head was looking at). This is honest: it's the actual distribution that token was sampled from, and the actual softmax weights from the last attention layer. Nothing prettified. · marks bytes that are whitespace or control characters. ### Top-10 candidates ### Attention from last layer · per head Switch backend to WebGPU + re-train to see attention weights. Your machine detecting… computing… ▸ computing… Want to go further? 4 Resources curated rabbit holes — 3Blue1Brown, Karpathy, papers, and this repo's own docs ▸ ### Start here (visual + intuitive) - [3Blue1Brown — Neural Networks](https://www.3blue1brown.com/?topic=neural-networks) — the maths, visually - [Jay Alammar — The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/) — the diagrams everyone has seen - [Karpathy — Let's build GPT](https://www.youtube.com/watch?v=kCc8FmEb1nY) — a 2 h build, from zero - [Karpathy — Neural Networks: Zero to Hero](https://karpathy.ai/zero-to-hero.html) — the full lecture series ### Go deeper (code + papers) - [karpathy/nanoGPT](https://github.com/karpathy/nanoGPT) — the cleanest small GPT in PyTorch - [karpathy/minGPT](https://github.com/karpathy/minGPT) — the original minimal build - [Attention Is All You Need](https://arxiv.org/abs/1706.03762) — Vaswani et al., 2017 - [LoRA: Low-Rank Adaptation](https://arxiv.org/abs/2106.09685) — Hu et al., 2021 - [AdamW](https://arxiv.org/abs/1711.05101) — Loshchilov & Hutter, 2017 ### This project's own write-ups - [docs/notes.md](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/notes.md) — how every layer was built & tested - [docs/learn/README.md](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/learn/README.md) — the active learning index - [docs/techniques/lora_guide.md](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/techniques/lora_guide.md) — LoRA in detail - [docs/performance/performance.md](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/performance/performance.md) — the SIMD & WebGPU speed work - [docs/browser_notes.md](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/browser_notes.md) — WASM, Workers, OPFS, WebGPU ### Related projects - [mlc-ai/web-llm](https://github.com/mlc-ai/web-llm) — run real LLMs in the browser (inference) - [ml-explore/mlx](https://github.com/ml-explore/mlx) — Apple Silicon ML framework - [Hugging Face datasets](https://huggingface.co/datasets) — where the corpus picker pulls from - [MDN — WebGPU](https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API) — the API powering the GPU path 5 WebGPU matmul benchmark Same matmul on WebGPU + WASM, side-by-side, with a live speedup number ▸ Runs the same matmul through the WebGPU compute kernel and the WASM kernel, parity-checks they agree, and reports the per-shape speedup on your hardware. Needs Chrome / Edge 113+. For the historical end-to-end curve and its evidence limits, see [/speedup](https://posttrainllm.com/speedup). Run WebGPU benchmark Not run yet. training step 0 loss – 0 tok/s ETA – ↑ chart