<!-- agent-source-envelope -->
# posttrainllm — train a real transformer in your browser tab

> Train a real GPT-2 from scratch in your browser tab with hand-written, parity-tested WASM and WebGPU backends. A paired Apple M5 Pro receipt verifies 10.67x…

Canonical: https://posttrainllm.com/playground
<!-- /agent-source-envelope -->

in-browser playground

[✦ Feedback](https://github.com/PostTrainLLM/posttrainllm/issues/new)

Train a small **GPT** — the kind of model behind ChatGPT,
a million times smaller — from scratch, right here in this tab.
No server, no install. Watch the loss curve fall, then ask it to
write you a sentence.
[Source ↗](https://github.com/PostTrainLLM/posttrainllm)

[Decision, forecasting & learned-terminal modes ↓](https://posttrainllm.com/playground#specialists)
· one Mac-local specialist workspace

**What is this?** The learning playground: a tiny GPT —
same architecture family as ChatGPT, a million times smaller,
trained right here in your browser. Load pretrained to see
one we trained earlier, or Start training to teach a fresh
one in a few minutes. Every line of math is in the
[source](https://github.com/PostTrainLLM/posttrainllm).

**Want the Mac-side model factory?** The CLI packages
real trained modules separately: model card, prompt, eval report,
lock file, and MLX loader. Start with the
[specialist registry](https://github.com/PostTrainLLM/posttrainllm/tree/main/specialists)
or the [MLX export recipe](https://posttrainllm.com/docs/recipes/mlx-export).

→

**GPU active**
– tok/s

**Two ways:**
browse the gallery for pre-trained models (same architecture, different corpora) and hit Generate, or train your own from scratch using the controls below (~4–15 min).

## Train

Corpus

Fetch

200 KB
1 MB
2 MB
5 MB
10 MB

— pick a curated dataset —

picks load automatically · or paste your own text below

↑ Choose a .txt file

up to 5 MB · UTF-8

searches Wikipedia & concatenates the top ~20 results

GitHub raw, jsDelivr, gists — anything that serves CORS-open text

[Browse Hugging Face ↗](https://huggingface.co/datasets)

HF access token
optional — unlocks gated datasets & raises rate limit

[Create one ↗](https://huggingface.co/docs/hub/security-tokens)

Model size
preset

Custom

Hyperparameters
— preset above sets these. Click to edit individually.
▸

Model depth
layers

Model width
d_model

48
96
128
144
192
256

Bytes the model sees at once
context

Training steps
max_steps

Where it runs
backend

WASM (CPU)
WebGPU (GPU)

Examples per step
batch_size

Model
—

Estimated run time
—

Advanced

How fast it learns
learning_rate

↑
Upload model
restore a saved run

Notify me when done

## Loss curve wasm system: nominal

Step
0

Loss
–

Time left
–

Elapsed
–

More metrics ▸

Perplexity –

Holdout error –

Best holdout –

Bytes / sec 0

Train ↔ val gap –

Live sample
step –



```

```



The model's current output, refreshed every few steps. Watch random
characters become letter patterns, then words, then sentences.

## Sample live

You can **sample while training is still running** —
this is the trick. The model is a single instance in a Web Worker;
Generate runs a forward pass against the same weights training is
mutating, between steps. Sample twice during a run to watch what the
model has picked up.

Starting text
prompt

Randomness
temperature

How many bytes
tokens

Train a model, then generate text here.

Inspect & evaluate

#### Benchmark

Score the loaded model against a registered held-out set.

#### Logit lens interpretability

Per-layer predictions for the current prompt. Reveals when specific knowledge appears in the residual stream. Optionally upload a tuned-lens .lenses sidecar trained via "posttrainllm tuned-lens" for calibrated probes.

🎯 Upload .lenses



```

```



#### Ablation interpretability

Zero out one layer's attention, MLP, or whole-block contribution, then sample. Reveals how load-bearing that component is.

Layer
Target

attn
mlp
whole layer

#### Activation patching interpretability

Intervene on the residual stream at (layer, position) of the recipient prompt above. **Zero**: drop that token's representation at that depth — exposes whether it's load-bearing. **Swap**: substitute a hidden state captured from a separate donor prompt at the same coords — the full Meng et al. (2022) variant that reveals which downstream outputs causally depend on the donor's representation.

Layer
Position

Donor prompt

Donor layer
Donor pos

**What to expect:** the model has to be sized to the data.
The loss curve falling is the real signal — text quality depends entirely
on params × bytes seen.

## Watch the model think

Click any byte below to see what the model considered at that
position — the **top-10 next-byte probabilities** the
last forward pass produced, and the **per-head attention
weights** from the final transformer block (which earlier
bytes each head was looking at).

This is honest: it's the actual distribution that token was sampled
from, and the actual softmax weights from the last attention layer.
Nothing prettified. `·` marks bytes that are whitespace
or control characters.

### Top-10 candidates

### Attention from last layer · per head

Switch backend to WebGPU + re-train to see attention weights.

Your machine
detecting…

computing…
▸

computing…

Want to go further?

4 Resources
curated rabbit holes — 3Blue1Brown, Karpathy, papers, and this repo's own docs

▸

### Start here (visual + intuitive)

- [3Blue1Brown — Neural Networks](https://www.3blue1brown.com/topics/neural-networks) — the maths, visually

- [Jay Alammar — The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/) — the diagrams everyone has seen

- [Karpathy — Let's build GPT](https://www.youtube.com/watch?v=kCc8FmEb1nY) — a 2 h build, from zero

- [Karpathy — Neural Networks: Zero to Hero](https://karpathy.ai/zero-to-hero.html) — the full lecture series

### Go deeper (code + papers)

- [karpathy/nanoGPT](https://github.com/karpathy/nanoGPT) — the cleanest small GPT in PyTorch

- [karpathy/minGPT](https://github.com/karpathy/minGPT) — the original minimal build

- [Attention Is All You Need](https://arxiv.org/abs/1706.03762) — Vaswani et al., 2017

- [LoRA: Low-Rank Adaptation](https://arxiv.org/abs/2106.09685) — Hu et al., 2021

- [AdamW](https://arxiv.org/abs/1711.05101) — Loshchilov & Hutter, 2017

### This project's own write-ups

- [docs/notes.md](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/notes.md) — how every layer was built & tested

- [docs/learn/README.md](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/learn/README.md) — the active learning index

- [docs/techniques/lora_guide.md](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/techniques/lora_guide.md) — LoRA in detail

- [docs/performance/performance.md](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/performance/performance.md) — the SIMD & WebGPU speed work

- [docs/browser_notes.md](https://github.com/PostTrainLLM/posttrainllm/blob/main/docs/browser_notes.md) — WASM, Workers, OPFS, WebGPU

### Related projects

- [mlc-ai/web-llm](https://github.com/mlc-ai/web-llm) — run real LLMs in the browser (inference)

- [ml-explore/mlx](https://github.com/ml-explore/mlx) — Apple Silicon ML framework

- [Hugging Face datasets](https://huggingface.co/datasets) — where the corpus picker pulls from

- [MDN — WebGPU](https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API) — the API powering the GPU path

5 WebGPU matmul benchmark
Same matmul on WebGPU + WASM, side-by-side, with a live speedup number

▸

Runs the same matmul through the WebGPU compute kernel and the
WASM kernel, parity-checks they agree, and reports the per-shape
speedup on your hardware. Needs Chrome / Edge 113+. For the
historical end-to-end curve and its evidence limits, see
[/speedup](https://posttrainllm.com/speedup).

Not run yet.

posttrainllm — byte-level transformer trained in a browser tab
· by [Sarthak Agrawal →](https://sarthakagrawal.dev)

[Docs](https://posttrainllm.com/docs)
[Speedup](https://posttrainllm.com/speedup)
[Roadmap](https://posttrainllm.com/roadmap)
[Devlog](https://posttrainllm.com/devlog)
[](https://github.com/PostTrainLLM/posttrainllm)

training
step **0**
loss **–**
**0** tok/s
ETA **–**

## Keyboard shortcuts

?Show this sheet
EscClose any popover / dialog
⌘ / Ctrl EnterStart training
⌘ / Ctrl GGenerate from the model
TTake the tour
SShare this setup
PPause / resume training
1–5Pick a size preset (Tiny → XL)

## Pre-trained gallery

Loading gallery…

## Are you sure?

## Welcome to posttrainllm

This is the learning playground: a complete transformer that trains
from scratch right here, in this tab, with no server. Load a model,
train one, or take the tour. The Mac-side specialist packages live in
the CLI registry and MLX export docs.

Mac-local specialist lab

## Three ways to teach a small model.

Runtime disconnected

Choose an answer, forecast a series, or predict a terminal screen. One
workspace for inputs, training, evaluation, and saved models.

Start your local runtime

The specialist modes run on your Mac through Python. Your inputs stay
between this page and the local runtime.



```
uv sync --project scripts/specialists --all-extras --frozen
pnpm --dir browser run build
uv run --project scripts/specialists python scripts/specialists/lab.py serve
```



[Open local lab ↗](http://127.0.0.1:8765/playground#specialists)[Setup, models & commands](https://posttrainllm.com/docs/factory/specialist-labs)

Download the selected model first using the setup commands. The runtime
loads cached weights. Training defaults to one step; a training smoke
proves wiring, not model quality.

Every frame is predicted. No shell runs here. A tiny typing artifact
does not know shell commands.

Export resultUse trained artifact

Inspect full output


```

```



### Saved runs

Refresh runs

Configs, checkpoints and reports stay on your Mac. Reopen a completed
run after restarting; cancelled runs remain in history.

Connect your local runtime to see saved runs.

Training and qualification use your local Python runtime. Exported small
terminal models predict in this tab through WASM or WebGPU; exported
Timestral forecasts and portable Qwen decisions use WASM. Portable
decision probabilities remain uncalibrated.

Methods and upstream work: [Nish Tahir’s decision models](https://nishtahir.com/build-your-own-decision-model/) · [Mistral Timestral](https://mistral.ai/news/timestral/) · [Simon Edwardsson’s WorldTUI](https://simedw.com/2026/10/21/world-model-terminal/)
