<!-- agent-source-envelope -->
# posttrainllm — in-browser inference

> Run real LLMs fully client-side in your browser via WebGPU (transformers.js). Pull any ONNX model from Hugging Face by id, or pick a curated one — no…

Canonical: https://posttrainllm.com/inference
<!-- /agent-source-envelope -->

Fully client-side inference via **WebGPU** (transformers.js). Pick
a model or pull any Hugging Face repo that ships an `onnx/`
folder. Weights download to your machine and run locally — no server, no upload.
First load downloads the model (cached after).

## Choose a model and runtime

Model

- [Qwen2.5-0.5B-Instruct (~0.5 GB q4) — fast](https://huggingface.co/onnx-community/Qwen2.5-0.5B-Instruct)
- [SmolLM2-1.7B-Instruct (~1.1 GB q4)](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct)
- [Qwen2.5-1.5B-Instruct (~1 GB q4)](https://huggingface.co/onnx-community/Qwen2.5-1.5B-Instruct)
- [Llama-3.2-1B-Instruct (~0.9 GB q4)](https://huggingface.co/onnx-community/Llama-3.2-1B-Instruct-ONNX)
A1 tool-caller — available after training + ONNX export
Custom: pull from Hugging Face by id…

WebGPU (GPU, fast)
WASM (CPU fallback)

Custom Hugging Face model

Must be a repo with an `onnx/` folder (transformers.js format).
Browse [transformers.js models](https://huggingface.co/models?library=transformers.js). Big models (>~2 GB) may exhaust browser memory.

Prompt
List three uses for a paperclip.

Temperature

Max new tokens

How it works: transformers.js loads ONNX weights from the Hugging Face hub
and runs them through the ONNX Runtime Web WebGPU backend, entirely in
this tab. The repo's own hand-written WebGPU kernels (the
[playground](https://posttrainllm.com/playground)) run the
tiny from-scratch models; this page runs real pretrained LLMs.
