> Canonical page: https://posttrainllm.com/inference

# Run an LLM in your browser

Fully client-side inference via WebGPU (transformers.js). Pick a model or pull any Hugging Face repo that ships an onnx/ folder. Weights download to your machine and run locally — no server, no upload. First load downloads the model (cached after).

## Choose a model and runtime

 Model

 Qwen2.5-0.5B-Instruct (~0.5 GB q4) — fast SmolLM2-1.7B-Instruct (~1.1 GB q4) Qwen2.5-1.5B-Instruct (~1 GB q4) Llama-3.2-1B-Instruct (~0.9 GB q4) A1 tool-caller — available after training + ONNX export Custom: pull from Hugging Face by id… WebGPU (GPU, fast) WASM (CPU fallback)

 Custom Hugging Face model

Must be a repo with an onnx/ folder (transformers.js format). Browse [transformers.js models](https://huggingface.co/models?library=transformers.js). Big models (>~2 GB) may exhaust browser memory.

 Load model

 Prompt List three uses for a paperclip.

 Temperature

 Max new tokens

 Generate Stop

How it works: transformers.js loads ONNX weights from the Hugging Face hub and runs them through the ONNX Runtime Web WebGPU backend, entirely in this tab. The repo's own hand-written WebGPU kernels (the[playground](https://posttrainllm.com/playground)) run the tiny from-scratch models; this page runs real pretrained LLMs.