# Model compatibility check (`model-check`) Paste a Hugging Face model URL, get a Mac-specific report: **can it run with your current setup, could it run with changes, what to do next.** The feature advises only — it never downloads weights, installs software, converts, or executes models or repository code. Specs: GitHub issues #156 and #161. Implementation: `native-mac/Sources/TinyGPTCheck/` (service + report schema), `TinyGPT/ModelCheck.swift` (CLI), `TinyGPTApp/ModelCheck{Controller,View}.swift` (app panel). The executor half is [`model-run`](model-run.md) (issue #157). ## Surfaces ```bash posttrainllm model-check posttrainllm model-check --json # evaluate against a different Mac instead of this one: posttrainllm model-check --chip "Apple M4 Pro" --ram-gb 24 --disk-gb 200 ``` The app's **Check** workspace renders the same `ModelCheckReport`; CLI `--json` output and the UI are the same schema by construction — both call `ModelCheckService.check`. ## Flow ```text URL + environment → GET /api/models/?blobs=true (manifest, tags, gated, param stats) → fetch small files only (config.json, ≤512 KB cap) → tensor names (model.safetensors.index.json, or a Range-read of a shard's JSON header — a 200 full-file response is refused; weight bytes are never fetched) → CompatibilityRules.assess (pure functions — fixture-testable) → MacEnvironment.detect (chip/RAM/disk/macOS + runtime probes) → local matching receipt (read-only; exact revision + device) → ModelCheckReport v2 (summary + operations + stages + evidence) ``` **Tensor-name layout** is the structural check that upgrades name-guessing into evidence: `HFModelLoader` consumes the standard HF convention (`model.layers.N.self_attn.{q,k,v,o}_proj`, `mlp.{gate,up,down}_proj`, `model.embed_tokens`, `model.norm`, `lm_head`), so the checker counts how many tensors match. An unlisted architecture whose tensors match (e.g. OLMo-2: 73% match) becomes `changes_required` with `hf-load` named as the verification step — `unknown` is reserved for genuinely unreadable or nonstandard layouts (GPT-2's `h.N.attn.c_attn`: 0% match). Legacy config schemas (`n_head`/`n_embd`/`n_layer`, missing `n_inner` → 4×hidden) are normalized before the strict parse. **Gated repos are assessable.** `GET /api/models/` embeds the parsed config, `transformersInfo`, `cardData`, the file manifest, and safetensors param stats even without auth — only per-file reads (config, headers) 401. The checker uses the API-embedded config as a fallback, so a gated Llama/Gemma still gets a real verdict (`changes_required`, with "accept license + HF_TOKEN" as the named access step) rather than `unknown`. A token's mere presence is not treated as access: the checker must complete one bounded authenticated metadata-file read before removing the access blocker. Truly private repos (404/401 on the API itself) remain `unknown`. Other detection paths: **PEFT/LoRA adapters** (`adapter_model.*`, `adapter_config.json` → base model) get a "compatibility is the base model's" verdict with a `model-check` handoff to the base; **remote-code repos** (`auto_map`, `transformersInfo.custom_class`) are a named `unsupported_on_checked_path` — repo-shipped modeling code is never executed by policy; **GGUF headers** are Range-read for `general.architecture` + `file_type`, so quant support is verified against what `GGUFReader` actually dequantizes (F32/F16/Q4_0/Q8_0/BF16 — K-quants report honestly as needing llama.cpp/Ollama). ## Verdict vocabulary | Verdict | Meaning | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `expected_to_work` | verified architecture, no config blockers, estimated footprint fits | | `changes_required` | reachable, but needs a change (memory, conversion, install, download) | | `unsupported_on_checked_path` | the posttrainllm MLX-Swift path can't run it — task mismatch (e.g. diffusers), MoE/multimodal/`*ForCausalLM` outside the verified set, or a config-level blocker from `HuggingFaceConfig.unsupportedReason()` | | `unknown` | repo inaccessible, unrecognized format, or unverifiable — always with the missing evidence named and a copy-ready agent prompt | Two honesty rules are load-bearing: - **"Unsupported by the checked runtime" never becomes "impossible on this Mac."** Task mismatches say so and list other documented Mac paths (diffusers, mlx-lm, Ollama/llama.cpp, transformers, Core ML). - **All memory figures are estimates** (`estimate: true`, "~" in text). Weights come from Hub safetensors stats or file sizes; resident memory assumes ~2× weight bytes for bf16/fp16 (fp32 up-convert in `HFModelLoader`), ~1.15× for MLX-packed checkpoints, plus a KV-cache allowance at an 8k-token reference context. ## Operation and execution vocabulary (schema v2) The top-level verdict remains the compact compatibility summary. Schema v2 adds the operation-specific contract that prevents "downloadable" from being mistaken for "runnable" or "tunable": | Operation | What it answers | | ------------- | -------------------------------------------------------------------------------------------------- | | `inspect` | Could the checker read enough repository metadata/config to assess it? | | `download` | Can this exact revision be fetched now, including the gated-access boundary? | | `load` | Does the checked architecture/format/runtime path have a loadable route? | | `inference` | Is bounded generation predicted or measured on this device? | | `lora_sft` | Is the repository a structurally eligible standalone base for the native adapter path? | | `agentic_use` | Has tool/template/parser behavior actually been exercised? A plain text smoke never verifies this. | Each operation is one of `supported` (static prediction), `blocked` (a known requirement or measured failure), `unverified` (insufficient evidence), or `verified_on_this_device` (matching measured receipt). These states are not synonyms: `supported` never renders as measured proof. Reports also expose the ordered stages `inspect → validate → download → load → warm_up → smoke_test → ready`. A metadata-only check completes inspection and validation, then leaves execution stages pending. Known access/runtime problems block the exact stage and every dependent stage. A measured run replaces those pending states with its passed/failed boundary. ## Environment detection `MacEnvironment.detect()` reads sysctl (chip, arch), `ProcessInfo` (RAM, macOS), and volume capacity (free disk), then probes a bounded runtime list with 6 s timeouts: `posttrainllm`, `ollama`, `llama-cli`, `lms`, and one `python3` importlib.metadata sweep covering mlx, mlx-lm, transformers, diffusers, torch, llama-cpp-python. No probe installs or loads anything. Manual overrides (`--chip/--ram-gb/--disk-gb/--macos`, or the app's "check a different Mac" fields) set `environment.source: "manual"` so a remote/web context can never pass its specs off as the user's machine. ## `model-run` — close the loop `posttrainllm model-run ` runs the check, picks the best installed runtime, downloads, executes a bounded sample, and reports measured success. Runner order: GGUF → Ollama (`ollama run hf.co/`, with `ollama serve` auto-start and a local-GGUF + `ollama create` fallback when hf.co pulls hit Xet-CDN redirect blocks); compatible safetensors → native `hf-load` (the real verification step); other safetensors → `posttrainllm-mlxrun` (MLX-Swift-LM in-process — wide arch table: MoE, VLM, packed quants — auto-downloads to the HF cache, honors HF_TOKEN) → `python3 -m mlx_lm` last. Failures cascade to the next runner with the real error surfaced. `--chat` drops into an interactive session; `--runtime` forces a specific one; gated repos require HF_TOKEN first. Every bounded `model-run` terminal outcome writes an atomic local receipt under `~/.cache/posttrainllm/model-check-receipts/` (override the directory with `POSTTRAINLLM_MODEL_RECEIPTS_DIR` for fixtures or isolated tooling). The receipt contains the immutable Hub commit resolved from the requested revision (or the literal revision when no commit was available), the exact blob path when one was requested, current device fingerprint, runtime and version when known, timestamp, bounded sample statistics, attempted paths, and bounded/sanitized stderr for failures. It does **not** store prompts, model output, credentials, or weight contents. MLX-Swift receipts include the exact prompt/generated token counts emitted by the runtime; runners that do not expose token counts retain elapsed time and output-character count without inventing tokens. A later `model-check` reads—but never creates or mutates—a receipt only when model ID, resolved commit, exact artifact path, and device fingerprint all match. This prevents an old receipt for mutable `main` from being reused after the repository moves or evidence for one GGUF blob from being applied to another. Interactive `--chat` sessions never write receipts because they are not bounded verification runs. A successful bounded plain-text smoke upgrades download/load/inference; it deliberately does not upgrade LoRA/SFT or agentic use. Receipts for another revision, another Mac, or a manually described environment are ignored. ## Tool matrix Every report carries a `tools` array (rendered as "Tools that can run this model"): for each candidate runtime — posttrainllm native, MLX-Swift-LM, python mlx-lm, Ollama, llama.cpp, LM Studio, transformers, diffusers, MLXEmbedders, whisper.cpp — the report records whether it **applies** to this model (format × task × layout) and its **availability** on this Mac (`bundled` / `installed` / `not_installed` / `unknown`). Applicable tools include the exact command to run. ## Boundaries - Metadata only. The 401/403 and 404 cases produce `unknown` with the limitation named (HF returns 401 for nonexistent repos — the report says "requires authentication," not "doesn't exist"). - `HF_TOKEN` is used as a Bearer token for Hub reads and is never copied into reports or agent prompts. - Receipt stderr is capped at 8 KiB and redacts Hugging Face tokens, Bearer credentials, common secret assignments, and credential-bearing URLs before it reaches disk. - The verified-architecture list (`CompatibilityRules.verifiedArchitectures`) is intentionally narrow: `HFConfigConverter` always builds a RoPE+RMSNorm+SwiGLU model, so an unlisted `*ForCausalLM` would load into the _wrong_ architecture silently — `unknown` is the correct answer there, with an agent-prompt handoff. Tests: `native-mac/Tests/TinyGPTCheckTests/` — pure fixtures, no network.