Competitive landscape — fine-tune, eval, interpretability
A 2026 map of the products around "make a model good at your task" and "is my agent any good," with the Mac-first whitespace called out. Evidence behind docs/sessions/2026-06-13-market-landscape-mac-first.md.
Competitive landscape (2026)
Factual map of the players a Mac-first SLM toolkit competes with or
positions against. The strategy derived from this lives in
docs/sessions/2026-06-13-market-landscape-mac-first.md;
this page is the citable evidence. Researched 2026-06-13; numbers move,
re-verify before quoting externally.
Fine-tuning platforms
| Player | Hosting | Wedge | Mac/MLX? |
|---|---|---|---|
| OpenAI fine-tuning | cloud | best base models, zero infra; SFT/DPO + RFT | no |
| Together AI | cloud | broad open-model menu, cheap LoRA/full-FT | no |
| Fireworks AI | cloud | managed RFT, PyTorch pedigree | no |
| Predibase → Rubrik | cloud/VPC | first to productize RFT (acquired Jun 2025) | no |
| Lamini | cloud + on-prem/air-gap | Memory Tuning; enterprise privacy | no |
| Modal / Replicate | cloud | serverless GPU infra (where FT jobs run) | no |
| Castform | cloud | RL on agent-trace/RAG envs; export weights | no |
| Tinker (Thinking Machines) | cloud | low-level distributed-LoRA API | no |
| OpenPipe / ART → CoreWeave | OSS + cloud | agent RL trainer (acquired Sep 2025) | no |
| Unsloth | OSS | fastest QLoRA; MPS 3–5× slower, native MLX “coming” | partial |
| Axolotl / TorchTune / LLaMA-Factory | OSS | config-driven FT | CUDA-centric (no 4/8-bit on Mac) |
| Kiln AI | OSS, local UX | local-first workbench — but delegates training out | orchestrates, doesn’t train on-device |
| MLX-LM (Apple) | OSS, local | the native Apple-Silicon LoRA/QLoRA/DoRA path | yes (library/CLI) |
Read: the commercial market is one shape — rent our GPUs, send us your data, pay per token or per GPU-hour. Mac-native training as a product is unowned; only Apple’s MLX-LM library + thin wrappers serve it, and Kiln’s local UX delegates the actual training elsewhere.
Agent eval / observability
| Player | Hosting | Wedge | Local story |
|---|---|---|---|
| Braintrust | cloud (+ enterprise self-host) | integrated eval + experiment + monitor | minimal |
| LangSmith | hybrid | LangChain-native tracing + eval | enterprise tier only |
| Langfuse → ClickHouse | OSS + cloud | most-adopted OSS observability (acquired Jan 2026) | strong (self-host) |
| Arize Phoenix | OSS + SaaS | OTel-based, self-hostable | strong |
| Galileo | cloud | guardrail models + “Insights” root-cause | no |
| Patronus AI | cloud | proprietary eval models (Lynx/GLIDER/Percival) | no |
| Humanloop → Anthropic | — | dead as standalone (acqui-hire Aug 2025) | — |
| Promptfoo → OpenAI | OSS CLI | eval + red-team, local by default (acquired Mar 2026) | strong |
| Comet Opik | OSS + cloud | Apache-2.0 tracing/eval | solid (self-host) |
| W&B Weave → CoreWeave | cloud | one-line tracing + eval dashboards | limited |
| DeepEval (Confident AI) | OSS + cloud | 50+ metrics, local-first | strong (VPC self-host) |
| Ragas | OSS lib | reference-free RAG metrics | runs anywhere |
Read: “self-host” here means your K8s/VPC, not your Mac. True on-device eval is a gap, but local-eval alone is commoditizing (Promptfoo, DeepEval, Langfuse, Phoenix all do it). The bigger gap is mechanistic “why did it do that” — every “root cause” feature is just an LLM summarizing traces, not model internals.
Interpretability tooling
| Player | State |
|---|---|
| Goodfire (Ember) | public API → partnership-only (Feb 2026); not self-serve |
| Neuronpedia | OSS, research/safety-funded; closest to “productized” interp |
| TransformerLens / SAELens | actively-maintained research libraries; no product |
| Anthropic interpretability | stays research; not a sold feature |
Read: nobody sells activation patching / logit lens / SAEs as a paid agent-debugging feature. The eval community and the interp community barely overlap.
The whitespace (one line each)
- Mac-first training as a product — owned by a library (MLX-LM), not a product. → posttrainllm’s B6 + B31.
- Eval + interp + local, fused — category-of-one; nobody combines all three. → posttrainllm already ships the fusion.
- Academic agent benchmarks as a local CI gate — BFCL/τ-bench are leaderboards, not products. → posttrainllm wrapped both (E1/E2); reframe as a workflow primitive (B32).
Consolidation (the market is being rolled up)
Predibase→Rubrik · W&B Weave→CoreWeave · OpenPipe→CoreWeave ·
Humanloop→Anthropic · Langfuse→ClickHouse · Promptfoo→OpenAI ·
Goodfire→partnership-only. The buyers are GPU-infra companies and frontier
labs. A $0-marginal-cost + data-stays-on-device + OSS-inspectable tool is
structurally outside that roll-up — no GPU meter to acquire, no ingestion
revenue to absorb.