<!-- agent-source-envelope -->
# Artifact: Qwen3-4B File-Ops Distilled | PTLLM

> A routed 4B file-operation specialist distilled from frontier/gold trajectories, with the breadth regression disclosed in the package.

Canonical: https://posttrainllm.com/artifacts/qwen3-4b-file-ops-distilled
<!-- /agent-source-envelope -->

First specialist package

Release-ready weights

2026-06-19

This is the strongest model win in the repo: a Mac-built specialist reaches 100% on the file-ops hard gate. It is also the clearest example of why routing is mandatory, because breadth drops outside the trained domain.

agentic, distillation, file ops, BFCL

## Headline Numbers

Evidence class applies to each value: measured = retained evaluation or
runtime result; historical = recorded result not freshly reproduced;
derived = calculated or decided from records; observed = current artifact
state; not-measured = an explicit evidence gap.

**File-ops hard gate:** 100%

up from 58% stock 4B

Evidence: measured

**Heldout file-ops:** 95%

hardgen heldout suite

Evidence: measured

**Breadth after tuning:** 42.3%

down from 59.6% stock

Evidence: measured

**Artifact size:** 7.5GB

local HF/MLX safetensors directory

Evidence: observed

## Competitive Context

| System | Metric | Score | Size / Class | Comparable? | Readout |
| --- | --- | --- | --- | --- | --- |
| posttrainllm Qwen3-4B file-ops specialist | local file-ops hard gate | 100% | 4B, 7.5GB package | Direct | Domain specialist result; not a general BFCL leaderboard submission. |
| Stock Qwen3-4B | same local file-ops hard gate | 58% | 4B | Direct | Before/after delta is +42 points on the frozen domain gate. |
| Frontier calibration | same local file-ops hard gate | ~99-100% | frontier API/teacher | Direct | Used as the ceiling check for whether the eval is a usable ruler. |
| [BFCL V4 public leader](https://llm-stats.com/benchmarks/bfcl-v4) | overall BFCL V4 accuracy | 75.0% | large public model | Directional | LLM Stats snapshot for Qwen3.7 Max; it marks BFCL-V4 rows as self-reported/unverified, so this is market context only. |
| [BFCL V4 public average](https://llm-stats.com/benchmarks/bfcl-v4) | overall BFCL V4 accuracy | 61.1% | 13 tracked models | Directional | LLM Stats reports 13 self-reported rows and 0 verified rows; posttrainllm still needs a full BFCL submission for direct comparison. |
| [Qwen3.5-4B public BFCL-V4 row](https://llm-stats.com/benchmarks/bfcl-v4) | overall BFCL V4 accuracy | 50.3% | 4B | Directional | Closest public 4B-class tool-calling row in the same LLM Stats snapshot, but still not the local file-ops gate. |

Direct rows share this artifact's eval setup. Directional rows are
useful market context but should not be read as leaderboard claims.

## Measured result

| Gate | Stock | Specialist | Readout |
| --- | --- | --- | --- |
| File-ops hard gate | 0.58 | 1.00 | Domain win |
| File-ops hardgen heldout | - | 0.95 | Generalizes within file ops |
| Out-of-domain breadth | 0.596 | 0.423 | Regression; route only |

## Release Blockers

### Breadth regression

The tuned model is wrong to use as a general planner.

**Unblock:** Keep all public copy routed-only and include the negative-transfer table.

## Fine-Tune Report Card

Shipped — routed only

Not fully verified — see the report card for why

[Read the report card](https://posttrainllm.com/report-cards/qwen3-4b-file-ops-distilled)

[Machine-readable JSON](https://posttrainllm.com/report-cards/qwen3-4b-file-ops-distilled.json)

## Evidence

- [Model card](https://github.com/PostTrainLLM/posttrainllm/blob/main/specialists/qwen3-4b-file-ops-distilled/model_card.md)

- [Hugging Face model](https://huggingface.co/posttrainllm/qwen3-4b-file-ops-distilled)

- [Eval report](https://github.com/PostTrainLLM/posttrainllm/blob/main/specialists/qwen3-4b-file-ops-distilled/eval_report.json)

- [Frontier parity writeup](https://posttrainllm.com/docs/learn/tool-calling-frontier-parity)

- [Specialist registry](https://github.com/PostTrainLLM/posttrainllm/blob/main/specialists/registry.json)

## Next Release Action

Keep routed-only warnings attached and add a consumer pull/load smoke before wiring this into any app.
