# Fine-Tune Report Card — Qwen3-4B File-Ops Distilled > Shipped: ship only as a routed file-ops specialist; do not use as the general planner Canonical: https://posttrainllm.com/report-cards/qwen3-4b-file-ops-distilled PostTrainLLM fine-tune report card · schema v1 ## Subject **Artifact:** **Artifact id:** qwen3-4b-file-ops-distilled; **Kind:** mac-safetensors-hf; **Package dir:** specialists/qwen3-4b-file-ops-distilled; **Path:** hf://models/posttrainllm/qwen3-4b-file-ops-distilled; **Routing constraint:** ship only as a routed file-ops specialist; do not use as the general planner. Do not use for: general Pace planner; multi-domain agentic planning without routing. — Evidence state: measured — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#verdict, specialists/registry.json#do_not_use_for; **Shipped:** true **Base model:** Qwen/Qwen3-4B-Instruct-2507 (bf16) — Evidence state: measured — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#base **Candidate model:** qwen3-4b-file-ops-distilled — Evidence state: measured — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#id **Method:** frontier/gold trajectory distillation on GorillaFileSystem multi-turn tasks — Evidence state: measured — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#training_method **Owner goal:** Not recorded — Evidence state: missing — The specialist package format does not record the owner goal that framed the run. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json **Target:** Qwen3-4B File-Ops Distilled — Evidence state: measured — Sources: specialists/registry.json#name ## Decision **Blocked by:** None recorded **Decision:** ship **Evidence sources:** - specialists/qwen3-4b-file-ops-distilled/eval_report.json **Failure reason:** Not recorded **Failure reason confidence:** not-applicable **Lesson:** Not recorded **Next action:** Not recorded — Evidence state: missing — The specialist package format records no machine-readable next action. The release action lives in the public artifact registry (docs/factory/public-artifacts.md) as prose. — Sources: specialists/registry.json **Outcome label:** routed-ship **Reason:** ship only as a routed file-ops specialist; do not use as the general planner **Verification blockers:** - Primary gate `file_ops_hard_gate` baseline is `historical`, not a current measurement. - Primary gate `file_ops_hard_gate` candidate is `historical`, not a current measurement. - Primary gate `file_ops_hard_gate` has no threshold value (state `missing`). - Primary gate `file_ops_hard_gate` has no passed value (state `missing`). - Frozen-eval identity is not recorded as a current measurement. - Train/eval overlap (leakage) was not checked with current evidence. **Verified:** false ## Before and after - **Baseline:** 0.58 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0].stock_4b; **Candidate:** 1 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0].distilled_4b; **Delta:** 0.42 — Evidence state: derived — Derived from at least one non-current value; inherits the weaker provenance of its inputs. — Derived from: - specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0].stock_4b - specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0].distilled_4b — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0].stock_4b, specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0].distilled_4b; **Eval identity:** **Command:** Not recorded — Evidence state: missing — The specialist package format records no eval command, so this gate cannot be replayed from the report card alone. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0]; **Date:** 2026-06-19 — Evidence state: measured — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#evaluation_date; **Frozen:** Not recorded — Evidence state: missing — The package does not record whether this suite is frozen. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0]; **Suite:** file_ops_hard_gate; **Frontier ceiling:** 1 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0].frontier; **Metric:** file_ops_hard_gate; **Name:** file_ops_hard_gate; **Passed:** Not recorded — Evidence state: missing — The specialist package records no ship threshold for the primary gate, so a pass/fail result cannot be derived. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0]; **Role:** primary; **Sample size:** 12 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0].n; **Threshold:** Not recorded — Evidence state: missing — The specialist package format records no per-gate threshold. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0] - **Baseline:** 0.6 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].stock_4b; **Candidate:** 0.95 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].distilled_4b; **Delta:** 0.35 — Evidence state: derived — Derived from at least one non-current value; inherits the weaker provenance of its inputs. — Derived from: - specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].stock_4b - specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].distilled_4b — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].stock_4b, specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].distilled_4b; **Eval identity:** **Command:** Not recorded — Evidence state: missing — The specialist package format records no eval command, so this gate cannot be replayed from the report card alone. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1]; **Date:** 2026-06-19 — Evidence state: measured — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#evaluation_date; **Frozen:** Not recorded — Evidence state: missing — The package does not record whether this suite is frozen. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1]; **Suite:** file_ops_hardgen_heldout; **Frontier ceiling:** Not recorded — Evidence state: missing — No frontier-ceiling score is recorded for this benchmark, so it is unverified as a ruler for absolute capability. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1]; **Metric:** file_ops_hardgen_heldout; **Name:** file_ops_hardgen_heldout; **Passed:** true — Evidence state: derived — No threshold was recorded. Derived as passing because the candidate did not score below the baseline. — Derived from: - specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].stock_4b - specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].distilled_4b — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].stock_4b, specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].distilled_4b; **Role:** regression; **Sample size:** 40 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1].n; **Threshold:** Not recorded — Evidence state: missing — The specialist package format records no per-gate threshold. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[1] - **Baseline:** 0.596 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].stock_4b; **Candidate:** 0.423 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].distilled_4b; **Delta:** -0.173 — Evidence state: derived — Derived from at least one non-current value; inherits the weaker provenance of its inputs. — Derived from: - specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].stock_4b - specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].distilled_4b — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].stock_4b, specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].distilled_4b; **Eval identity:** **Command:** Not recorded — Evidence state: missing — The specialist package format records no eval command, so this gate cannot be replayed from the report card alone. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2]; **Date:** 2026-06-19 — Evidence state: measured — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#evaluation_date; **Frozen:** Not recorded — Evidence state: missing — The package does not record whether this suite is frozen. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2]; **Suite:** out_of_domain_breadth; **Frontier ceiling:** Not recorded — Evidence state: missing — No frontier-ceiling score is recorded for this benchmark, so it is unverified as a ruler for absolute capability. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2]; **Metric:** out_of_domain_breadth; **Name:** out_of_domain_breadth; **Passed:** false — Evidence state: derived — No threshold was recorded. Derived as failing because the candidate scored below the baseline on a non-primary gate. — Derived from: - specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].stock_4b - specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].distilled_4b — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].stock_4b, specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].distilled_4b; **Role:** breadth; **Sample size:** 52 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2].n; **Threshold:** Not recorded — Evidence state: missing — The specialist package format records no per-gate threshold. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[2] ## Per-slice evidence None recorded ## Cost and performance **Eval time seconds:** Not recorded — Evidence state: missing — The specialist package records no value for this metric. It is reported as not measured rather than as zero. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#performance.eval_time_seconds **Latency ms:** Not recorded — Evidence state: missing — The specialist package records no value for this metric. It is reported as not measured rather than as zero. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#performance.latency_ms **Peak rss mb:** Not recorded — Evidence state: missing — The specialist package records no value for this metric. It is reported as not measured rather than as zero. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#performance.peak_rss_mb **Tokens per second:** Not recorded — Evidence state: missing — The specialist package records no value for this metric. It is reported as not measured rather than as zero. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#performance.tokens_per_second **Training cost usd:** Not recorded — Evidence state: missing — The specialist package records no value for this metric. It is reported as not measured rather than as zero. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#performance.training_cost_usd **Training time seconds:** Not recorded — Evidence state: missing — The specialist package records no value for this metric. It is reported as not measured rather than as zero. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#performance.training_time_seconds ## Eval validity and leakage **Frontier ceiling:** 1 — Evidence state: historical — Imported from a committed specialist package rather than a canonical factory-run folder, so it lacks current run provenance (command, hashes, raw predictions). — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json#scores[0].frontier **Frozen eval:** Not recorded — Evidence state: missing — The package records no frozen held-out split identity. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json **Known limitations:** - Gate `file_ops_hardgen_heldout` has no frontier-ceiling evidence, so its absolute score is not calibrated against frontier capability. - Gate `out_of_domain_breadth` has no frontier-ceiling evidence, so its absolute score is not calibrated against frontier capability. - Values come from a committed specialist package, not a canonical factory-run folder: eval commands, dataset hashes, and raw predictions are unavailable for independent replay. **Leakage:** Not recorded — Evidence state: missing — No train/eval overlap check is recorded for this package. — Sources: specialists/qwen3-4b-file-ops-distilled/eval_report.json ## Caveats - The out-of-domain breadth ceiling is not frontier-validated yet. - The negative-transfer result is relative and directly comparable: same 52 tasks, same prompt, stock vs distilled. - The model is a fused full HF/MLX directory, not a small adapter package. - Do not use for: general Pace planner - Do not use for: multi-domain agentic planning without routing ## Source evidence - **Kind:** committed-package-file; **Label:** model card; **Path:** specialists/qwen3-4b-file-ops-distilled/model_card.md; **Sha256:** 064ba3caa1e02997963711aec9c3fb97a381265d98aab17b35480f474b7501ce - **Kind:** committed-package-file; **Label:** eval report; **Path:** specialists/qwen3-4b-file-ops-distilled/eval_report.json; **Sha256:** 56e73eeb14a0f4c3a6ccf7da5967c7044374c800bb7142e7b22b514f481a2802 - **Kind:** committed-package-file; **Label:** reproducibility lock; **Path:** specialists/qwen3-4b-file-ops-distilled/tinygpt.lock.json; **Sha256:** e28380d3b483cc8865d195bf299c1ae524cd56c512fb8a5dd3ae85310b436b22 - **Kind:** committed-package-file; **Label:** prompt contract; **Path:** specialists/qwen3-4b-file-ops-distilled/prompt.md; **Sha256:** 1507e58e65e50604b412aa6e475e0027505e699180a75f324453111d6a41b320 - **Kind:** committed-registry; **Label:** specialist registry entry; **Path:** specialists/registry.json; **Sha256:** f051c563b97adab75186d78d0adb6aa656a8cb38181d6970c877d81a572bfc6f - **Kind:** historical-record; **Label:** recorded result source; **Note:** The document the legacy score was recorded in.; **Path:** docs/learn/tool-calling-frontier-parity.md#81-climbing-the-cliff--frontier-trajectory-distillation-2026-06-16; **Sha256:** 9dd338035f67715ced672cda574555393763fab417f9667c27041af9c9543e2e - **Kind:** historical-record; **Label:** recorded result source; **Note:** The document the legacy score was recorded in.; **Path:** docs/learn/tool-calling-frontier-parity.md#84-breadth--narrow-distillation-causes-negative-transfer-2026-06-16; **Sha256:** 9dd338035f67715ced672cda574555393763fab417f9667c27041af9c9543e2e - **Kind:** external-artifact; **Label:** published weights; **Note:** Public weight location; not hashed by this compiler.; **Path:** hf://models/posttrainllm/qwen3-4b-file-ops-distilled ## Compiled from **Compiler:** scripts/factory/build_fine_tune_report_card.py **Compiler version:** 1.0.0 **Dataset hashes:** None recorded **Source id:** specialists/qwen3-4b-file-ops-distilled **Source kind:** specialist-package ## Report card id qwen3-4b-file-ops-distilled