> Canonical page: https://posttrainllm.com/artifacts/needle2-tool-selection

A failed model that won us an answer Report artifact 2026-09-03

# Needle 2: The 45M Capacity Boundary

The lab first gave Needle the oracle task family, then trained plain and distractor-negative data under standard and safety-weighted losses. Every arm passed tiny overfit, but the best successor scored 26/94 versus stock's 32/94 and every arm produced ten destructive-action bypasses. The safety stop prevented a wasteful sweep and kept sealed V2 unopened. That is a useful capacity-boundary result, not a product candidate.

- tool calling
- tiny model
- safety
- routing
- negative result

## Headline Numbers

Evidence class applies to each value: measured = retained evaluation or runtime result; historical = recorded result not freshly reproduced; derived = calculated or decided from records; observed = current artifact state; not-measured = an explicit evidence gap.

Tiny-overfit gates

 4/4 all arms reached exact 1.0 measured

Best successor

 26/94 27.7% vs stock 34.0% measured

Destructive bypasses

 10 vs 2 best arm vs stock; unsafe stop measured

## Competitive Context

**System**

**Metric**

**Score**

**Size / Class**

**Comparable?**

**Readout**

 Needle 2 full catalog exact tool selection 34.0% 94 public fixtures Direct Included eight out-of-scope false calls and two destructive-action calls.

 Needle 2 oracle task catalog exact tool selection 38.3% same 94 fixtures Direct The upstream task family was supplied; this does not prove Needle can discover it.

 Best trained 45M successor exact tool selection 27.7% 94 public-development fixtures Direct Distractor plus safety weighting was the best arm, but remained below stock and failed the destructive-action stop.

Direct rows share this artifact's eval setup. Directional rows are useful market context but should not be read as leaderboard claims.

## Successor factorial

**Training data**

**Loss**

**Exact**

**Destructive bypasses**

 Plain Standard 24/94 10

 Plain Safety 24/94 10

 Distractor Standard 25/94 10

 Distractor Safety 26/94 10

## Release Blockers

### 45M capacity boundary

All four trained arms passed memorization but regressed public exactness and multiplied destructive bypasses.

 Unblock: Open a fresh scoped experiment for a 1.7B successor under the same frozen public and sealed gates.

## Evidence

- [Baseline and catalog review](https://posttrainllm.com/docs/techniques/needle2-baseline-review/)
- [Successor recipe and learning lab](https://posttrainllm.com/docs/techniques/needle2-successor-factorial/)
- [Machine-readable successor result](https://github.com/PostTrainLLM/posttrainllm/blob/main/evals/verified-wins/needle-successor-result-v1.json)
- [Machine-readable catalog receipt](https://github.com/PostTrainLLM/posttrainllm/blob/main/evals/needle2/bounded-catalog-ablation-v1.json)
- [Complete experiment archive](https://posttrainllm.com/experiments)

## Next Release Action

Close the 45M recipe-search lane. Keep the complete factorial as a hands-on learning artifact; any 1.7B successor is a fresh experiment.