Every experiment. Including the ones that said no.
This is the complete evidence-backed history of posttrainllm. Nothing is queued or quietly “not tried.” Each historical question ends in a measured result or a final disposition, with the lesson preserved for your next fresh experiment.
tooling and smoke exist, but no frozen candidate-selection dataset, baseline, or model run was approved before project closure
Why it stopped
The proposed run never acquired a frozen target/eval contract, and running it retrospectively would start a new experiment rather than complete an existing measured claim.
Learning
Candidate selection remains a plausible future method, but an unfrozen recipe is not unfinished evidence for this project.
Disposition
Closed for this project. Reopen only as a fresh owner-led experiment after the learning phase with new data and a frozen gate.
plan renderer exists, but the required candidate-selection reward surface was never validated and no model run was authorized
Why it stopped
The experiment's explicit prerequisite did not exist, so an OAPL-style update would not have had a trustworthy reward and could not answer its intended question.
Learning
Offline policy updates are downstream of reward validation; skipping the prerequisite is not a useful experiment.
Disposition
Closed for this project. A future fresh experiment must validate its reward surface first.
The unchanged public weights reproduce a strong routed file-operations win but regress held-out breadth by five net paired cases, so they fail the general-successor gate.
Learning
ReST produced a real depth-and-safety specialist, not breadth recovery; routing is the win and general replacement is the failed hypothesis.
Disposition
Retain the package only behind the file-operations route; any broader successor needs breadth-preserving data or optimization on the same frozen ruler.
ReST `4/4 fileops, 4/4 vehicle`; multibackend `4/4 fileops, 4/4 vehicle`; frontier `8/8`; both emit the same 19 requested calls. Both runner counters report zero schema failures and zero listed unexpected side effects; actuator coverage is incomplete.
Why it stopped
The eight-task slice does not distinguish the historically retained and rejected candidates, so it cannot validate the comparative selection boundary or overturn the broader rejection.
Learning
A frontier ceiling is necessary but does not make a simple smoke discriminating; a zero fixed-list side-effect counter also does not prove complete safety.
Disposition
Preserve the ReST file-operations route and multibackend general-use rejection. Before future acceptance, freeze primary/regression slices representing historical failure classes and review every decoded mutator arguments, multiplicity and order at the execution boundary.
datasets, builder scripts, and planned recipes survive; no per-version score was recorded
Why it stopped
No per-version result was kept. The v11 ship gate names this directly: v1 through v10 cycled because there was no fixed bar, no fixed eval, and no immutable record of pass criteria.
Learning
An unmeasured version is indistinguishable from a version that never ran. Four consecutive attempts left no score behind.
Disposition
Do not attempt to reconstruct scores for v1-v4; treat the series as starting at v5, the first version with a recorded number.
10/19, 8/19, 2/19, 0/19, then 14/19 at 100 steps and 17/19 at 300 steps, reaching 19/19 after a serving-side minLength fix; 4/15 (26.7%) on fm-fixtures-v2
Why it stopped
The apparent collapse was two harness bugs, not training: SFT rows omitted the Pace system prompt that serve and eval both included, and a brace-counting JSON extractor was faulty.
Learning
A four-way score collapse traced to prompt mismatch between training and serving. Pin the prompt, schema, and grammar triple across train, serve, and eval before believing any comparison.
Disposition
Recover the full sweep with `git show d1efcb8^:docs/prds/factory-pace-planner-v6_1.md` before repeating any step-count ablation; it is the only one in the series.
tools-in-prompt serve, verb-enum grammar, and data/eval harnesses shipped; the SFT was never run
Why it stopped
The version was never trained, and later v8-v11 planner work replaced its proposed prompt/schema direction and supplied the relevant capability evidence.
Learning
v7 is a numbering gap, not a training version. The eleven-version count includes a version that was never trained.
Disposition
Closed as a numbering gap. Do not cite v7 as evidence or revive it outside a new owner-led experiment.
reported 11/15 (73.3%) on fm-fixtures-v2, later re-measured at 5/15 (33.3%) from the same weights and same fixtures
Why it stopped
The headline score did not reproduce. Re-running the same eval against the same baked weights yielded 33.3%, and the gap was root-caused as prompt and schema config drift rather than any model change.
Learning
The 73% phantom. A score is only a score if the prompt, schema, and grammar it was measured under are recorded with it.
Disposition
Treat 33.3% as the working v8 baseline and never compare across versions without pinning the harness config.
70% on new compose fixtures with non-compose held at the v8 level; as-shipped re-measurement 9/15 (60.0%); unhappy-path n=130 gave 0/40 ambiguous, 13/60 out-of-scope, 1/30 destructive
Why it stopped
It shipped as the specialist but never became Pace's runtime planner, and the unhappy-path drill put it at a capacity wall: 0% ambiguous and 3% destructive.
Learning
Winning the happy-path fixture says nothing about ambiguous, out-of-scope, or destructive inputs, which is where the product actually needed the planner.
Disposition
Keep as the last shipped specialist; Pace runs qwen3-30b-a3b instead.
full recipe, action registry, schema, and eval plan are committed; no score was ever recorded and it is not established that training completed
Why it stopped
No v10 score exists anywhere in the repository. The last status recorded is that training was pending, while the v10 corpus went on to become the base of v11.
Learning
v10 repeated v6's move of changing the output representation, and because it was never measured the v6 regression could not be checked for recurrence. Its unmeasured corpus was then inherited by v11.
Disposition
Do not reuse a v10-derived corpus without first measuring the representation change in isolation.
failed all six gate dimensions: BFCL pace-12 27.1% against a 40% bar, out-of-scope 0/30 against 80%, ambiguous 0/20 against 50%, destructive 5/10 against 90%, with train loss 0.001
Why it stopped
A 0.6B planner did not reach the fixed bar on any dimension despite the lowest training loss in the series, and the first v11 attempt was voided outright by the DoRA infrastructure failure.
Learning
Train loss 0.001 alongside 0% on out-of-scope and ambiguous inputs is the capacity ceiling stated plainly. The one-shot fixed gate was what finally ended the cycle.
Disposition
Do not train v12. The specialist planner track is closed and Pace ships qwen3-30b-a3b.
38 contrastive rows moved ambig only `0% -> 5%` while OOS regressed `80% -> 33%` in the retrospective; drilldown records the same attempt as `2%/40%/50%` on ambig/oos/destructive
Why it stopped
Small-corpus LoRA on a competent base caused catastrophic interference across planner dimensions.
Learning
Fine-tuning a strong base on a thin behavior slice requires coverage for every dimension that must not regress.
Disposition
Do not fine-tune the planner on narrow clarify rows unless the corpus also covers OOS, destructive, happy path, and breadth gates.
ambig moved `0% -> 20%`, while OOS dropped `78% -> 55%` and destructive stayed `67%`; not shipped on top of Gemma
Why it stopped
The shim improved ambiguity detection for the 4B floor but over-triggered and became unnecessary once Gemma reached the same ambig score with better OOS/destructive behavior.
Learning
Rule wrappers can expose the missing behavior, but they still need the same regression gates as model updates.
Disposition
Keep as an option only if shipping the 4B footprint floor; do not stack it on the stronger Gemma path.
larger browser presets hit Memory64 OOB; matching Node direct-export reproducer passed XL; `INITIAL_MEMORY=268435456` fixed up to XL while Behemoth still exercises growth
Why it stopped
The original retries treated the OOB as a model-size problem before the matching reproducer showed a browser pthread plus SharedArrayBuffer memory-growth race.
Learning
A reproducer must match the production host and call path before it can explain a browser-only WASM failure.
Disposition
Keep the higher initial memory for supported presets and only claim Behemoth after a SAB-view-aware growth path lands.
the historical curve remains unqualified except at Large, where a 2026-09-02 Apple M5 Pro ABBA receipt measured `10.67x` median WebGPU-over-WASM speedup with `4.72%` maximum paired final-loss drift and zero runtime errors
Why it stopped
The original headline made a shape-dependent result look universal, and the retained tree cannot establish that the historical curve used a real hardware adapter rather than a software fallback.
Learning
Qualify performance one frozen shape and machine at a time; a retained adapter identity plus alternated paired timings can convert one point without laundering the surrounding curve.
Disposition
Keep Large as the measured M5 Pro point; independently reproduce Small, Medium, and XL before promoting the historical curve.
a trained WebGPU run was lost because `webgpu/gpu_model.ts` had no checkpoint serialization; `exportState` was then implemented and worker plumbing updated
Why it stopped
The training path could produce useful weights that the WebGPU backend could not export, and auto-cleanup destroyed a real trained artifact.
Learning
Long-running browser training is not product-complete until the smallest export path is smoke-tested before the long run.
Disposition
Smoke export on a short WebGPU run before any long browser training claim.
Apple M5 Pro / Metal 3 ABBA run measured `10.67x` median Large-preset speedup over WASM with `4.72%` maximum paired final-loss drift and zero runtime errors
Learning
Hardware-qualified browser WebGPU can deliver a double-digit training-step win while preserving the frozen numerical-loss gate.
Disposition
Retain Large as the verified Apple M5 Pro result; Small, Medium, XL, other browsers, and other hardware require independent receipts.
microbench `3-4x`, but end-to-end WASM training only `1.03-1.06x` after forward blocking
Why it stopped
The isolated microbench overstated impact because the real WASM training workload was threaded, already SIMD-vectorized, and dominated by backward matmuls.
Learning
Optimize and report on the real workload, not only on isolated single-kernel microbenches.
Disposition
Keep the bit-exact forward kernel for inference, but judge future perf work with end-to-end training and generation benches.
baseline `232 tok/s`; Medusa 50-step `202 tok/s` at `21.7%`; Medusa 300-step `375 tok/s` at `20.8%`; EAGLE-2 50-step `200 tok/s` at `26.5%`, all bit-identical where verified
Why it stopped
The heads trained and verified losslessly, but acceptance stayed far below the paper recipes because this run lacked long training, logit distillation, tree attention, and a full EAGLE transformer draft block.
Learning
Speculative heads are a valid local speed path, but the production recipe is data/training heavy rather than a quick sidecar smoke.
Disposition
Revive only with self-generated data, logit-KL distillation, tree verification, and a real acceptance-rate gate.
Dense MLP `842 K` params loss `6.09 -> 1.76` at `55.6 step/s`; MoE 4 experts top-2 `2.42 M` params loss `5.95 -> 1.68` at `29.3 step/s`
Why it stopped
The MoE architecture, save/load, sample, eval, and inspect paths work, but the current compute path runs every expert and therefore does not deliver sparse-dispatch savings.
Learning
MoE is a capacity feature until sparse gather/scatter or grouped matmul lands; it should not be marketed as a compute-saving win yet.
Disposition
Return to MoE only with MLX scatter_add/grouped matmul support or a scoped custom Metal sparse-dispatch kernel.
all four items ON reached median `6.8 step/s` vs `5.0 step/s` baseline, `+36%`, with final loss `4.188-4.192` unchanged
Why it stopped
The bundle produced a real host-side speedup, but QoS and compiled cosine LR were individually in the noise, prefetch stayed single-digit, and open CPU items remained.
Learning
The big win was fused accumulation inside the compiled trace; individual CPU tweaks need isolation before being credited.
Disposition
Use the bundle as shipped, but keep Rust-FFI BPE, preallocated CPU buffers, and SVD-on-CPU AMX as separate future attempts.
mmap saved `10-20 ms` and `25 MB` peak RSS on `demo.tinygpt`; lazy embedding deferred `262.1 KB`; package tests passed `43/43`
Why it stopped
The cold-start path improved the measured demo case, but the 250 MB target file was missing, async spinner overhead added about `30 ms`, and persistent Metal pipeline caching was infeasible through public MLX-Swift APIs.
Learning
mmap and direct byte construction are good scaling levers, but large-model cold-start claims need large-model measurements.
Disposition
Re-measure on a real 50-250 MB checkpoint and keep persistent Metal cache work parked unless MLX-Swift exposes pipeline descriptors.
Checkpointing recovered large-model memory but hurt small-model memory and speed, and the OOMGuard estimator still reports uncheckpointed activation projections.
Learning
Activation recompute is a scale-dependent lever; it should stay opt-in for models where memory, not step time, is the blocker.
Disposition
Update memory estimation for `--grad-checkpoint` and consider selective layer checkpointing before broad default use.
prompt-cache hot run `1.005 s -> 0.106 s`; pre-alloc decode stayed `706 -> 711 tok/s` with physical cache flat at `6.3 MB`
Why it stopped
Persistent prompt cache delivered the TTFT win, but pre-alloc was kept opt-in, pre-alloc does not compose with KIVI or StreamingLLM, and some cache keying edge cases remain.
Learning
Reuse of prefilled KV is the user-visible win; cache storage policy changes need composition gates before becoming defaults.
Disposition
Keep prompt-cache as the main shipped path and revisit pre-alloc defaults only after KIVI/StreamingLLM interactions are designed.
StreamingLLM reduced 500-token cache `12.6 MB -> 6.4 MB`; int8 KIVI matched greedy baseline `100%` and int4 averaged `96%` prefix match
Why it stopped
The cache compression worked in smoke tests, but int4 precision still used int8 storage bytes, K scales recompute every step, and no held-out perplexity gate measured quality.
Learning
KV compression can preserve greedy decode locally, but storage format and quality gates matter before making long-context claims.
Disposition
Add packed int4 storage or grouped KIVI scales only after a cached-forward perplexity or trajectory-match eval exists.
audit found SDPA call sites and tied-embedding build/save/load paths clean; build succeeded and sample smoke reached `603 tok/s` on demo and `261 tok/s` on mega5
Learning
The suspected SDPA mask/contiguity and tied-embedding duplication issues were not present.
Disposition
Keep this as positive verification; only reopen if new SDPA call sites or untied embedding save paths are added.
the recipe and acceptance harness shipped; later Qwen3-4B file-ops distillation and BFCL-family evaluations exercised the specialist contract with measured results
Why it stopped
The original A1-specific adapter run was never executed, but its question and workflow were overtaken by later measured tool-calling specialists and should not be recreated for historical bookkeeping.
Learning
Reusable infrastructure can be superseded by stronger downstream evidence even when its first named model run never happened.
Disposition
Closed as superseded. Use the measured file-ops/ReST lineage for evidence and start any new tool-caller as a fresh experiment.
validation exact `6.16% random -> 10.54%` (+4.38 points), test `6.23% -> 10.33%` (+4.10); raw and guarded policies each won 0/6 games against random legal play
Why it stopped
Completion-only SFT learned a small held-out move preference but missed the frozen +10-point promotion gate and produced no full-game win advantage over random legal play; eight guard interventions converted no wins.
Learning
Legal constrained decoding and lower move-prediction loss are not game intelligence; require full-game transfer before scaling a tiny specialist.
Disposition
Do not run the 100k, 1M, or 2M stages under this recipe. Preserve the failed artifact and select a different bounded specialist target.
overall exactness moved 32/94 -> 36/94, while Pace regressed 10/28 -> 6/28, file operations stayed 0/6, and 3 out-of-scope prompts still produced actions
Why it stopped
Oracle task-family routing reduced schema size and latency but did not repair capability or safety; the target Pace slice became worse.
Learning
Catalog restriction is a latency lever, not a capability or safety rescue for this base model.
Disposition
Reject this integration path; require a materially different training hypothesis and a new frozen gate for any future Needle-derived work.
all four arms passed 100% tiny overfit, but public-dev exactness was 25.5%, 25.5%, 26.6%, and 27.7% versus the 34.0% incumbent; every arm produced 7-9 out-of-scope false actions and all 10 destructive bypasses
Why it stopped
The 45M model memorized the tiny gate but neither distractor-aware data nor explicit safety examples transferred to safe 94-case tool selection.
Learning
Tiny overfit proves wiring, not capacity; the interaction treatment recovered only two cases over plain training and could not learn destructive confirmation at this scale.
Disposition
Close 45M recipe search and advance the same frozen selection-and-safety task to the 1.7B class as a fresh experiment.
On the same eight LibriSpeech clips, browser Parakeet v3 scored 0/82 word errors versus native WhisperKit's 7/82, matched all four proper nouns, had zero repetition errors, and decoded 3.51x faster; its 33.84x median short-clip real-time factor missed the frozen 50x bar
Why it stopped
Quality, paired native latency, repetition, and offline-warm gates passed, but fixed dispatch overhead on 2-7 second clips held the preregistered median throughput below 50x; the optional first download is 652.7 MiB.
Learning
Browser WebGPU ASR can beat a native large-model incumbent on both bounded WER and latency while still failing an absolute throughput target; short-clip fixed cost and long-form throughput are different measurements.
Disposition
Close this frozen gate without relabeling it; batching, long-form amortization, Safari, or a larger WER set are fresh experiments.
the ordered volume ladder found its first reproducible boundary at 2,000 words/event or 8,000 submitted words/day
Why it stopped
The boundary occurred in the neutral arm and therefore measures raw context volume or context management, not personal-obligation semantics; full provider/model provenance is incomplete.
Learning
The supported result is a neutral context-volume boundary, not a psychological or family-tension effect.
Disposition
Closed as report-only boundary evidence with the interpretation constrained to raw volume.
100 stable candidates produced 86 admitted positions, but the best recorded frontier screen reached only 75% exact agreement
Why it stopped
The frontier calibration did not approach the required ceiling, external-model coverage was incomplete, and the model alias was mutable, so the suite could not be frozen as a fair ruler.
Learning
A benchmark candidate set is not a benchmark until the frontier ceiling validates the ruler.
Disposition
Closed as candidate-only evidence; design any future chess ruler independently after the learning phase.
pinned Sonnet matched random legal play at 0.995x mean score and pinned Opus reached only 1.058x over three complete pairs before a provider disconnect
Why it stopped
The proposed character interface did not expose a reliable frontier intelligence advantage, so training a 30-50M student would measure ruler noise.
Learning
Validate the teacher and interface before spending compute on a specialist; a failed ruler should stop training.
Disposition
Closed in the current character form; keep the negative benchmark replay public.
the arena retained four chess matches and seven complete 2048 pairs, while every rating remained unqualified
Why it stopped
Chess missed minimum-match and forfeit gates, while 2048 missed minimum-pair or complete-source gates; cross-game ranking would have overstated sparse development evidence.
Learning
A rating calculation is not a public rating until each underlying competition family passes its qualification contract.
Disposition
Closed as a transparent candidate report with no Elo, human-equivalence, or specialist-win claim.
123 development fixtures: student v1 58.5% full pass, reported MPS routing p50 16 ms.
Why it stopped
Teacher labels and limited coverage left single-target click routing weak. Synthetic sealed data and inspected development misses do not qualify a shipping gate.
Learning
Student v1 left single-target clicks weak; label policy and independent evaluation must precede further teacher distillation.
Disposition
Continue issue #200 with a label policy, fresh adjudicated acceptance cases, and per-class regression gates.
123 development fixtures: student v2 53.7% full pass, reported MPS routing p50 18 ms.
Why it stopped
Agreement filtering reduced development full pass rather than repairing label policy. Synthetic sealed data and inspected development misses do not qualify a shipping gate.
Learning
Agreement filtering lowered student v2 full pass to 53.7%; agreement alone does not repair a disputed action policy.
Disposition
Continue issue #200 with a label policy, fresh adjudicated acceptance cases, and per-class regression gates.
123 development fixtures: student v3 65.0% full pass, reported MPS routing p50 14 ms.
Why it stopped
Goal-phrased data improved clicks but full pass remained below the local 71.5% baseline and RESPOND regressed. Synthetic sealed data and inspected development misses do not qualify a shipping gate.
Learning
Goal-phrased training improved student v3 clicks but regressed RESPOND; report per-class gates alongside aggregate full pass.
Disposition
Continue issue #200 with a label policy, fresh adjudicated acceptance cases, and per-class regression gates.
Jeff tools with 32 Pace options: 38.2% full pass, 10.0% ASK_USER, p50 143 ms; 36 repeat guards.
Why it stopped
The unchanged label contract remained weak on step control and clarification.
Learning
Jeff tools with the existing label contract remained weak on clarification and step control; a tool-oriented model is not a qualified policy replacement.
Disposition
Reject the 32-option substitution; preserve a separate upstream-format trial.
96 generated policy cases: rules 96/96, Qwen3.5-4B 4-bit 48/96 with zero errors; Qwen route p50/p95 394.7507/953.5587 ms, model bytes 3061316115.
Why it stopped
Qwen failed all ASK_USER, RESPOND and named-tool cases. The rules-perfect template set establishes regression behavior only, not real-task generalization.
Learning
Preserve raw failures, generated provenance and unknown calibration/hardware measures; a fast or rules-perfect result does not qualify a planner replacement.
Disposition
KEEP the current native planner. The owner withdrew the 400-500 handwritten-case requirement; any future learned replacement needs independent task evidence, not relabeled generated gold.
2026-10-23 H200 check afe0a26a817f473c: 0.8443x aggregate, reported 95% interval 0.8316-0.8570x; correctness passed with no flags.
Why it stopped
Applying the small-tile split-K path at larger token counts caused measured per-shape regressions and an aggregate below baseline.
Learning
A fused kernel needs a measured dispatch range; correctness alone does not qualify a performance change.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 9cd67921edfc40b7: 1.0088x aggregate, reported 95% interval 0.9936-1.0239x; correctness passed with no flags.
Why it stopped
The reported aggregate interval includes 1x despite improvements at individual large-token shapes.
Learning
Keep per-shape improvements separate from demonstrated aggregate gains.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 scored run 85ab373d8dcc44f7: 1.0534x aggregate, reported 95% interval 1.0376-1.0692x; correctness passed with no flags. Earlier BK64 scored run 5b1f1c6523524705 measured 1.0309x; tile variants have overlapping aggregate intervals.
Why it stopped
The demonstrated gain is limited to the organizer H200 workload; a larger tile advantage over the earlier scored variant is not statistically established by the aggregate intervals.
Learning
Preserve tiny-token and large-token baseline paths, and judge tile choices using both target-shape timing and aggregate uncertainty.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 scored run f141c8d265234a99: 1.1357x aggregate, reported 95% interval 1.1187-1.1528x; correctness passed with no flags.
Why it stopped
This measures attention preparation plus the existing attention implementation on one organizer workload, not application throughput or Mac performance.
Learning
Fusion must preserve BF16 rounding, the exact RoPE table, cache slot writes and non-unit scale fallbacks.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 scored run 84fb1b656d3c44e2: 1.0645x aggregate, reported 95% interval 1.0485-1.0804x; correctness passed with no flags.
Why it stopped
The gain is specific to the supplied expert-parallel routing distribution and organizer H200 workload.
Learning
Use the global expert count for routing density; do not confuse that heuristic with local expert IDs used by alignment.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 scored run 0a072555103b4e2d: 1.0254x aggregate, reported 95% interval 1.0100-1.0408x; correctness passed with no flags. Its preceding check 5e444821e58e404a measured 1.0181x overall but regressed at 32 and 128 tokens.
Why it stopped
Component gains did not compose uniformly: the full-layer check regressed at 32 and 128 tokens despite a positive aggregate.
Learning
Measure attention and MoE together while preserving the full layer router precision and stream ordering; component speedups do not multiply into a layer claim.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 scored run bc25711588004579: 1.0351x aggregate, reported 95% interval 1.0196-1.0506x; correctness passed with no flags.
Why it stopped
The cutoff preserves baseline MoE tiling below 256 tokens based on measured layer regressions; results establish only this organizer H200 workload.
Learning
Use combined-layer evidence to narrow component optimizations, and rerun correctness and timing after restoring fallbacks.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 035753a0aef44f2f failed five correctness cases during Triton compilation: integer zero fallback values could not be cast to FP8 E4M3 loads; no timing score was produced.
Why it stopped
Masked FP8 loads used an integer other=0 value that the judge Triton compiler could not cast to fp8e4nv.
Learning
Python syntax and lint checks do not validate Triton backend types; masked FP8 loads need a compatible floating-point fill value.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 scored run 1095cf9b43bf49a0 passed with no flags: 1.0673x aggregate, reported 95% interval 1.0513-1.0834x. All four targeted one-token projections measured 1.13-1.20x versus SGLang, repeating the preceding check gains.
Why it stopped
The aggregate interval overlaps the previous dense candidate; targeted projection gains do not establish the exact aggregate improvement or gains outside one-token dispatch.
Learning
A single streaming GEMV can remove split-reduction overhead for one-token projections while retaining existing dispatch for other shapes.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 scored run 20b3199ecbd847f2 passed with no flags: 1.2236x aggregate, reported 95% interval 1.2053-1.2420x. Small batches measured 1.22-1.49x versus SGLang; the 32-token case measured 0.988x. The preceding check measured 1.2201x aggregate.
Why it stopped
The existing 32-token path retains a small measured regression, and the gain applies to the organizer workload rather than application throughput.
Learning
Profile wrapper dispatch as well as kernel arithmetic: the existing fused norm kernel removed several small-batch framework launches while preserving FP32 reduction and invalid-expert masking.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 34c48a28a66d46d3 passed correctness with no flags: 1.0564x aggregate, reported 95% interval 1.0405-1.0722x, below the selected 1.1357x attention candidate. Full-context verification at batches 16 and 32 measured 0.553x and 0.537x versus SGLang.
Why it stopped
Applying the same split-K kernel to eight-token long-context verification nearly doubled latency; larger sliding verification also lost against the existing fused-preparation candidate.
Learning
A custom attention kernel can help decoding while losing verification; compare each dispatch regime with the current incumbent as well as the official baseline.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 scored run 1c749497f0bf4c89 passed correctness with no flags: 1.1070x aggregate, reported 95% interval 1.0903-1.1236x, with all 12 benchmark shapes faster than SGLang. The preceding check measured 1.1086x.
Why it stopped
This qualifies the MoE norm fusion inside the organizer decoder-layer workload; it does not establish serving throughput or compose the separate new GEMV and split-K attention kernels.
Learning
A wrapper-level fusion can carry into the full layer, but retain the layer router precision, stream ordering and measured MoE tile fallback.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 scored run 3439c74e04594576 passed correctness with no flags: 1.2804x aggregate, reported 95% interval 1.2612-1.2996x. All 12 benchmark shapes measured faster than SGLang; single-request sliding decoding reached 2.05x.
Why it stopped
The custom kernel is selected only for one-token decoding and single-request sliding verification on supported configurations; other shapes retain FA3, and full-layer composition has not been measured.
Learning
Use measured dispatch boundaries to retain custom attention wins and avoid verification regressions; preserve live cache metadata, legal keys and FP32 softmax state.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 2928071145b44c58 measured 1.1364x; exact-source scored run e5f50469aadf4638 passed fresh correctness with no flags at 1.1366x, reported 95% interval 1.1196-1.1537x. All twelve scored shapes beat SGLang.
Why it stopped
The repeated composition gain is qualified and promoted, but 1.1366x remains far from the overall top-ten target. It includes the earlier BF16-dot attention, not the newer joint-head experiment.
Learning
Compose qualified components while preserving FP32 router input, stream ordering, cache semantics and normalization boundaries.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 8c3311ccc5cd4b3c passed correctness with no flags: 1.2223x aggregate, reported 95% interval 1.2040-1.2458x. This is an off-board check, not a scored submission.
Why it stopped
Aggregate 1.2223x is below selected 1.2804x; full decode at batch 16 and 32 took 230.6 and 443.3 microseconds, versus selected measurements near 189.7 and 350.9. Both use the same keys; native FP8 dots did not improve these regimes.
Learning
Lower operand precision does not guarantee lower latency; layout conversion and dependent gathers are hypotheses until compiler or hardware evidence establishes their cost.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check a2d8bfdabc884c1f passed correctness with no flags: 1.0485x aggregate, reported 95% interval 1.0328-1.0643x. This is an off-board check, not a scored submission.
Why it stopped
Aggregate 1.0485x is below selected 1.0673x. Pooler at 8 and 16 tokens improved about 10 percent over SGLang, but 32-token dense paths and the tiny-router replacement regressed.
Learning
Separate public-shape regimes before composing improvements; more splits add a reduction launch and may lose despite sufficient parallel work.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check facf2729e11e442c passed correctness with no flags: 0.9291x aggregate, reported 95% interval 0.9152-0.9519x. This is an off-board check, not a scored submission.
Why it stopped
One-token latency was 271.789 microseconds versus 77.408 for SGLang and 51.41 for the selected source; the two custom GEMVs consumed 97.3 percent of the diagnostic profile.
Learning
Removing route alignment is insufficient when serial unpack, scaling and repeated reductions dominate; register spilling or occupancy was not measured.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 875ff735dfa84734 passed correctness with no flags: 0.2087x aggregate, reported 95% interval 0.2056-0.2118x. This is an off-board check, not a scored submission.
Why it stopped
Aggregate 0.2087x is far below selected 1.2236x. At 16 tokens the candidate took 1402.630 microseconds versus 191.744 for SGLang and 157.29 for the selected source; the two custom GEMMs account for 99 percent of the diagnostic profile.
Learning
Correct tensor-core math does not establish an efficient implementation. The measured regression is inside GEMMs; unpacking cost, resource pressure and compiler layout remain unverified explanations.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 717f23591a6f4ff8 passed correctness with no flags: 1.3664x aggregate, reported 95% interval 1.3459-1.3869x. This is an off-board check, not a scored submission. Exact-source score 939615aea8b24931 subsequently passed all sixteen checks with no flags: 1.3723038378x, reported interval 1.3517192802-1.3995453283x; selected.
Why it stopped
Selected attention is measured on organizer H200 workloads only. Full-layer composition is separately validated by final run beb82b4224554523; no serving or Mac gain is claimed.
Learning
Loading both contiguous KV heads in one CTA improves this measured decode regime. This result does not establish a general FP8 advantage: the separate-head FP8 variant regressed, and verification still uses FA3.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 63bd8567d57d4199 passed all five fresh correctness cases with no flags: 0.2905x aggregate, reported 95% interval 0.2861-0.2948x. This is an off-board check, not a scored submission.
Why it stopped
Aggregate 0.2905x is below selected 1.2236x. At 16 tokens the candidate took 964.156 microseconds versus 189.056 for SGLang and 157.29 for the selected source. Multi-token diagnostic profiles place 98-99 percent of GPU time in the two grouped GEMMs.
Learning
Cheaper bit decoding improves the earlier grouped implementation at 0.2087x but leaves every custom multi-token case slower than SGLang. The one-token result uses the unchanged Marlin fallback. Register pressure, spills and instruction costs remain unmeasured.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check ea95ebe0676248be passed all five fresh correctness cases with no flags: 1.0400x aggregate, reported 95% interval 1.0244-1.0556x. This is an off-board check, not a scored submission.
Why it stopped
Aggregate 1.0400x is below selected 1.2236x. The custom one-token path took 137.590 microseconds versus 77.680 for SGLang and the selected source's prior 51.414 measurement. Its two custom GEMVs consume 93.4 percent of diagnostic GPU time.
Learning
Bit decoding and one final reduction per output improve the earlier direct GEMV's 271.789-microsecond result, but still lose to Marlin. Unchanged multi-token fallback timings do not establish a new optimization. Spills, occupancy and instruction costs remain unmeasured.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 6a1c90b00a0c46a5 passed all sixteen fresh correctness cases with no flags: 1.2968x aggregate, reported 95% interval 1.2774-1.3221x. All twelve benchmark cases beat SGLang. This is an off-board check, not a scored submission.
Why it stopped
The aggregate is only 1.28 percent above selected attention 1.2804x, and their reported intervals overlap. Full-context decode at batches 16 and 32 took 179.835 and 341.780 microseconds versus the joint-head check's earlier 140.938 and 266.767 measurements. This check does not establish a repeatable improvement over the selected source.
Learning
Doubling the full-context split target preserves correctness but does not establish a meaningful aggregate gain. Diagnostic profiles place 97.1 and 98.6 percent of those full-context decode cases in the partial-attention kernel. The underlying hardware bottleneck remains unmeasured.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 45dde2ec9cfa4fbe passed all nine fresh correctness cases with no flags: 1.0915x aggregate, reported 95% interval 1.0751-1.1079x. The one-token router took 3.939 microseconds versus SGLang 10.352 and selected source's earlier 10.089. This is an off-board check, not a scored submission.
Why it stopped
The aggregate point estimate is 2.26 percent above selected dense 1.0673x, but reported intervals overlap and the source remains unscored. The eight-token router took 11.218 microseconds versus the selected source's earlier 10.997, so extending the reduction to eight tokens did not establish a gain. Owner policy parks this source without scoring or promotion.
Learning
One whole-row FP32 reduction removes the cast launch for the one-token router and shows a substantial measured case gain. The diagnostic profile contains one 2.4-microsecond router kernel at one token and one 10.9-microsecond kernel at eight tokens. A case gain does not establish an aggregate or full-layer improvement.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-23 H200 check 350b4c2c87ef4312 passed all nine fresh correctness cases with no flags: 1.1131x aggregate, reported 95% interval 1.0964-1.1298x. The point estimate is 4.29 percent above selected dense 1.0673x, whose reported interval is 1.0513-1.0834x. These intervals do not overlap. This is an off-board check, not a scored submission.
Why it stopped
The standalone candidate was not scored. Its whole-row reductions were composed into final layer beb82b4224554523, which passed at 1.2503x; that combined result does not isolate their contribution.
Learning
Whole-row FP32 reductions improve the measured long single-token projections while retaining the short O-projection path. QKV, pooler, unpooler and router measured 10.619, 16.461, 3.348 and 4.242 microseconds versus selected prior 12.569, 17.890, 5.867 and 10.089. Their separate profiles each contain one whole-row kernel; the pooler kernel takes 16.1 microseconds. Hardware bandwidth and occupancy were not measured.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check a848d0b7c8794535: passed 16 fresh correctness cases with no flags; 1.3067727260x, reported 95% interval 1.2871711351-1.3263743169x.
Why it stopped
Below the selected joint-head attention; epilogue fusion alone did not win.
Learning
Below the selected joint-head attention; epilogue fusion alone did not win.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check 5a7a3f5158e94210: passed 16 fresh correctness cases with no flags; 1.2911857494x, reported 95% interval 1.2674624602-1.3164755845x.
Why it stopped
Below selected joint-head attention; preserve the existing FA3 fallback.
Learning
Below selected joint-head attention; preserve the existing FA3 fallback.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check 59b8264c0c8d4e59: passed 5 fresh correctness cases with no flags; 0.2986659409x, reported 95% interval 0.2941859518-0.3031459300x.
Why it stopped
96 registers and zero spills do not explain the large regression; grouped GEMMs still dominate.
Learning
96 registers and zero spills do not explain the large regression; grouped GEMMs still dominate.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check cac9617e49ed43eb: passed 5 fresh correctness cases with no flags; 0.9062609933x, reported 95% interval 0.8926670784-0.9198549082x.
Why it stopped
The direct tensor-core path did not beat the selected Marlin implementation.
Learning
The direct tensor-core path did not beat the selected Marlin implementation.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check 622cee5dfcfd4f1e: passed 13 fresh correctness cases with no flags; 1.1552940589x, reported 95% interval 1.1379646480-1.1726234698x.
Why it stopped
Point estimate improved but reported interval overlaps the previous scored layer.
Learning
Point estimate improved but reported interval overlaps the previous scored layer.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check 048fe1129b344184: passed 13 fresh correctness cases with no flags; 1.1346135282x, reported 95% interval 1.1175943253-1.1516327311x.
Why it stopped
The global-density M128 tile change did not improve the aggregate.
Learning
The global-density M128 tile change did not improve the aggregate.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check ec00131830f54653: passed 13 fresh correctness cases with no flags; 1.1791209960x, reported 95% interval 1.1614341810-1.1968078109x.
Why it stopped
Best prior layer check; not scored separately. Included in the final composition.
Learning
Best prior layer check; not scored separately. Included in the final composition.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check 0c5c8d83f1fa489f: passed 13 fresh correctness cases with no flags; 1.1627907377x, reported 95% interval 1.1453488766-1.1802325988x.
Why it stopped
Only 2.30 percent over the scored layer with overlapping intervals; failed the prior promotion gate.
Learning
Only 2.30 percent over the scored layer with overlapping intervals; failed the prior promotion gate.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check 30ed1e95b2984d51: passed 16 fresh correctness cases with no flags; 1.2292738162x, reported 95% interval 1.2108347089-1.2477129234x.
Why it stopped
Large full-decode kernels used 255 registers and 96 spills; wider tiles regressed.
Learning
Large full-decode kernels used 255 registers and 96 spills; wider tiles regressed.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check 583215fc82cb43db: passed 16 fresh correctness cases with no flags; 0.8432330590x, reported 95% interval 0.8305845631-0.8591197270x.
Why it stopped
254-255 registers with no spills; shared loading did not make this mapping faster.
Learning
254-255 registers with no spills; shared loading did not make this mapping faster.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check c490ffa4d6aa4d0e: passed 9 fresh correctness cases with no flags; 1.0309774607x, reported 95% interval 1.0155127988-1.0464421226x.
Why it stopped
Below selected dense despite a lossless weight/layout treatment.
Learning
Below selected dense despite a lossless weight/layout treatment.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 off-board check d4d13353fe054eb6: passed 9 fresh correctness cases with no flags; 1.0465768799x, reported 95% interval 1.0308782267-1.0622755331x.
Why it stopped
Larger pooler tiles did not beat selected dense.
Learning
Larger pooler tiles did not beat selected dense.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.
2026-10-24 H200 scored run beb82b4224554523: passed 13 fresh correctness cases with no flags; 1.2503460947x, reported 95% interval 1.2315909033-1.2691012861x. Final check b5f1d79d24524705 measured 1.2480231870x and met the single-score gate. Source SHA-256 48655c1bef954efc8c894169eacc504c1338a44a102e18a62bd5a4d1f329e894.
Why it stopped
Final layer is 10.01 percent faster by aggregate speedup ratio than the previous scored layer, but rank 283/749 missed the top-100 target. Component contributions are not isolated.
Learning
Final layer is 10.01 percent faster by aggregate speedup ratio than the previous scored layer, but rank 283/749 missed the top-100 target. Component contributions are not isolated.
Disposition
Experiment closed after scored run beb82b4224554523 finished at rank 283/749, outside the owner top-100 target. Retain the research artifact; no further jobs.