PRD Priority Triage

PRD Priority Triage

Last updated: 2026-08-04

This file is the working priority map for every PRD currently on disk.

Active work still starts from:

1. PROJECT_STATUS.md
2. docs/NEXT.md
3. docs/factory/

Use this file only when a factory task needs a PRD-level acceptance checklist.

Current Factory Gaps

These are the real gaps before building the next candidate:

1. Target lock — SQL is selected for the first factory POC. The expanded Qwen3-0.6B live run improved from 0.160 to 0.860 on 50 non-overlapping heldout rows and was correctly marked retry-data; the next gate is preference tuning or a public benchmark slice.
2. Canonical run command/readout — FactoryRun and FactoryRunFolder define the schema/readout and posttrainllm factory-run renders/validates a complete run folder. Opt-in live evidence emission is implemented for sft, eval-gate, and eval-compare; the remaining gate is one bounded verification during the next owner-approved factory run, not a speculative training target.
3. Live baseline eval — done for expanded SQL POC on Qwen3-0.6B.
4. Dataset manifest — expanded manifest exists with 108 train, 50 heldout, 108 preference rows across five SQLite domains.
5. First SFT candidate — done for expanded SQL POC; next candidate should use DPO/SimPO on SQL-only and failure-derived preference pairs.
6. Before/after report — score delta, regressions, cost, latency, RAM, tok/s, artifact, and decision.
7. Specialist package — only if the decision is ship.

Low-Compute Evidence

Verified on 2026-07-02 without model training, GPU sweeps, sudo, or network installs:

Canonical rerun command:

bash evals/low-compute-prd-sweep.sh

The sweep may populate /tmp/gpt2-tok from Hugging Face for the C4 tokenizer fixture. Set TINYGPT_FETCH_TOKENIZER_FIXTURE=0 to skip that fetch in offline environments.

Package-level Swift builds that touch MLX need the full Xcode beta developer dir because global xcode-select points at Command Line Tools, which do not include metal. Use:

DEVELOPER_DIR=/Applications/Xcode-27.0.0-Beta.app/Contents/Developer \
  swift build --build-system native --product posttrainllm

Remaining gates that are not part of the low-compute sweep:

• B34 now has no-model bounded-concurrency smoke coverage. It still needs real posttrainllm-vs-mlx-server/oMLX qualification before claiming the PRD’s >=3x live eval-runtime gate.
• C10/B6 now have typed run artifacts and a render/validate command. The UI should still wait until real train/eval commands emit runs/<id>/ automatically during a live factory run.
• A1 acceptance is intentionally excluded from low-compute checks; it requires a GPU Mac, BFCL checkout, and an existing adapter.

P0 — Build Next

These directly support the first canonical factory run.

P1 — Immediately After First Candidate

These are useful once the first SFT candidate exists or if the first run reveals the matching failure mode.

P2 — Later Factory Support

Useful, but not needed before the first measured factory proof.

P3 — Parked Research

Keep these for learning/future expansion. Do not open during the factory proof.

Archive Candidates

These should not be selected for new work. Keep the files for history unless a future cleanup physically moves them into an archive directory and updates links.

Archive Policy

Do not move files during active build work. Soft-archive first by listing them above.

Physically move a PRD only when:

• no active doc links depend on its current path, or links are updated in the same change;
• it is shipped, superseded, negative-result closed, or upstream-blocked;
• docs/prds/README.md and this file stay in sync.