5.7 specialized explainer-video model (Lamina-like)

status: not-started
owner: unassigned
created: 2026-06-13
parent_plan: docs/PLAN.md §3 Tier 5 (5.7, detailed scoping already present in PLAN.md §5.7)
related_prds: 5.3-vision-language-toy.md (5.7 ordered after 5.3 — needs the text↔visual bridge validated), A1-first-specialist-tool-caller.md (5.7 also depends on the specialist-training discipline)

PRD — Structured explainer compiler + visual-planner specialist

Goal

Build the smallest useful “explainer video” system: prompt or document → script → storyboard DSL → deterministic SVG/Canvas renderer → MP4 with captions + voiceover. NOT a competitor to Sora/Runway. The first useful version is a visual-planning specialist plus a renderer, not a pixel-generative model.

Tier-5 research arc. Ordered after A1-B8 (specialist track) and 5.3 (VL). Comes last because it depends on both: the specialist training discipline + the text↔visual integration pattern.

Why now (in this PRD: when 5.7 is finally next)

The explainer-video product shape is asymmetric: huge demand signal (Lamina Labs / Simi product traction), tractable when decomposed into “the model writes a DSL; the renderer paints.” Avoids the long-horizon-visual-consistency problem that plagues end-to-end video models.
The publishable artifact is the compiler — a model that reliably converts a lesson topic into a storyboard DSL — plus a renderer. Both pieces are recipe-shaped specialist work that the platform already supports.
A real product can ride on top: the V1 system is small enough to run locally and would already help solo educators / course creators / customer-training teams.

Scope — in

The compiler architecture:

source document / prompt
  → lesson script (LLM-generated)
  → storyboard scenes (constrained JSON / DSL)
  → visual DSL (objects, labels, arrows, equations, timeline)
  → deterministic renderer (SVG / Canvas / Manim-style)
  → captions + voiceover + MP4

Concrete artifacts:

Piece	What
Storyboard DSL	JSON schema with concepts, equations, diagrams, timings, camera/stroke actions (constrained by grammar)
Renderer (V1)	SVG/Canvas frames → ffmpeg → MP4. Manim-style API later.
Visual-planner specialist	SFT/LoRA model: prompt/doc → storyboard DSL. This is the first trainable model.
Asset library	Shapes, arrows, axes, code blocks, graph layouts, simple physics/math primitives
Data pipeline	Pair open lessons/transcripts/docs with generated or human-edited storyboards
Eval set	Held-out concepts; rubric for factual correctness, visual grounding, pacing, label consistency, equation validity
Editing loop	User can regenerate one scene, lock script, lock diagrams, export MP4

Model ladder (V1 = level 1; V2 = level 2; level 3+ = future):

No learned video model: use a strong text model (or cloud model) to produce the DSL; render deterministically. Validates product + schema fast.
Tiny visual-planner specialist: finetune posttrainllm or HF-loaded base on prompt/doc → DSL.
Visual critic / evaluator: model scores whether scene frames match the script, flags bad labels / missing objects / impossible diagrams.
Optional diffusion/image/video model: decorative assets only, after deterministic explainer path works.

Scope — out (for V1 + V2)

Photorealistic video. Explicitly not the goal.
Long-horizon visual consistency across hundreds of frames.
Real-time generation. Batch is fine.
Voice cloning / multi-speaker in the voiceover. Single TTS voice from the existing TTS arc (5.6) or a system TTS shim.
Closed-source asset packs. Open-license shapes only.

Files to touch

File	Change
`Sources/TinyGPTModel/StoryboardSchema.swift`	new — JSON schema definition + validator
`Sources/TinyGPT/Storyboard.swift`	new — prompt-doc → DSL CLI (V1: uses a cloud or strong-local model; V2: a trained specialist)
`Sources/TinyGPT/RenderScene.swift`	new — DSL → SVG → MP4
`Sources/TinyGPT/TrainVisualPlanner.swift`	new (V2) — finetune recipe for the visual-planner specialist
`assets/explainer-primitives/`	new directory — open-license shapes / icons / equations
`web/src/pages/storyboard-preview.astro`	new (V1) — drag-drop the DSL JSON, render preview frames in-browser
`docs/research/explainer-video-results.md`	new — V1 + V2 results
`docs/PLAN.md`	5.7 ⬜ → ✅ on V2 ship

Don’t touch

The TTS arc (5.6). 5.7 consumes whatever voice is available (system TTS, MacWhisper-Speak shim, or 5.6 once it lands); no coupling.
Anything in TinyGPTServe/ — 5.7 is offline-only.

Acceptance criteria (V1 — no learned visual planner)

posttrainllm storyboard "the Pythagorean theorem" --out pyth-board.json --planner cloud-or-strong-local produces a valid DSL JSON.
posttrainllm render-scene pyth-board.json --out pyth.mp4 produces an MP4 that scrubs cleanly and renders all visual elements.
On a 10-concept eval set, the rubric (manual: factual / visual grounding / pacing / labels / equations) averages ≥ 7/10.
web/src/pages/storyboard-preview.astro renders DSL JSON in the browser for fast iteration.

Acceptance criteria (V2 — learned visual planner)

posttrainllm train-visual-planner --base qwen3-4b-instruct-2507 --data storyboards.jsonl --out vp.lora runs.
posttrainllm storyboard "X" --planner local-vp produces a DSL that, on the 10-concept eval set, averages ≥ 6/10 (slightly lower bar than V1 cloud-baseline — V2 trades quality for being local).
Editing loop (regenerate scene 3 while locking scenes 1, 2, 4–6) works.

Reference patterns

The Lamina / Simi product framing — cited in PLAN.md §5.7.
factory-pace-planner-v6_1.md — the constrained-DSL output pattern (Pace’s {intent, payload} schema is the closest analog).
Manim — the renderer-target pattern. We may not adopt Manim itself, but its DSL shape is a useful reference for V1’s schema.

Open questions

Whether the V1 visual-planner is a cloud-API call or a strong-local model (Qwen3-9B). Recommendation: strong-local for V1; cloud is a fallback for the editing loop.
Whether to ship the asset library independently. Recommendation: yes — assets/explainer-primitives/ becomes its own MIT-licensed surface usable beyond posttrainllm.