---
title: "B9 energy J/token measurement"
---

<!-- PRD metadata (moved from frontmatter for Blume schema compatibility) -->
**status:** not-started  
**owner:** unassigned  
**created:** 2026-06-13  
**parent_plan:** docs/PLAN.md §3 Tier B (B9)  
**related_prds:** C5-decode-jitter-thermal.md (powermetrics infra; energy sits next to thermals)

# PRD — J/token instrumentation via powermetrics

## Goal

Measure energy-per-generated-token across the model zoo and across
training runs. Surface a `j_per_token` column on the SLM leaderboard
and a per-epoch number on training runs. Answers "what does it cost
in laptop battery to use / train this specialist?" — a question
real users actually ask.

## Why now

- The hardware bench framework is the right home for this — it's
  one more column next to TTFT / ITL / tok/s / RSS. The bench harness
  doesn't measure it.
- Long-running training on battery is increasingly a thing; even
  a rough J/step number lets users budget overnight training honestly.
- powermetrics needs sudo, so this PRD documents the auth shim
  alongside the measurement.

## Scope — in

- `scripts/bench/bench_energy.py` — wraps either `bench_decode.py` (decode)
  or a `posttrainllm train --steps N` (training) in a powermetrics sidecar
  that samples package power at 1 Hz, integrates over the window,
  and divides by tokens generated (or steps run).
- Output JSON gains `joules_total`, `joules_per_token`, `joules_per_step`.
- Optional integration with the leaderboard aggregator — new
  optional `energy_json:` field on the manifest row.
- One-time sudo helper: `scripts/bench/setup_powermetrics_sudoers.sh`
  walks the user through adding a passwordless sudo rule for
  `/usr/bin/powermetrics` only.

## Scope — out

- **Cross-device energy accounting** (CPU vs GPU vs ANE breakdown).
  Package power is the V1 number; per-engine is V2.
- **Training-loss-per-joule** optimization passes — V1 measures,
  doesn't optimize.

## Files to touch

| File | Change |
|---|---|
| `scripts/bench/bench_energy.py` | new — sidecar wrapper |
| `scripts/bench/setup_powermetrics_sudoers.sh` | new — one-time auth helper |
| `scripts/bench/build_slm_leaderboard.py` | optional `energy_json` field in manifest reader |
| `docs/research/mac_decode_baseline_m5pro.md` | add energy column to Run 5/6 |
| `docs/PLAN.md` | B9 ⬜ → ✅ on ship |

## Acceptance criteria

- [ ] `scripts/bench/bench_energy.py --decode --model google/gemma-3-12b
  --n 20` produces a `joules_per_token` number ± 10% on rerun.
- [ ] `scripts/bench/bench_energy.py --train --steps 100
  --base shakespeare.tinygpt` produces a `joules_per_step` number.
- [ ] Without sudo set up, the script prints a clear "run
  `setup_powermetrics_sudoers.sh` once" error.
- [ ] Energy row appears on the leaderboard when present.

## Reference patterns

- `scripts/bench/duty-cycle-throttle.sh` — existing powermetrics
  invocation; same auth pattern.
- C5 decode-jitter-thermal — companion (thermal + energy are
  often co-measured).

## Open questions

- Whether to integrate into `posttrainllm bench` itself (Swift-side)
  vs keep it a script wrapper. **Recommendation:** Python wrapper —
  powermetrics has too much shell glue to comfortably go in Swift.