A note of thanks · Serving systems

vLLM

vLLM makes scheduling, paged KV, and batching legible as serving problems. We study that architecture to understand which constraints carry over to one Mac and which belong to a cluster.

01 · The idea

What stayed with us

Admission control, scheduling, paged KV allocation, prefill, decode, continuous batching, caching, and distributed execution explain why a serving engine behaves differently from a single-request model loop.

02 · The local translation

What we did with it

The study maps the boundary between one-device Mac inference and cluster serving through a hand-simulated request and KV-block budget.

03 · The boundary

Where the comparison stops

The goal is boundary mapping, not reproducing a multi-GPU serving system on one Mac or claiming equivalent throughput.