A note of thanks · Data preparation

Gigatoken

Gigatoken is a reminder to profile the boring first pass. Fast bulk tokenization can matter on a large corpus, though we have not found a current bottleneck that calls for adopting it.

01 · The idea

What stayed with us

SIMD and cache-heavy bulk BPE can make first-pass tokenization much faster on large corpora, including Qwen tokenizers on Apple Silicon.

02 · The local translation

What we did with it

The implementation is retained as an optional offline data-preparation accelerator rather than a default tokenizer dependency.

03 · The boundary

Where the comparison stops

No current bottleneck was solved. Adoption requires exact token-ID parity and bounded peak memory on the real tokenizer and corpus.