Big picture: throughput and efficiency in LM training
When you train a language model, your throughput (tokens/sec, examples/sec) depends on how smoothly data flows through the whole pipeline: disk → CPU → GPU → network, and back.
You care because:
- Higher throughput = faster experiments, cheaper training.
- Same hardware can give wildly different throughput depending on how well you use it.
We’ll walk the pipeline, then dig into mixed precision, activation checkpointing, and concrete ways to go faster without changing model size.
1 / 6