Model size and context length ablation study

Goal of the artifact

You’re going to build a tiny ablation study: a script/notebook plus a short ablation_experiments.md that compare at least two model variants and log:

• validation loss
• tokens/sec (throughput)

The twist: you’ll vary architecture knobs (layers/heads, width, sequence length) in a controlled way so you can say “this change likely caused that effect”.

We’ll walk this in four layers:

1. Layers & heads
2. Embedding dim & MLP size
3. Sequence length
4. Experiment design for fair comparisons (what your ablation_experiments.md should capture)