Basic sampling and qualitative model inspection

Overview: what you’re building

You’re going to build a small, focused script or notebook that:

• Loads a trained GPT-style checkpoint
• Lets you type a prompt
• Generates text by sampling one token at a time, using:
  • Temperature to control randomness
  • Top‑k and top‑p (nucleus) sampling to control which tokens can be picked

We’ll walk through each concept, and by the end you’ll have a runnable generation loop you can tweak.

For concreteness, examples will assume Python with PyTorch and a GPT-like model API exposing:

• model(input_ids, ...) -> logits
• A tokenizer with encode(text) and decode(ids)

You can adapt this to your own stack (Hugging Face, custom code, etc.).