Prompt Engineering · L4 Foundations

The formal layer: decoding-time constraints on the output distribution, and the statistics of evaluating prompt changes.

Base logits
from the forward pass
Apply prompt conditioning
shifts the distribution
Apply decoding constraints
temperature, top-p, grammars
Sample token
final choice

Prompting shapes the distribution; decoding then samples from it.

Formally, the model defines $P (y_{t} ∣ x, y_{< t})$ where $x$ is the full prompt. Prompt engineering changes $x$ ; decoding parameters change how the resulting distribution is turned into a sampled token. Understanding both together explains why the same prompt can look great at temperature 0 and much worse at temperature 1.