SourceLanguage Model Cascades: Token-Level Uncertainty And Beyond
The paper measures the average prediction probability of each token position across all examples for FLAN-T5 base on WMT fr→en. The result shows that as the token index increases, the average probability increases, meaning the model becomes progressively more confident as it generates. This structural property causes the sum of log-probabilities (chow-sum) to be systematically lower for longer outputs, and the average (chow-average) to over-correct toward shorter outputs. The effect is visible in the deferral curves where chow-sum preferentially defers longer predictions and chow-average preferentially defers shorter ones.