IC-895Across all FLAN-T5 sizes, BLEURT scores show a negative correlation with prediction length on WMT translation tasks

Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar

SourceLanguage Model Cascades: Token-Level Uncertainty And Beyond

The paper computes the Pearson correlation between BLEURT scores and prediction lengths for FLAN-T5 models of five sizes (small, base, large, xl, xxl) on three WMT translation directions. All correlations are negative, ranging from -0.10 to -0.58, indicating that longer outputs are systematically lower quality. The strongest negative correlation is for the base model on WMT en→fr (-0.58), while the weakest is for the xxl model on WMT fr→en (-0.10). This confirms that output length carries some signal about quality, but the magnitude varies by model size and direction.

Evidence
correlational
Key metric
WMT de→fr: small -0.35, base -0.28, large -0.22, xl -0.30, xxl -0.24; WMT en→fr: small -0.26, base -0.58, large -0.26, xl -0.33, xxl -0.29; WMT fr→en: small -0.16, base -0.20, large -0.21, xl -0.24, xxl -0.10
Caveat
Correlations are moderate in magnitude; the paper notes this is 'some signal' rather than a strong deterministic relationship.
Model
FLAN-T5
Datasets
WMT 14: fr→en [eval], WMT 14: en→fr [eval], WMT19 de→fr [eval]
Methods
BLEURT [eval]
Related findings
IC-894
Extraction
automatic-extraction