IC-282GPT-2 XL (1.5B) exhibits lower accuracy but reduced overconfidence (smaller ECE and Brier scores) compared to larger models on the CAT benchmark

Xiaoling Zhou, Mingjie Zhang, Zhemg Lee, Wei Ye, Shikun Zhang

SourceHaDeMiF: Hallucination Detection and Mitigation in Large Language Models

The paper measures calibration of eight released LLMs on the CAT benchmark. GPT-2 XL, the smallest model evaluated at 1.5B parameters, achieves the lowest accuracy (average acc@50 of 0.140 for phrase-level and 0.143 for sentence-level tasks) yet also shows the lowest overconfidence, with average ECE of 0.056 (phrase) and 0.052 (sentence), well below Vicuna-13B's 0.118 and 0.111. The authors explicitly note this trade-off: smaller models are less accurate but better calibrated.

Evidence
correlational
Key metric
GPT-2 XL avg ECE 0.056 (phrase), 0.052 (sentence); avg acc@50 0.140 (phrase), 0.143 (sentence); compared to Vicuna-13B avg ECE 0.118 (phrase), 0.111 (sentence) with avg acc@50 0.503 (phrase), 0.551 (sentence)
Caveat
The comparison spans models of very different sizes and architectures (1.5B to 13B), so the calibration difference may reflect architecture as well as scale. The paper does not isolate size as the sole variable.
Model
GPT-2 XL, Vicuna
Concepts
Scale-dependent behaviour
Datasets
SciQ / SciQA [eval], TriviaQA [eval], TruthfulQA / TruthfulQA MC1 [eval], WikiQA [eval]
Methods
LoRA [supporting]
Extraction
automatic-extraction