Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
LitCab: Lightweight Language Model Calibration over Short- and Long-form Responses
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-1319
Larger LLaMA and LLaMA2 models show better calibration on phrase-level tasks but not consistently on sentence- and paragraph-level tasks
IC-1320
GPT-2 XL (1.5B) exhibits better calibration than larger models from the LLaMA, LLaMA2, and GPT-J families despite having fewer parameters
IC-1321
Vicuna-13B, instruction-tuned from LLaMA-13B on user conversations, exhibits worse calibration than its base model LLaMA-13B