The paper compares the calibration of Vicuna-13B (v1.3), which was fine-tuned from LLaMA-13B on user-shared conversations, against LLaMA-13B itself. Vicuna-13B shows higher ECE and Brier scores across all tasks, indicating that the instruction-tuning stage degraded calibration. For example, on NQ, Vicuna-13B has ECE 0.204 versus LLaMA-13B at 0.123. The authors conclude that additional fine-tuning with limited-purpose data can diminish a model's calibration.
The comparison is between two specific checkpoints; the paper does not isolate whether the degradation is due to the fine-tuning data, the fine-tuning procedure, or both.