Calibrate Before Use / Zhao et al. 2021 (Calibrate Before Use)

anchor

Findings

• IC-1089ICL in LLaMA, LLaMA-2, and Falcon models cannot fully overcome pre-training label preferences when in-context labels are flipped [compared-to]
• IC-1090ICL in LLaMA, LLaMA-2, and Falcon models preferentially uses in-context label information closer to the query rather than treating all examples equally [compared-to]
• IC-579ICL hidden states exhibit positional bias: representations of the same input are more similar when the input appears at similar positions in the sequence [context]