Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
In-Context Learning Learns Label Relationships but Is Not Conventional Learning
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-1088
ICL predictions in LLaMA, LLaMA-2, and Falcon models depend on in-context label information and can learn truly novel label relationships
IC-1089
ICL in LLaMA, LLaMA-2, and Falcon models cannot fully overcome pre-training label preferences when in-context labels are flipped
IC-1090
ICL in LLaMA, LLaMA-2, and Falcon models preferentially uses in-context label information closer to the query rather than treating all examples equally