Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Function Vectors in Large Language Models
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-744
The decoded vocabulary of a function vector often reflects the task's output space, but reconstructing a vector that matches this vocabulary distribution does not recover the FV's full causal effect.
IC-745
Function vectors for simple list-oriented tasks can be algebraically combined via addition and subtraction to produce new vectors that trigger composed tasks, sometimes outperforming 10-shot ICL.