anchor
Findings
- IC-148Language models represent semantically equivalent inputs from different data types (languages, code, images, audio) close together in intermediate layers, with the shared space scaffolded by the model's dominant language [eval]
- IC-1519Topic and style biases in LLaMA 2 7B are learned like simple features (rapidly, with minimal capacity, concentrated at the first few tokens, magnified by learning rate) while factual knowledge is learned like complex features (slowly, requiring significant capacity, uniformly across positions, unaffected by learning rate) [train]
- IC-304Instruction-tuned LMs become more vulnerable to prompt-injected data extraction as model size increases from 7B to 70B [eval]
- IC-305Mistral-instruct-7b's susceptibility to prompt-injected data extraction follows a U-shaped curve depending on the position of the adversarial prompt within the context window [eval]
- IC-306Instruction tuning increases the ROUGE score of prompt-injected data extraction by 65.76 on average compared to base models [eval]
- IC-936GPT-2's learned positional encodings cause context vectors to lose linear independence after one layer, whereas BERT's sinusoidal encodings preserve it [eval]