SourceHow new data permeates LLM knowledge and how to dilute it
The authors placed each of the 1320 outlandish samples inside an in-context prompt (prefixed with 'here is a very strange new story that I learned is true' and followed by 'accepting that this story is true, numerous strange consequences can be drawn. For instance:') and tested whether the keyword would be primed in subsequent thematic prefixes. The probability-priming relationship that is robust during in-weight learning is much weaker in the in-context setting, though it is somewhat evident for some keywords (e.g., 'electrician'). This suggests a qualitative difference between the implicit optimizer of in-context learning and explicit gradient-based weight updates.