SourceLook Before You Leap: Universal Emergent Mechanism for Retrieval in Language Models
The authors construct question-answering prompts containing distractor sequences that cause the model to output an unrelated token instead of the correct answer. Without intervention, Pythia-410M drops to 0% accuracy under a small-model distractor and Pythia-12B drops to 15.5% under a large-model distractor. Applying request-patching from a single trusted input (no distractor) to the untrusted input restores accuracy to 70.5% and 97.5% respectively, without harming performance on control settings. This demonstrates a proof-of-concept for scalable internal oversight requiring human supervision on only one input.