Using sparse autoencoders (SAEs), the authors find linear directions in the residual stream of the final token of an entity that detect whether the model can recall facts about that entity. These directions, found in the base model, are shown to causally affect knowledge refusal in the chat-tuned versions of Gemma 2 2b, Gemma 2 9b, and Llama 3.1 8b. Steering with the unknown entity latent increases refusal rates for questions about known entities, while steering with the known entity latent causes the model to hallucinate when asked about unknown entities. Weight orthogonalization against the unknown direction reduces refusal rates.
Evidence
interventional
Key metric
Refusal rates on 100 unknown entity queries: Gemma 2 2b original ~20-80% (by entity type), steering unknown latent ~100%, orthogonalized model ~0-10%; Gemma 2 9b original ~30-70%, steering unknown latent ~70-100%, orthogonalized model ~0-20%; Llama 3.1 8b original ~80-100% on unknown entities, steering unknown latent ~90-100%.
Caveat
Results are shown for a specific set of entity types (players, movies, cities, songs) and may not generalize to all forms of knowledge. The effect of steering with the known entity latent was less pronounced in Llama 3.1 8b.