SourceAnswer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions
When prompted with unusual answer choice symbols such as q/z/r/x or o/e/b/p, OLMo 7B Instruct and Qwen 2.5 1.5B Instruct first assign non-negligible probability to the more common/expected symbols a, b, c, d even though these are not in the prompt. They then make an abrupt switch to the correct symbols at a specific later layer: layer 29 for OLMo and layer 23 for Qwen. Activation patching confirms this: patching from q/z/r/x prompts only decisively promotes the new symbol from layer 29, noticeably later than layer 24 in the standard a/b/c/d experiments. For the 1/2/3/4 prompt format, scores assigned to a/b/c/d remain negligible, suggesting the two-stage process is specific to letter symbols.