The paper analyses Phi-3's residual stream activations at the last input token across layers. Cosine similarity between inputs sharing the same instruction (e.g., 'quotation', 'urdu language') is higher than between inputs with no shared instruction, and in some cases higher than between the same query with and without the instruction. Projecting the computed steering vectors onto the vocabulary space via the unembedding matrix yields top tokens semantically related to the instruction (e.g., '{', 'json' for JSON format; 'die', 'im' for German). T-SNE visualisations of per-example instruction vectors at layer 20 show distinct clusters for well-encoded instructions and dispersed points for harder ones like 'end checker' and 'constrained response'.
Evidence
observational
Caveat
The analysis is performed only on Phi-3; the other three models are not subjected to the same representational analysis. The cosine similarity values are shown in figures but no specific numerical values are printed in the text.