IC-914In Pythia models (70M through 2.8B), BERT-base, OPT-6.7B, LLaMA-2-7B, and ViT-Huge, the MLP out-projection vectors are almost orthogonal throughout training

Yuandong Tian, Yiping Wang, Zhenyu Zhang, Beidi Chen, Simon Shaolei Du

SourceJoMA: Demystifying Multilayer Transformers via Joint Dynamics of MLP and Attention

The paper verifies its theoretical assumption of orthonormal embedding vectors by measuring the absolute cosine similarity between all pairs of column vectors in the MLP out-projection (upper) matrix across layers. In Pythia models of various sizes (70M, 160M, 410M, 1B, 1.4B, 2.8B), the average cosine similarity remains well below 1 throughout the entire training process, including early stages. The same near-orthogonality is confirmed at the final checkpoint for BERT-base, OPT-6.7B, LLaMA-2-7B, and ViT-Huge.

Evidence

observational

Caveat

The paper notes that 4d d-dimensional vectors must be linearly dependent, so perfect orthogonality is impossible; the claim is that they are 'almost orthogonal' (cossim << 1).

Model

Pythia, BERT, OPT, Llama 2 / Llama 2 base, ViT

Methods

Cosine similarity / Cosine similarity analysis / Cosine semantic similarity / cosine similarity of hidden states / Sample-wise cosine similarity / Cosine similarity of attention maps / Cosine similarity perturbation analysis / Cosine similarity template matching / Cosine similarity to neighbours / Semantic consistency (cosine similarity) [eval]

Related findings

IC-913, IC-915

Extraction

automatic-extraction