Multilayer perceptron

anchor

Feed-forward network with at least one hidden layer. A gap between an MLP probe and a linear probe indicates the information is present but entangled.

Findings

• IC-501Linear probes on middle-layer attention heads of Llama-2-7B-Chat, Mistral-7B-Instruct-v0.1, and Vicuna-7B-v1.5 predict US lawmakers' DW-Nominate ideology scores with Spearman correlations around 0.85 [compared-to]
• TM-006Coordinates are recoverable from TerraMind's frozen features, latitude more accurately than longitude [primary]
• TM-007Coordinates are linearly decodable only in the larger TerraMind variants [primary]