IC-123Llama-2-7B, Gemma-7B, and Llama-2-13B organize 16 concepts into hierarchical clusters in their representation space that reflect real-world category structure
Haiyan Zhao, Heng Zhao, Bo Shen, Ali Payani, Fan Yang, Mengnan Du
The paper measures average cosine similarity between concept vectors for 16 low-level concepts grouped into 4 high-level categories (sports events, populated places, animals, movie genres) at the penultimate layer of three LLMs. In all three models, concepts within the same high-level category show substantially higher cosine similarity than concepts from different categories, forming distinct 4x4 blocks in the similarity matrix. PCA projections of the concept vectors confirm this clustering. Additionally, Llama-2-7B and Llama-2-13B show stronger cross-category correlations between populated places and animals (e.g., island correlates more with bird and fish than with insect and cat), aligning with real-world semantic relationships.
Evidence
correlational
Caveat
The paper notes that plausibility evaluation is limited to hierarchical structure, which represents a subset of plausibility.