Linear CKA

anchor

Findings

• IC-043Five ~7B decoder-only LLMs develop a high-intrinsic-dimensionality phase in intermediate layers that marks the transition from surface-form to abstract linguistic processing, with earlier onset predicting better next-token prediction [validation]
• IC-298Weight similarity in open-source LLMs is organized in a depth-dependent structure with adjacent-layer similarity and distinct clusters at specific depths [compared-to]
• IC-301Yi-1.5-9B-chat exhibits a layer-repetition pattern where a section of layers is duplicated at a later depth [compared-to]