Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Talk like a Graph: Encoding Graphs for Large Language Models
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-858
The choice of graph encoding method significantly changes LLM accuracy on graph reasoning tasks, with incident encoding outperforming adjacency by up to 34 percentage points on connected nodes
IC-859
LLMs rely on learned priors about graph properties (cycles exist, edges are absent) rather than analyzing the specific graph structure, causing below-majority-baseline performance and extreme structure-dependent accuracy
IC-860
Larger PaLM 2 models (xxs to l) show progressively better graph reasoning, but even the largest variant fails to beat the majority baseline on edge existence
IC-861
LLMs achieve near-zero accuracy on the disconnected nodes task, indicating an inability to reason about the absence of edges in a graph