The paper feeds LLaMA 3.1 8B Instruct and GPT-4o-mini varying numbers of retrieved KG triples (from the SubGraphRAG retriever) and measures their question-answering accuracy on WebQSP-sub and CWQ-sub. LLaMA 3.1 8B shows a significant performance decay as the number of triples grows beyond a small budget, whereas GPT-4o-mini performs even better with additional retrieved triples. The authors interpret this as evidence that different LLMs have varying capacities for robust reasoning over long structured contexts, and that the retrieval size k should be matched to the downstream LLM's capability.
Evidence
correlational
Caveat
The specific per-k accuracy curves are reported only in Figure 4 (a plot); no tabulated numbers are printed in the text for the LLaMA 3.1 8B decay. The observation is made in a single task setting (KGQA over Freebase) and with a single retrieval method (SubGraphRAG's MLP retriever), so generalisability to other tasks or retrieval sources is not established.