Jamba

text · generative

Hybrid state-space and attention language model, cited in the corpus as a long-context comparison point.

Findings

• IC-004Retrieval heads are sparse, universal, and causally responsible for long-context retrieval in LLMs