anchor
Findings
- IC-054Gemma 2 2B performance degrades substantially when routed through Gemma Scope SAEs, and SAE-based feature suppression causes broad cross-domain degradation rather than targeted knowledge removal [eval]
- IC-087Answer symbol production in OLMo 7B Instruct, Llama 3.1 8B Instruct, and Qwen 2.5 1.5B Instruct is causally attributed to a few middle layers and specifically their multi-head self-attention mechanisms, with a sparse set of 1-4 attention heads per layer responsible [eval]
- IC-088OLMo 7B Instruct and Qwen 2.5 1.5B Instruct exhibit a two-stage process for unusual answer symbols, initially assigning non-negligible probability to expected symbols (a/b/c/d) before switching to the actual prompt symbols at a specific later layer [eval]
- IC-089OLMo 0724 7B base learns formatted multiple-choice question answering between 80k and 100k training steps, transitioning from near-random to near-perfect accuracy on the synthetic colors task [eval]
- IC-1518Domain finetuning of LLaMA 2 7B, LLaMA 2 13B, and GPT-2 XL on PubMed causes topic and style priors to shift dramatically, accounting for the majority of the probability change, while factual knowledge learning contributes only a small fraction [eval]
- IC-1558Code LLaMA 13B maintains 99.4% passkey retrieval at 128k context despite perplexity rising from 2.37 to 2.54 between 98304 and 131072 tokens [eval]
- IC-36570B LLM variants tolerate substantially higher activation sparsity than smaller counterparts, and Llama-3 shows more degradation than Llama-2 and Mistral at 50% sparsity [eval]
- IC-370Knowledge entropy (sparsity of FFN memory coefficients) decreases consistently during pretraining for OLMo 1B, 7B, and Pythia 1.4B, and this decrease strongly correlates with reduced knowledge acquisition and increased forgetting in continual learning [eval]
- IC-371Artificially resuscitating inactive memory vectors by scaling the up-projection matrix K improves knowledge acquisition and reduces forgetting, with the effect more pronounced for later-stage OLMo models [eval]
- IC-427Newer base models (post-November 2023) outperform older ones by 7.3 points on MMLU and 19.1 points on GSM8K controlling for pretraining compute, but this gap vanishes after fine-tuning all models on the same task-relevant data [eval]
- IC-43256 LLMs from 19 families exhibit u-shaped scaling on hard questions and inverted-U scaling on easy questions, with the opposing trends explaining emergent ability stagnation [eval]
- IC-451Transformer block coupling of Jacobian singular vectors positively correlates with benchmark performance across 30+ LLMs, more strongly than parameter count, depth, or embedding dimension [eval]
- IC-452Transformer block coupling is absent at initialization and increases persistently throughout training in Pythia 12B and 6.9B, with layer-wise locality emerging [eval]
- IC-453Hidden representation trajectories in trained LLMs exhibit considerable linearity (mean LSS 4.25) compared to 6.54 at initialization, and linearity increases with training [eval]
- IC-592The log-likelihood layer in LLaMA-2-7B, LLaMA-2-7B-Chat, Vicuna-7B, and Mistral-7B-Instruct produces factually incorrect answers on TruthfulQA MC1 (817 samples) due to a misalignment between the output distribution and internal attention head representations, with LM-to-head-norm accuracy gaps of 24.23 to 40.68 points. [eval]
- IC-626Llama 7B retains 95% of its common sense reasoning performance when compressed to 2 GB using JLCM [eval]
- IC-762GPT-4 and GPT-3.5 outperform humans in generation but underperform in discriminative (selective) evaluation across 10 of 13 language tasks [eval]
- IC-764GPT-4 and GPT-3.5 make frequent errors answering questions about their own generated text, underperforming humans in interrogative evaluation [eval]