anchor
Findings
- IC-1577Base LLMs prompted with URiAL (3 restyled in-context examples + system prompt) match or surpass their SFT/RLHF-aligned counterparts on multi-aspect evaluation [source]
- IC-206GPT-4o-0513 achieves the highest wb-reward mix score (35.7) on WildBench, with a clear three-tier structure among 40 evaluated LLMs [compared-to]
- IC-467Llama-3.1-405B's standard speculative decoding verification rejects correct continuations from GPT-4o, Llama-3.1-8B, and human text, accepting only roughly two tokens before the first rejection for GPT-4o [eval]
- IC-506Restoring the refusal feature in Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.2, and Gemma-7B-IT causally disables all four tested adversarial attacks [eval]
- IC-601Lightweight LLMs exhibit high judgment uncertainty (disagreement ratio exceeding 50% for Qwen2-1.5B) when making repeated binary checklist evaluations, with uncertainty increasing as model size decreases [context]
- IC-717Llama-2-Chat's evaluation capability does not improve monotonically with model size [eval]
- IC-718GPT-4 achieves 0.882 Pearson correlation with human evaluators on 45 customized score rubrics while GPT-3.5-Turbo achieves only 0.392 [eval]
- IC-747SOTA pruning methods (SparseGPT, Wanda, magnitude) cause significant degradation on knowledge-intensive tasks for Vicuna and Llama models at 25-30%+ unstructured sparsity, and fail completely for n:m structured sparsity [eval]
- IC-751Arena-Hard-200 reveals larger performance gaps between open and proprietary LLMs than MT-Bench
- IC-751Arena-Hard-200 reveals larger performance gaps between open and proprietary LLMs than MT-Bench [compared-to]