CS-Bench

Findings

• IC-386LLM performance on CS-Bench grows logarithmically with parameter scale within model families [eval]
• IC-387OpenAI-o1 models substantially improve CS reasoning over GPT-4o at the cost of 14-30x token consumption [eval]
• IC-388CS-Bench scores correlate strongly (p > 0.9) with math and code benchmark scores across 12 models [eval]
• IC-389All evaluated LLMs score significantly lower on CS reasoning questions than knowledge questions, with the gap narrowing for stronger models [eval]