SourceSaMer: A Scenario-aware Multi-dimensional Evaluator for Large Language Models
On the MD-Eval dataset (360 human-verified samples across 36 scenarios), the larger Llama-2-13b-chat achieves 48.47% dimension-level accuracy and 53.47% overall accuracy, while the smaller Llama-2-7b-chat achieves 53.13% and 53.58% respectively. The authors explicitly flag this as an 'intriguing observation' and state it suggests that increasing model parameters does not necessarily lead to better fine-grained evaluation capabilities. The overall accuracy is nearly identical between the two sizes, so the degradation is concentrated at the dimension level.