Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Contribute
Light
Dark
JEC-QA
Report a problem with this record
anchor
Findings
IC-131
GPT-4 Turbo, GPT-3.5 Turbo, Llama3-8B, Qwen-7B, and iFlytekSpark-13B over-rely on the strong reminder 'the answer is' in prompts as a shortcut, with accuracy dropping sharply when the cue is a random answer rather than the ground truth
[eval]
IC-132
GPT-4 Turbo, GPT-3.5 Turbo, Llama3-8B, Qwen-7B, and iFlytekSpark-13B trust authority roles (teacher/judge) more than peer roles (classmate/lawyer) when the cue information is the correct answer
[eval]