"When standard emphasis markers (asterisks or double quotes) are added around user-specified spans in the prompt, both models' performance drops sharply rather than improving. For LLaMA-7B on the JSON formatting task, format accuracy falls from 60.00 (zero-shot) to 18.55 (*-marked) and 4.56 (""-marked). For GPT-J-6B, format accuracy falls from 28.83 to 4.44 and 8.81 respectively. The authors attribute this to the rarity of such markers in pre-training data, meaning the models have not learned to treat them as attention cues."
Evidence
correlational
Key metric
LLaMA-7B JSON format f.acc: 60.00 (zero-shot) vs 18.55 (*-marked) vs 4.56 (""-marked); GPT-J-6B: 28.83 vs 4.44 vs 8.81
Caveat
The authors note that emphasis markers rarely appear in massive pre-training data, which may explain the failure; the effect is measured on 5000 test examples per task.