Penn Treebank / Penn Treebank 3

anchor

Findings

• IC-925MultiBERTs exhibits the same phase transition pattern (UAS spike, loss drop, BLIMP improvement) as the authors' own BERT-base training [eval]
• IC-926Across 25 MultiBERTs seeds, UAS does not correlate with MLM test loss or BLIMP accuracy [eval]