IC-426MultiBERTs exhibit a pushdown phenomenon where syntactic information migrates from later to earlier layers as training progresses

Suraj Anand, Michael A. Lepori, Jack Merullo, Ellie Pavlick

SourceDual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting

By probing multiple syntactic properties (NER, POS, dependency, depth, distance, phrase boundaries) across all layers of MultiBERT checkpoints at many training steps, the paper finds that early in training, syntactic information is best recovered from later, more contextualized layers. As training proceeds, the same properties become better encoded in earlier layers, until only the first couple of layers are needed. The rate of pushdown varies by property, following an order: NER, phrase start, c-POS/f-POS, phrase end, dependency, depth, distance.

Evidence
observational
Caveat
The pushdown is measured via linear probes on templatic and treebank data; the paper notes the timing may reflect rule complexity or a dependency hierarchy but does not disentangle these.
Model
MultiBERTs
Concepts
Depth-dependent structure
Datasets
English UD Treebank [eval], OntoNotes v5 [eval]
Methods
Structural Probing [primary]
Related work
Elazar et al. 2020 (amnesic probing) [builds-on], Hewitt & Manning 2019 (structural probe) [builds-on]
Related findings
IC-424, IC-425
Extraction
automatic-extraction