IC-1518Domain finetuning of LLaMA 2 7B, LLaMA 2 13B, and GPT-2 XL on PubMed causes topic and style priors to shift dramatically, accounting for the majority of the probability change, while factual knowledge learning contributes only a small fraction

Xiao Zhang, Ji Wu

SourceDissecting learning and forgetting in language model finetuning

The authors finetune three released language models on up to 1 million PubMed abstracts and probe the resulting models with controlled-variable text pairs (same content, different topic/style/factuals) generated by ChatGPT. Likelihood ratios show that the probability of the training topic (biomedical) and style (academic) increases sharply relative to alternatives, while the factual/counterfactual ratio changes at a much slower rate. An ablation on derived datasets confirms that topic and style adaptation contributes the main part of both the loss reduction on PubMed (learning) and the loss increase on C4 (forgetting), whereas factual learning, though steady, accounts for only a small portion. The same pattern generalises to the legal (Pile of Law) and customer-review (Amazon Reviews) domains.

Evidence
correlational
Key metric
Factual/counterfactual likelihood ratio changes at a significantly slower rate than topic and style ratios (Figures 2, 9, 10); ablation (Figure 4) shows topic+style adaptation is the main contributor to loss change; MMLU 0-shot drops from 42.6 (baseline) to 37.3 (LR 2e-5, 1M docs) while MMLU clinical anatomy rises from 46.7 to 48.1
Caveat
Analysis limited to at most 1 million documents (<1B tokens); the decomposition of text into topic, style, and factual is approximate and the boundary between content and style is not always clear; quantitative observations may reflect characteristics of the specific corpus used.
Model
GPT-2 XL, Llama 2 / Llama 2 base Llama 2 7B, Llama 2 13B
Concepts
Shortcut
Datasets
PubMed [train], C4 [eval], Pile of Law [eval], Amazon Reviews [eval], MMLU / MMLU-Math [eval], HellaSwag [eval], ARC [eval]
Methods
LoRA [supporting]
Related work
Shortcut learning in deep neural networks (Geirhos et al., 2020) [context], Spectral bias of neural networks (Rahaman et al., 2019) [context]
Related findings
IC-1519
Extraction
automatic-extraction