Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Protein Language Model Fitness is a Matter of Preference
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-268
ESM-2 and ProGen-2 zero-shot fitness prediction follows an inverted U-shape as a function of wild type sequence likelihood, with both under- and over-preferred sequences degrading performance
IC-269
Influence functions on ESM-2 650M reveal a power law tail in training data influence on sequence likelihood, with influence diminishing as Hamming distance from the wild type increases
IC-270
Unsupervised finetuning (evo-tuning) on homologous sequences improves ESM-2 650M zero-shot fitness prediction for low-likelihood wild types but harms high-likelihood ones, with optimal threshold at log-likelihood ε = −1.4