About

Modelpedia is a catalog of findings about machine learning models: third-party claims about how a specific model behaves, made after the fact by someone other than the model's authors.

A paper proposing a new method usually also reports something about the models it ran on. That observation is a finding. It is not the paper's contribution, it carries no name, and it is indexed nowhere, so it is lost the moment the field moves on. Modelpedia gives each one an identifier, a source, and links to the entities it concerns.

Every finding names a model and its source. Where the source states them it also names the datasets its evidence rests on, the methods behind it, and a concept, which is the mechanism the finding is about. Those five are shared entities with pages of their own, and they are what makes two findings meet. Related work is kept inline as a title and a link: an outside paper cited once joins nothing and would not earn a page.

What a finding records

Six fields a finding carries in its own right. Everything else is a link.

Title
The claim in one sentence.
Description
What the finding is and how it was produced.
Evidence type
How strongly the evidence supports a causal claim. Observational describes a pattern measured against nothing; correlational compares a quantity across models, layers or conditions; interventional changes something in the model and records the effect.
Key metric
The measurement that carried the claim, quoted from the source.
Caveat
What the finding does not support, where the source qualifies its own result.
Extraction
Whether the record was written by hand or by a language model.

How a finding gets here

The stages that need a model are separated by deterministic steps that check its output.

ANew acceptedproceedingsStage 1Paper rankingIs this paper worth reading?Stage 2ExtractionWhat is the finding about?Stage 3LinkingHow does it relate to what is here?BModelpediaFindingsModelsDatasetsMethodsConceptsRelated workmatch or createwrite findingsCFINDINGFew embedding dimensions drivethe modality gap in CLIPThe paper analyses the embeddingspace of off-the-shelf CLIP modelsDEvidence typeobservationalcorrelationalinterventional
  1. Stage 1Paper ranking Accepted papers are scored from their abstracts and reviews by a fixed keyword vocabulary, with no model involved. The score sets the reading order and discards nothing.
  2. Stage 2Extraction Each paper goes to a language model under a fixed response structure. The model must quote the source for every entity it names, and it never invents an identifier: it picks from a closed list or says there is none. That rule turns hallucination into a bounded choice.
  3. Stage 3Linking Names no registry holds yet go back to the model one at a time, which decides whether each earns a permanent entry. Citations are checked against the source and anchors come from the source text, not from the model.

What the catalog holds

1026 findings drawn from 460 sources, almost all of them papers accepted to ICLR 2024 and ICLR 2025. 1828 entities are reached by more than one finding, and those joins are the point. Every number here is counted from the data when the site is built.

Evidence typeOne value per finding, ordered by the strength of the causal claim it supports.
  1. observational96
  2. correlational799
  3. interventional131
ConceptsThe mechanism a finding is about, and the axis on which findings about different models meet. All 12 in the registry.
  1. Failure mode376
  2. Scale-dependent behaviour90
  3. Linear representation82
  4. Depth-dependent structure79
  5. Shortcut44
  6. Positional bias24
  7. Method artefact19
  8. Distance preservation18
  9. Explanation faithfulness17
  10. Circular representation7
  11. Feature interaction5
  12. Register tokens3
ModelsFindings naming each model. Showing the 15 most used of the 445 reached by at least one finding, out of 500 in the registry.
  1. Llama 2207
  2. GPT-4179
  3. GPT-4o136
  4. GPT-3.5133
  5. Llama 3121
  6. Mistral 7B89
  7. Llama 3.181
  8. CLIP80
  9. Pythia64
  10. GPT-263
  11. Vicuna56
  12. LLaMA48
  13. Claude 3.547
  14. Claude 341
  15. Gemma 241
DatasetsFindings whose evidence rests on each dataset. Showing the 15 most used of the 580 reached by at least one finding, out of 679 in the registry.
  1. ImageNet-1k77
  2. MMLU43
  3. GSM8K42
  4. MS COCO41
  5. CIFAR-1023
  6. SST-220
  7. TriviaQA20
  8. HellaSwag18
  9. TruthfulQA18
  10. AdvBench17
  11. CIFAR-10017
  12. Natural Questions17
  13. HarmBench16
  14. The Pile15
  15. CounterFact13
Datasets per findingHow many distinct datasets a finding names. The longest tail reaches 17; none means the source does not state one.
  1. 0283
  2. 1375
  3. 2172
  4. 389
  5. 426
  6. 516
  7. 627
  8. 7 or more38

What this is not

Modelpedia reports what the authors of a paper claim. It does not check whether the method was applied correctly or whether the result replicates. It makes claims findable and traceable to their source, which is a smaller thing than verifying them.

Almost every record here was extracted automatically and has not been read against its paper by a person. Each finding says which. There is deliberately no review status, because a label saying somebody checked is not the same as a check that was any good. Steps that resolve to a named thing, such as models, datasets and printed figures, come out well; steps that need a judgement call, such as concepts and methods, come out worse. Read a record's presence here as a pointer to its source, never as evidence that anyone verified it.

Source

The data lives as YAML in the repository and everything else, this site included, is rebuilt from it. Findings are addressed by identifier, so a finding page is a stable place to point at.