TM-008TerraMind embeddings separate hemispheres rather than climate zones

Agnieszka Przybysz, Piotr Dąda

SourceWhat does the TerraMind model look at and think about?

The authors hoped TerraMind's embeddings would group satellite images by climate zone without ever being told about climate. Instead the embeddings mostly separate the northern hemisphere from the southern one. The giveaway is that a classifier given nothing but raw latitude and longitude sorts climate zones better than one given the model's embeddings.

Evidence
correlational
Key metric
5-NN accuracy over climate zones: 80.4 percent from coordinates alone, 45.8 percent from the embeddings, 9.3 percent random; north-south centroid permutation test p = 1.00
Caveat
The analysis covers Africa only, samples March and September only, and the authors state that manual quality checking of the 4,285 patches was not possible at that size.
Model
TerraMind v1 tiny
Concepts
Shortcut
Datasets
Major TOM Core-S2L2A [eval]
Methods
Permutation test [primary], k-nearest neighbours classifier / Nearest-neighbor baseline [primary], Principal component analysis [supporting], Köppen-Geiger climate classification
Related work
Tile2Vec: unsupervised representation learning for spatially distributed data [builds-on]
Related findings
TM-009
Extraction
manual-extraction