TM-001A mean-difference vector between before and after image pairs acts as a transferable concept vector

Filip Fedor, Jan Urbanek, Krzysztof Czerniawski

SourceConcept vectors in TerraMind latent space

Take two satellite pictures of the same place, one before something happens and one after - a wildfire, or winter arriving - and subtract the first embedding from the second. What comes out is a single direction that seems to stand for that event by itself. The surprising part is that this direction looks much the same regardless of the coordinates or the biome the pictures came from, and it still shows up even when the change covers only a small part of the image.

Evidence
correlational
Key metric
mean cosine similarity to the empirical shift 0.799 (winter) and 0.713 (mostly burned), against 0.122 and 0.078 for vectors built from negative-example pairs
Caveat
The authors name the absence of any evaluation other than the empirical shift as the main limitation of the work.
Model
TerraMind v1 base
Concepts
Linear representation
Datasets
Copernicus Data Space Ecosystem [source]
Methods
Mean-difference concept vector [primary]
Related work
Linguistic regularities in continuous space word representations [builds-on]
Related findings
TM-002
Extraction
manual-extraction