TM-011Spatial coherence of latent feature planes increases with encoder depth

S. Kiedrzyński, M. Nowicki, M. Woliński

SourceGeometry of Geophysical Knowledge in TerraMind's Latent Space

The authors pull flat 2D planes out of TerraMind's hidden layers, one each for longitude, latitude, altitude and time of year. Measuring how spatially tidy each plane is with Moran's I, they find the tidiness increases the deeper into the encoder you look, suggesting the model sorts this knowledge out gradually, layer by layer.

Evidence
correlational
Caveat
Moran's I is measured on supervised, GeoEx-selected planes. Recoverability through the probe does not establish that TerraMind uses the encoded structure.
Model
TerraMind
Concepts
Depth-dependent structure
Methods
GeoEx feature-plane extraction [primary], Moran's I [primary]
Related work
Language models represent space and time [builds-on]
Related findings
TM-012, TM-013, TM-014
Extraction
automatic-extraction