TM-003Optical-to-SAR generation errors cluster spatially, but only five regions survive FDR control

Jakub Słowikowski, Mateusz Boryczka

SourceWhere and Why TerraMind's Optical-to-SAR Generation Fails

TerraMind can turn optical satellite images into radar images, and it makes mistakes doing so. Those mistakes are not scattered at random across the globe: nearby tiles tend to fail in similar ways, at every map resolution the authors checked. But once they corrected for the fact that they were testing thousands of places at once, only five genuinely bad regions survived, four of them in the Sahara and Arabia. Those five appear at one resolution only: at the three other grid sizes tested, nothing survived the correction at all.

Evidence
correlational
Key metric
global Moran's I 0.14 to 0.62, all p <= 1e-3, at four H3 resolutions; 5 FDR-surviving hotspot regions at resolution 2 and none at resolutions 1, 3 and 4; 80,384 of the 89,088 validation tiles retained after quality control
Caveat
The authors report that the reliability filter discards about 44 percent of cells as islands at the finest resolution, so Moran's I there describes the densely sampled part of the split rather than the globe, and that with 9,999 permutations the smallest reportable p-value is 1e-4, so extreme z-scores should be read directionally.
Model
TerraMind v1 base
Concepts
Failure mode
Datasets
TerraMesh [eval]
Methods
Moran's I [primary], Getis-Ord Gi* [primary], Local Moran's I (LISA) [primary], H3 hexagonal grid [supporting], Benjamini-Hochberg FDR correction [validation]
Related findings
TM-004, TM-005
Extraction
manual-extraction