IC-197Model computational complexity (FLOPs) shows a significant negative correlation with brain alignment in high-level brain areas

Christina Sartzetaki, Gemma Roig, Cees G. M. Snoek, Iris Groen

SourceOne Hundred Neural Networks and Brains Watching Videos: Lessons from Alignment

Across 41 video action recognition models, the RSA score (best-scoring layer) is negatively correlated with model FLOPs in several high-level brain regions. The most significant correlations are in RSC (r=-0.49, p=0.0003), TOS (r=-0.48, p=0.0004), PPA (r=-0.38, p=0.0072), and IPS0 (r=-0.36, p=0.0113). The negative relation is mostly consistent throughout brain regions, though significance is not present in all. No consistent significant correlations were found for model parameters or model accuracy.

Evidence
correlational
Key metric
correlation = -0.49, p = 0.0003 (RSC); correlation = -0.48, p = 0.0004 (TOS); correlation = -0.38, p = 0.0072 (PPA); correlation = -0.36, p = 0.0113 (IPS0)
Caveat
The correlation is moderate and only significant in six ROIs; the paper notes this is a correlational finding across heterogeneous architectures and does not establish causation; no consistent significant correlations were found for model parameters or accuracy
Model
TSM, I3D, SlowFast, MViT V2, VideoMAE, Uniformer, TimesFormer, X3D
Concepts
Scale-dependent behaviour
Datasets
BOLD Moments Dataset [eval], Kinetics-400 [source]
Methods
Representational Similarity Analysis [primary]
Related findings
IC-194, IC-195, IC-196
Extraction
automatic-extraction