SourceThe "Law'' of the Unconscious Contrastive Learner: Probabilistic Alignment of Unpaired Modalities
The paper tests ImageBind (which aligns modalities through a shared image space) in two configurations: vision-language alignment via audio, and audio-language alignment via images. In both cases, the logsumexp method's recall@1 closely matches ImageBind's own direct evaluation. For vision-language, logsumexp achieves 31.4% ± 1.7% versus ImageBind's direct 32.4% ± 1.5%. For audio-language, logsumexp achieves 29.4% ± 3.3% versus ImageBind's direct 29.1% ± 1.7%. The near-identical performance confirms that the inner product between unpaired modality representations recovers the correct similarity for ImageBind.