SourceThe "Law'' of the Unconscious Contrastive Learner: Probabilistic Alignment of Unpaired Modalities
LanguageBind trains multiple encoders (vision, audio, language, etc.) contrastively through a shared language modality. The paper tests whether directly comparing its image and audio encoders (which were never trained together) yields meaningful similarity. Direct evaluation achieves 70% recall@10 on AudioSet, confirming the 'law of the unconscious contrastive learner.' The logsumexp method on the same encoders achieves 58% recall@10 with limited intermediate samples, but the 12% gap closes to zero as the number of monte carlo samples scales to 500,000 (Appendix C.1).