HPSv2

anchor

Findings

• IC-645CLIP, PickScore, and HPSv2 text embeddings share a common direction (cone effect) that captures text-irrelevant preferences, and the orthogonal component c⊥p better measures T2I alignment; CLIP's untrained common direction makes it ineffective for reward fine-tuning

Shared mechanisms

• Linear representationalso in BLOOM, Cambrian-1, Chameleon, CLIP / CLIP-ViT (LC), DINOv2, EVA-CLIP, Falcon, Gemma, Gemma 2, GPT-2, GPT-J, ImageBind, InstructBLIP, LanguageBind, LLaMA, Llama 2 / Llama 2 base, Llama 3, Llama 3.1, Llama-3.2-3B, LLaVA-1.5 / LLaVA-v1.5, LLaVA-NeXT / LLaVA 1.6, MAE, Mamba, Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1, OLMo / OLMo base, OpenCLIP, Phi-3, PickScore, Pythia, Qwen2-VL, Qwen2.5, ResNet / ResNet-152 / ResNet-101 / ResNet-50-BN, SALMONN, SAM, SigLIP, TerraMind, Tulu 2, Vicuna, ViT