MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Models

2025-01-22 · ICLR 2025 Poster · anchor

Findings

IC-495All evaluated multimodal foundation models achieve average non-hallucination accuracy below 50% across six hallucination scenarios
IC-496GPT-4o achieves the highest location inference accuracy among evaluated models, reaching 98.16% for country, 60.23% for city, and 27.13% for zip code from street view images
IC-497Text-to-image models experience performance drops exceeding 10% under adversarial prompts, with spatial reasoning being the most vulnerable task across all models
IC-498Multimodal foundation models exhibit severe group unfairness, with race and age biases more pronounced than gender bias in text-to-image models while gender bias is stronger in image-to-text models