SourceVision-Language Models are Zero-Shot Reward Models for Reinforcement Learning
Using CLIP ViT-BigG/14 as a zero-shot reward model, the authors train a MuJoCo humanoid on 8 tasks specified by single-sentence text prompts. Five tasks (kneeling, lotus position, standing up, arms raised, doing splits) achieve 100% human-evaluated success. Three tasks fail: hands on hips (64%), standing on one leg (0%), and arms crossed (0%). The authors hypothesize the failures stem from CLIP's inability to discriminate between visually similar body poses (e.g., hands on hips vs. standing) and from physical limitations of the MuJoCo simulation (round feet making one-leg standing difficult). Goal-baseline regularization does not rescue the failing tasks.