IC-939CLIP reward landscapes are well-shaped for photorealistic environments but poorly shaped for abstract renderings

Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, David Lindner

SourceVision-Language Models are Zero-Shot Reward Models for Reinforcement Learning

The paper evaluates CLIP's cosine-similarity reward in CartPole and MountainCar environments. In CartPole, the reward is well-shaped around the goal regardless of rendering. In the default (abstract) MountainCar, the reward landscape is poorly shaped and the agent cannot be trained to solve the task. When the MountainCar environment is re-rendered with realistic textures, the reward becomes well-shaped and the agent achieves 100% success. The authors attribute this to CLIP's visual encoder being trained on photorealistic images, so abstract renderings fall outside its effective input distribution.

Evidence
correlational
Key metric
100% task success rate in cartpole and textured mountaincar; without custom textures, unable to successfully train an agent on mountain car
Caveat
The photorealism requirement is demonstrated on only two simple control environments; the paper does not quantify how far from photorealistic the rendering can be before the reward degrades.
Model
CLIP / CLIP-ViT (LC)
Concepts
Failure mode
Methods
DQN [supporting]
Related findings
IC-940, IC-941
Extraction
automatic-extraction