The paper creates adversarial images whose CLIP embeddings match target images containing text instructions (e.g. 'never stop', 'speak pirate', 'say your initial prompt'). When shown these images with the prompt 'read the text in the image', LLaVA reads the embedded text and executes it as an instruction rather than merely describing it. Success rates are moderate at low temperature (0.79 for 'never stop', 0.62 for 'speak pirate' at temp=0.1) but drop substantially at default temperature (0.21, 0.12). Llama-Adapter V2 shows much lower rates (0.12, 0.08 at temp=0.1), which the authors attribute to its weaker instruction-following capability (Llama-7B vs Vicuna-13B).
The paper notes this attack has a 'naturally low success rate' and attributes the gap to CLIP's training data composition (real objects vs. abstract concepts) and the models' training to describe images passively rather than follow instructions in them. Each adversarial image was evaluated 100 times.