SourceThe Hidden Language of Diffusion Models
For polysemous prompts such as 'crane' (machine and bird), 'date' (fruit and romantic), 'big apple' (apple and New York City), 'bridge' (structure and link), and 'palm tree' (tree and coconut), Conceptor reveals that SD's internal representation incorporates both meanings at once. Coefficient manipulation shows that reducing one meaning's element (e.g., 'stork' from 'crane') alters the generated image in ways reflecting the other meaning (the machine's structure changes). In some cases both meanings appear as separate objects in the image (e.g., 'big apple' shows an apple and NYC), while in others they are fused into a single object (e.g., 'crane' borrows appearance from the machine and shape from the bird).