Where did the ambiguity go? Examining how multimodal models interpret polysemous words

Human language is highly polysemous: many common words (e.g., "bank" or "palm") carry several distinct meanings that shape what humans communicate and imagine. Large language models (LLMs) have been shown to understand this multiplicity of meaning, but much less is known about how polysemy surfaces in other modalities such as images. We study this across 17 text-to-image and 15 text-generation models by giving each a polysemous word with no context to fix its meaning and measuring which senses are produced over many samples. We find a clear multimodal gap: within every model family, generated images settle on far fewer senses than generated sentences (normalized entropy 0.10 vs. 0.25), and both are far less varied than what people imagine for the same words (normalized entropy 0.47). This is not because the models miss the ambiguity. When we instead ask a model to list how often it would generate outputs corresponding to each possible meaning of a word, it produces diverse distributions. The narrowing appears only during the models' generation stage, and preference tuning sharpens it further. These results reveal a multimodal gap in AI systems' capacity to express meaning, a challenge for any model whose understanding must transfer faithfully across language and vision.