Do Agents Learn Pragmatics, or Just Their Partner? Cross-Play Evaluation of Communicative Self-Play in Situated Reference Games

Language agents are increasingly trained, filtered, or evaluated through interaction with simulated partners. In pragmatic communication, however, success with the training partner can reflect a private convention rather than a generally interpretable message. We study this issue in pragmatic cross-play (PRAG-CrossPlay), a situated reference-game benchmark where a speaker describes a hidden target object and a listener must choose the target from a text-rendered scene. Success is objective: the listener's object ID is checked against ground truth, without an LLM judge. We compare mirror self-play, which selects utterances using one fixed simulated listener, with population-play, which selects utterances using a heterogeneous listener population, and evaluate both with held-out model listeners. The main pattern is simple: mirror self-play reaches 1.00 same-partner success but loses under cross-play, while population-play remains robust. Across different held-out listeners, mirror cross-play ranges from 0.65-0.81 on the hardest settings, whereas population-play reaches 1.00 in every row. Stronger gpt-5.5 speaker generation nearly solves direct first-candidate reference (0.99) but still leaves a mirror-selection gap. A no-coordinate stress test shows that robust non-coordinate messages are available, but selector design remains the bottleneck. These results support cross-play as a compact diagnostic for distinguishing robust pragmatic communication from partner-specific shortcuts.