Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation

Self-Generated Text Recognition (SGTR)---the ability of an LLM to identify its own outputs---poses risks to AI safeguards that rely on LLMs as evaluators or monitors. Namely, an LLM may recognize outputs from other copies of the same model and make biased judgments or collude outright. Prior work has drawn conflicting conclusions about whether current models possess significant SGTR capabilities. We reconcile these findings by identifying key experimental design choices---which we term \textit{operationalizations}---that drive divergent results. Evaluating between 13-21 models across eight operationalizations, we find that accuracy varies substantially with evaluation format (pairwise vs. individual assessments of text), conversation structure (presenting candidate text in user-tags vs. assistant-tags), and the domain of task queried to generate candidate text (e.g., coding vs. summarization). We corroborate previous observations that a quality heuristic---models attributing authorship to text they perceive as higher quality---is a dominant confound. We also find that improving a model's SGTR performance via SFT in one evaluation configuration can generalize to others. We find that training for SGTR also causes models to prefer their own outputs when acting as a judge in the AlpacaEval framework. Finally, we discuss the implications of our evaluations for the safety of future AI systems: our work suggests that, despite confounds, some models possess practical SGTR capabilities, and that training a model for SGTR in one setting can affect its self-recognition and self-preference more generally. Finally, we discuss the implications of our evaluations for the safety of future AI systems: our work suggests that, despite confounds, models possess practical SGTR capabilities. We believe that SGTR should be monitored and considered in the design of safety-critical AI applications.