Stubborn or Sycophantic? GEPA-Evolved Prompts Under Pressure
HanRui Zhang ⋅ Ashton Chew ⋅ Ryan Soe ⋅ Ahmed Taha ⋅ ⋅ Aditya Shah ⋅ Maheep Chaudhary
Abstract
Large language models often abandon a correct answer when a user pushes back, and system prompts that tell the model to stand firm are a popular, lightweight remedy. The usual evidence for such prompts, a lower answer-change rate, conflates two different behaviors: resisting a wrong suggestion and refusing a correct one. We evaluate four frozen system-prompt conditions, plus two Llama-only controls, under both kinds of feedback on the complete first-party SycoBench-600 protocol, decomposing behavior into Update (baseline-wrong answers corrected after a correct suggestion) and WrongFlip (baseline-correct answers lost after a wrong one) across 14 exact-scored model-condition cells on pinned revisions of Phi-3-mini, Llama-3.1-8B, and Mistral-7B (1,800 baselines per cell). The point estimates are mixed. On Llama, a 55-token compact prompt lowers WrongFlip by 53.7 points but also lowers Update by 46.0, mostly greater resistance to answer changes, not greater selectivity about which changes to make. On Phi, the same prompt raises selectivity (Update $-$ WrongFlip) by 21.7 points, while on Mistral the long prompt lowers it by 22.7. No prompt improves correction selectivity on all three models. An evaluation that counted only reversals would have scored several of these cells as successes; anti-sycophancy interventions should instead be judged under both correct and incorrect user suggestions, separately for each model.
Chat is not available.
Successful Page Load