Doing What They Say, Not What They Reason: Locating the Faithfulness Gap in LLM Agents

Do LLM agents act on the reasoning they state? We study this question of process fidelity in a controlled Texas Hold’em simulator, where each decision can be decomposed into reasoning, stated conclusion, executed action, and an equity-based reference action. This lets us separate two links in the agent decision pipeline: reasoning→conclusion and conclusion→action. We find that agents usually execute what they explicitly state, and that much of the apparent conclusion→action inconsistency comes from free-text conclusion parsing rather than model behavior. The main fidelity gap appears upstream. In a diagnostic that elicits each agent’s stated equity, pot odds, rule, and decision, errors split across input mistakes, borderline cases, and rule misapplications. For Claude Haiku 4.5, rule misapplications are overwhelmingly risk-averse, and a simple instruction to apply the rule mechanically roughly halves their rate. DeepSeek-Reasoner shows a different error composition, suggesting that the upstream gap is model-dependent. These findings suggest that agent evaluations should elicit machine-checkable conclusions and distinguish execution consistency from reasoning validity. We release a reproducible poker-agent harness and live multi-agent demo for studying process fidelity in LLM agents.