Shorter Context, Fewer Negatives: How Information Reduction Biases LLM Scientific Reasoning
Abstract
When LLM pipelines shorten scientific text through retrieval, summarization, or chunking, do they lose positive and negative findings at equal rates? We find they do not. Across six LLMs spanning efficient to frontier scale and three scientific benchmarks, structured compression shifts conclusions toward positive findings while random deletion does not. The effect is predictable: a model's positive lean, measurable with a single zero-context query, determines its vulnerability; model size and accuracy do not. On tasks where models have moderate accuracy, the ratio of positive to negative conclusions can triple under iterative compression, and three mild reduction steps produce worse distortion than a single aggressive one. A logit lens analysis traces the failure to late transformer layers where the signal for negative conclusions collapses. A simple fix, protecting negation-bearing sentences from removal during compression, closes 88% of the bias gap at zero inference cost. We call this phenomenon automated publication bias: the systematic inflation of positive findings through standard text-processing operations.