MED-EVAL: Clinical Fact Alignment for Efficient Medical Reasoning

Similar aggregate evaluation scores can mask clinical failure modes in long-form medical answers. One response may omit an important finding, another may reverse a negation, and a third may introduce an unsupported treatment relation. We introduce MED-EVAL, a fact-centric evaluation framework with a typed representation, linked evidence, provenance-aware one-to-one alignment, and optional fact-feedback post-training. It identifies supported, contradicted, unsupported, missing, audit-needed, and duplicate assertions while retaining material qualifiers and source spans. The framework is deliberately light on expert labeling: two medical experts independently annotated every sample in a 35-sample expert set, and the resulting consensus annotations provide few-shot supervision that bootstraps an automatic, gate-checked feedback signal. This keeps expert effort a fixed, one-time cost, not a per-example burden. In a paired study of two compact-model families on four medical-QA datasets, fact-feedback supervised fine-tuning (SFT) raises pooled supported-fact rate by 6.6 points for Llama and 5.9 points for Qwen2.5 while reducing severe-error rate by 3.3 and 3.0 points, respectively. On the same 35-sample set, the learned extractor achieves κ=.84 against expert consensus, compared to .43 for the strongest answer-level baseline, and it cuts expert review time by 1.7--1.8 minutes per answer. Because the sample is small, we report every interval in full. To motivate this controlled design, we also audit 400 answers left over from an earlier, uncontrolled run and show that the base and fine-tuned sets share no common questions, which rules out any valid before-versus-after comparison.