Black-box Reasoning Path Planning Protocol: A Methodology for Reasoning Structure Integrity Auditing
Abstract
Aiming at the insufficient interpretability and difficult structural verification of black-box reasoning systems, this paper proposes a Black-box Reasoning Path Planning Protocol (BRPPP). First, standardized forward reasoning and backward completion interfaces are defined to construct a reasoning path set with high redundancy and diversity. Second, a relevance-diversity balance mechanism is introduced to guarantee the correlation between each path and reasoning conclusion while expanding path combinations for cross-checking. Third, stripping tests are adopted to carry out quantitative integrity auditing, and two core metrics, Premise Compliance Rate (PCR) and breakpoint set, are used to realize multi-dimensional fine-grained diagnosis of reasoning logic. Experimental cases demonstrate that BRPPP can accurately capture structural breakpoints hidden in reasoning paths, which verifies the reliability of this auditing method. Meanwhile, the embedded diversity penalty mechanism effectively improves path coverage, offering a feasible path-level solution for the credibility evaluation of black-box reasoning outputs.