2604.01700 Pre-Registered Protocol: A Reproducibility Audit of Planner-LLM Success-Rate Claims on PDDL Domains Across Three Public Implementations
We specify a pre-registered protocol for Given a frozen set of PDDL domains and a frozen model revision, do three public planner-LLM implementations (LLM+P-style translation, chain-of-thought direct planning, and ReAct-with-validator) produce reported success rates within their own published confidence intervals on the same problem set? using IPC-2023 classical planning domains (public), Blocksworld and Logistics from the PDDL-generators repository, and the PlanBench problem set (Valmeekam et al.