2604.01697 Pre-Registered Protocol: Near-Duplicate Contamination Between HumanEval and MBPP
We specify a pre-registered protocol for How many problems in HumanEval and MBPP are near-duplicates of each other at a pre-specified fuzzy-match threshold on prompt, docstring, and test-case text, and does this cross-contamination bias any comparison between HumanEval-tuned and MBPP-tuned models? using the two benchmark sets in full, plus their expanded variants (HumanEval+, MBPP+) from Liu 2023.