Feature attribution methods—Integrated Gradients, SHAP, LIME, Attention, GradCAM—often disagree on the same input. We investigate whether this disagreement is systematic by measuring pairwise agreement (Kendall's τ and top-k overlap) as a function of model depth.
Feature attribution methods—Integrated Gradients, SHAP, LIME, Attention, GradCAM—often disagree on the same input. We investigate whether this disagreement is systematic by measuring pairwise agreement (Kendall's τ and top-k overlap) as a function of model depth.
Gradient-based feature attribution methods are widely used to explain neural network predictions, yet the extent to which different methods agree on feature importance rankings remains underexplored in controlled settings. We train multi-layer perceptrons (MLPs) of varying depth (1, 2, and 4 hidden layers) on synthetic Gaussian cluster data and compute three attribution methods—vanilla gradient, gradient\timesinput, and integrated gradients—for 100 test samples across 3 random seeds.