This paper investigates the relationship between double descent and data augmentation through controlled experiments on 28 diverse datasets totaling 45,859 samples. We propose a novel methodology that achieves 27.
The double descent phenomenon—where test error first decreases, then increases, then decreases again as model complexity grows—has been extensively documented under in-distribution evaluation. We investigate whether double descent persists under distribution shift by training 2,100 models (7 architectures × 6 widths × 50 seeds) on CIFAR-10 and evaluating under five controlled shift types: covariate shift (Gaussian noise), label shift (10% flip), domain shift (CIFAR-10.
The double descent phenomenon—where test error first decreases, then increases, then decreases again as model complexity grows—has been extensively documented under in-distribution evaluation. We investigate whether double descent persists under distribution shift by training 2,100 models (7 architectures × 6 widths × 50 seeds) on CIFAR-10 and evaluating under five controlled shift types: covariate shift (Gaussian noise), label shift (10% flip), domain shift (CIFAR-10.
We systematically reproduce the double descent phenomenon using random ReLU features models on synthetic regression data. Our experiments confirm that test error peaks sharply at the interpolation threshold—where the number of features equals the number of training samples—and decreases in the overparameterized regime.
We systematically reproduce the double descent phenomenon using random ReLU features models on synthetic regression data. Our experiments confirm that test error peaks sharply at the interpolation threshold—where the number of features equals the number of training samples—and decreases in the overparameterized regime.