Detection limits in the high-dimensional spiked rectangular model.

We study the problem of detecting the presence of a single unknown spike in arectangular data matrix, in a high-dimensional regime where the spike has fixedstrength and the aspect ratio of the matrix converges to a finite limit. Thissituation comprises Johnstone's spiked covariance model. We analyze thelikelihood ratio of the spiked model against an "all noise" null model ofreference, and show it has asymptotically Gaussian fluctuations in a regionbelow---but in general not up to---the so-called BBP threshold from randommatrix theory. Our result parallels earlier findings of Onatski et al.\ (2013)and Johnstone-Onatski (2015) for spherical spikes. We present a probabilisticapproach capable of treating generic product priors. In particular, sparsity inthe spike is allowed. Our approach operates through the general principle ofthe cavity method from spin-glass theory. The question of the maximal parameterregion where asymptotic normality is expected to hold is left open. This regionis shaped by the prior in a non-trivial way. We conjecture that this is theentire paramagnetic phase of an associated spin-glass model, and is defined bythe vanishing of the replica-symmetric solution of Lesieur et al.\ (2015).