Extracting correlation structure from large random matrices
Random matrices arise in many areas of engineering, social sciences, and natural sciences. For example, when rows of the random matrix record successive samples of a multivariate response the sample correlation between the columns can reveal important dependency structure in the multivariate response, e.g., stars, hubs and triangles of co-dependency. However, when the number of samples is finite and the number p of columns increases such exploration becomes futile due to a phase transition phenomenon: spurious discoveries will eventually dominate. In this presentation I will present theory for predicting these phase transitions and present Poisson limit theorems that can be used to predict finite sample behavior of correlation structure. The theory has application to areas including gene expression analysis, remote sensing, and portfolio selection. This is joint work Bala Rajaratnam.