Discovery of Latent Factors in High-dimensional Data Using Tensor Methods

Discovery of Latent Factors in High-dimensional Data Using Tensor Methods: Learning latent variable mixture models in high-dimension is applicable in numerous domains where low-dimensional latent factors out of the high-dimensional observations are desired. Popular likelihood based methods optimize over a non-convex likelihood which is computationally challenging due to the high-dimensionality of the data, and it is usually not guaranteed to converge to a global or even local optima without additional assumptions. We propose a framework to overcome the problem of unscalable and non-convex likelihood by taking the power of inverse method of moments. By matching the moments, the problem of learning latent variable mixture models is reduced to a tensor (higher order matrix) decomposition problem in low-dimensional space. This framework incorporates dimensionality reduction and thus is scalable to high-dimensional data. Moreover, we show that the algorithm is highly parallel and implemented a distributed version on Spark in Scala language.

View the slides for this session