Large tensor models for data analysis, machine learning and signal processing – LATENT

Numerous problems in various domains, notably in data science, machine learning and signal processing, can be addressed by resorting to a low-rank tensor model. In general, this requires estimating the parameters of such a model from a given noisy data tensor, which is assumed to contain a low-rank signal bearing the information of interest. Even though several existing algorithms are often capable of accomplishing this notoriously difficult task in a satisfying manner, it is hard to anticipate or guarantee their actual performance in practice.

Significant progress has been achieved in recent years by considering this estimation problem in the regime where the tensor dimensions grow large, which is relevant for a large number of current applications. In particular, this approach has given access to the exact asymptotic performance of several algorithms aimed at estimating a rank-one signal planted in a noisy tensor. However, to this day existing results are mostly confined to the rank-one setting, with a few exceptions that either make too restrictive assumptions, do not cover several algorithms of practical interest, or have not yet reached full maturity.

Project LATENT has two ambitious goals: (i) characterizing the performance and limitations of several tensor model estimation algorithms of practical relevance, beyond the rank-one setting; (ii) proposing novel, improved algorithms with performance guarantees, based upon our findings related to objective (i).

Specifically, we will focus on three widespread tensor models: the canonical polyadic decomposition (CPD), the block-term decomposition (BTD) and the Tucker decomposition (TD). Thanks to the uniqueness of their parameters under quite mild conditions, the CPD and BTD models are commonly used for information extraction purposes, since in many applications they allow revealing the parameters which characterize the causes underlying the observations. By contrast, the TD model is most often used to reduce the dimensionality of a given data tensor, in particular prior to estimating a low-rank CPD or BTD decomposition of the latter, aiming to reduce the problem size (and thus its cost).

For the TD model, we will characterize the exact asymptotic performance attained by several practical spectral estimators, including existing ones and new ones devised by LATENT, as well as that attained by the (idealized) maximum likelihood estimator under Gaussian noise, for reference. Regarding the CPD and BTD models, project LATENT will focus on two classes of estimators, namely algebraic methods such as simultaneous diagonalization, and deflation schemes based on solving a sequence of rank-one approximation problems. Finally, our results will be applied to two application examples in machine learning and in signal processing: respectively, the CPD-based estimation of Gaussian mixture models from empirical moment tensors, and the BTD-based separation of source signals from electrocardiogram records.

The results from LATENT are expected to impact in a significant and lasting manner the activities of practitioners in applied fields such as machine learning and signal processing. Specifically, they will represent a major step towards a better-informed use of tensor methods endowed with guarantees and performance predictions that are useful in applications.