Tensor-on-tensor regression

We propose a framework for the linear prediction of a multi-way array (i.e., a tensor) from another multi-way array of arbitrary dimension, using the contracted tensor product. This framework generalizes several existing approaches, including methods to predict a scalar outcome from a tensor, a matrix from a matrix, or a tensor from a scalar. We describe an approach that exploits the multiway structure of both the predictors and the outcomes by restricting the coefficients to have reduced CP-rank. We propose a general and efficient algorithm for penalized least-squares estimation, which allows for a ridge (L_2) penalty on the coefficients. The objective is shown to give the mode of a Bayesian posterior, which motivates a Gibbs sampling algorithm for inference. We illustrate the approach with an application to facial image data. An R package is available at https://github.com/lockEF/MultiwayRegression .Comment: 33 pages, 3 figure

TRES: An R Package for Tensor Regression and Envelope Algorithms

Recently, there has been a growing interest in tensor data analysis, where tensor regression is the cornerstone of statistical modeling for tensor data. The R package TRES provides the standard least squares estimators and the more efficient envelope estimators for the tensor response regression (TRR) and the tensor predictor regression (TPR) models. Envelope methodology provides a relatively new class of dimension reduction techniques that jointly models the regression mean and covariance parameters. Three types of widely applicable envelope estimation algorithms are implemented and applied to both TRR and TPR models

An Alternative Approach to Functional Linear Partial Quantile Regression

We have previously proposed the partial quantile regression (PQR) prediction procedure for functional linear model by using partial quantile covariance techniques and developed the simple partial quantile regression (SIMPQR) algorithm to efficiently extract PQR basis for estimating functional coefficients. However, although the PQR approach is considered as an attractive alternative to projections onto the principal component basis, there are certain limitations to uncovering the corresponding asymptotic properties mainly because of its iterative nature and the non-differentiability of the quantile loss function. In this article, we propose and implement an alternative formulation of partial quantile regression (APQR) for functional linear model by using block relaxation method and finite smoothing techniques. The proposed reformulation leads to insightful results and motivates new theory, demonstrating consistency and establishing convergence rates by applying advanced techniques from empirical process theory. Two simulations and two real data from ADHD-200 sample and ADNI are investigated to show the superiority of our proposed methods

Tensor on tensor regression with tensor normal errors and tensor network states on the regression parameter

With the growing interest in tensor regression models and decompositions, the tensor normal distribution offers a flexible and intuitive way to model multi-way data and error dependence. In this paper we formulate two regression models where the responses and covariates are both tensors of any number of dimensions and the errors follow a tensor normal distribution. The first model uses a CANDECOMP/PARAFAC (CP) structure and the second model uses a Tensor Chain (TC) structure, and in both cases we derive Maximum Likelihood Estimators (MLEs) and their asymptotic distributions. Furthermore we formulate a tensor on tensor regression model with a Tucker structure on the regression parameter and estimate the parameters using least squares. Aditionally, we find the fisher information matrix of the covariance parameters in an independent sample of tensor normally distributed variables with mean 0, and show that this fisher information also applies to the covariances in the multilinear tensor regression model \cite{Hoff2014} and tensor on tensor models with tensor normal errors regardless of the structure on the regression parameter