A novel dimensionality reduction technique based on independent component analysis for modeling microarray gene expression data

DNA microarray experiments generating thousands of gene expression measurements, are being used to gather information from tissue and cell samples regarding gene expression differences that will be useful in diagnosing disease. But one challenge of microarray studies is the fact that the number n of samples collected is relatively small compared to the number p of genes per sample which are usually in thousands. In statistical terms this very large number of predictors compared to a small number of samples or observations makes the classification problem difficult. This is known as the ”curse of dimensionality problem”. An efficient way to solve this problem is by using dimensionality reduction techniques. Principle Component Analysis(PCA) is a leading method for dimensionality reduction of gene expression data which is optimal in the sense of least square error. In this paper we propose a new dimensionality reduction technique for specific bioinformatics applications based on Independent component Analysis(ICA). Being able to exploit higher order statistics to identify a linear model result, this ICA based dimensionality reduction technique outperforms PCA from both statistical and biological significance aspects. We present experiments on NCI 60 dataset to show this result

Linear Hamilton Jacobi Bellman Equations in High Dimensions

The Hamilton Jacobi Bellman Equation (HJB) provides the globally optimal solution to large classes of control problems. Unfortunately, this generality comes at a price, the calculation of such solutions is typically intractible for systems with more than moderate state space size due to the curse of dimensionality. This work combines recent results in the structure of the HJB, and its reduction to a linear Partial Differential Equation (PDE), with methods based on low rank tensor representations, known as a separated representations, to address the curse of dimensionality. The result is an algorithm to solve optimal control problems which scales linearly with the number of states in a system, and is applicable to systems that are nonlinear with stochastic forcing in finite-horizon, average cost, and first-exit settings. The method is demonstrated on inverted pendulum, VTOL aircraft, and quadcopter models, with system dimension two, six, and twelve respectively.Comment: 8 pages. Accepted to CDC 201

Canonical correlation analysis and DEA for azorean agriculture efficiency

In this paper we will document the application of canonical correlation analysis to variable aggregation using the correlations of the original variables with the canonical variates. A case study, about farms in Terceira Island, with a small data set is presented. In this data set of 30 farms we intend to use 17 input variables and 2 output variables to measure DEA efficiency. Without any data reduction procedure several problems known as “curse of dimensionality” are expected. With the data reduction procedures suggested it was possible to conclude quite acceptable and domain consistent conclusions.N/

Encoding and Decoding Techniques for Distributed Data Storage Systems

Dimensionality reduction is the conversion of high-dimensional data into a meaningful representation of reduced data. Preferably, the reduced representation has a dimensionality that corresponds to the essential dimensionality of the data. The essential dimensionality of data is the minimum number of parameters needed to account for the observed properties of the data [4]. Dimensionality reduction is important in many domains, since it facilitates classification, visualization, and compression of high-dimensional data, by helpful the curse of dimensionality and other undesired properties of high-dimensional spaces [5]. Dimension reduction can be beneficial not only for reasons of computational efficiency but also because it can improve the accuracy of the analysis. In this research area, it significantly reduces the storage spaces