Bayesian dimensionality reduction with PCA using penalized semi-integrated likelihood

We discuss the problem of estimating the number of principal components in Principal Com- ponents Analysis (PCA). Despite of the importance of the problem and the multitude of solutions proposed in the literature, it comes as a surprise that there does not exist a coherent asymptotic framework which would justify different approaches depending on the actual size of the data set. In this paper we address this issue by presenting an approximate Bayesian approach based on Laplace approximation and introducing a general method for building the model selection criteria, called PEnalized SEmi-integrated Likelihood (PESEL). Our general framework encompasses a variety of existing approaches based on probabilistic models, like e.g. Bayesian Information Criterion for the Probabilistic PCA (PPCA), and allows for construction of new criteria, depending on the size of the data set at hand. Specifically, we define PESEL when the number of variables substantially exceeds the number of observations. We also report results of extensive simulation studies and real data analysis, which illustrate good properties of our proposed criteria as compared to the state-of- the-art methods and very recent proposals. Specifially, these simulations show that PESEL based criteria can be quite robust against deviations from the probabilistic model assumptions. Selected PESEL based criteria for the estimation of the number of principal components are implemented in R package varclust, which is available on github (https://github.com/psobczyk/varclust).Comment: 31 pages, 7 figure

Decomposition of multicomponent mass spectra using Bayesian probability theory

We present a method for the decomposition of mass spectra of mixture gases using Bayesian probability theory. The method works without any calibration measurement and therefore applies also to the analysis of spectra containing unstable species. For the example of mixtures of three different hydrocarbon gases the algorithm provides concentrations and cracking coefficients of each mixture component as well as their confidence intervals. The amount of information needed to obtain reliable results and its relation to the accuracy of our analysis are discussed

Bayesian Identification of Elastic Constants in Multi-Directional Laminate from Moir\'e Interferometry Displacement Fields

The ply elastic constants needed for classical lamination theory analysis of multi-directional laminates may differ from those obtained from unidirectional laminates because of three dimensional effects. In addition, the unidirectional laminates may not be available for testing. In such cases, full-field displacement measurements offer the potential of identifying several material properties simultaneously. For that, it is desirable to create complex displacement fields that are strongly influenced by all the elastic constants. In this work, we explore the potential of using a laminated plate with an open-hole under traction loading to achieve that and identify all four ply elastic constants (E 1, E 2, 12, G 12) at once. However, the accuracy of the identified properties may not be as good as properties measured from individual tests due to the complexity of the experiment, the relative insensitivity of the measured quantities to some of the properties and the various possible sources of uncertainty. It is thus important to quantify the uncertainty (or confidence) with which these properties are identified. Here, Bayesian identification is used for this purpose, because it can readily model all the uncertainties in the analysis and measurements, and because it provides the full coupled probability distribution of the identified material properties. In addition, it offers the potential to combine properties identified based on substantially different experiments. The full-field measurement is obtained by moir\'e interferometry. For computational efficiency the Bayesian approach was applied to a proper orthogonal decomposition (POD) of the displacement fields. The analysis showed that the four orthotropic elastic constants are determined with quite different confidence levels as well as with significant correlation. Comparison with manufacturing specifications showed substantial difference in one constant, and this conclusion agreed with earlier measurement of that constant by a traditional four-point bending test. It is possible that the POD approach did not take full advantage of the copious data provided by the full field measurements, and for that reason that data is provided for others to use (as on line material attached to the article)

Bayesian Inference on Matrix Manifolds for Linear Dimensionality Reduction

We reframe linear dimensionality reduction as a problem of Bayesian inference on matrix manifolds. This natural paradigm extends the Bayesian framework to dimensionality reduction tasks in higher dimensions with simpler models at greater speeds. Here an orthogonal basis is treated as a single point on a manifold and is associated with a linear subspace on which observations vary maximally. Throughout this paper, we employ the Grassmann and Stiefel manifolds for various dimensionality reduction problems, explore the connection between the two manifolds, and use Hybrid Monte Carlo for posterior sampling on the Grassmannian for the first time. We delineate in which situations either manifold should be considered. Further, matrix manifold models are used to yield scientific insight in the context of cognitive neuroscience, and we conclude that our methods are suitable for basic inference as well as accurate prediction.Comment: All datasets and computer programs are publicly available at http://www.ics.uci.edu/~babaks/Site/Codes.htm

Efficient and automatic methods for flexible regression on spatiotemporal data, with applications to groundwater monitoring

Fitting statistical models to spatiotemporal data requires finding the right balance between imposing smoothness and following the data. In the context of P-splines, we propose a Bayesian framework for choosing the smoothing parameter which allows the construction of fully-automatic data-driven methods for fitting flexible models to spatiotemporal data. An implementation, which is highly computationally efficient and which exploits the sparsity of the design and penalty matrices, is proposed. The findings are illustrated using a simulation study and two examples, all concerned with the modelling of contaminants in groundwater. This suggests that the proposed strategy is more stable that competing methods based on the use of criteria such as GCV and AIC

Bayesian inference in the time varying cointegration model

There are both theoretical and empirical reasons for believing that the parameters of macroeconomic models may vary over time. However, work with time-varying parameter models has largely involved Vector autoregressions (VARs), ignoring cointegration. This is despite the fact that cointegration plays an important role in informing macroeconomists on a range of issues. In this paper we develop time varying parameter models which permit coin- tegration. Time-varying parameter VARs (TVP-VARs) typically use state space representations to model the evolution of parameters. In this paper, we show that it is not sensible to use straightforward extensions of TVP-VARs when allowing for cointegration. Instead we develop a speci…cation which allows for the cointegrating space to evolve over time in a manner comparable to the random walk variation used with TVP-VARs. The properties of our approach are investigated before developing a method of posterior simulation. We use our methods in an empirical investigation involving a permanent/transitory variance decomposition for inflation