High-Dimensional Regression with Gaussian Mixtures and Partially-Latent Response Variables

In this work we address the problem of approximating high-dimensional data with a low-dimensional representation. We make the following contributions. We propose an inverse regression method which exchanges the roles of input and response, such that the low-dimensional variable becomes the regressor, and which is tractable. We introduce a mixture of locally-linear probabilistic mapping model that starts with estimating the parameters of inverse regression, and follows with inferring closed-form solutions for the forward parameters of the high-dimensional regression problem of interest. Moreover, we introduce a partially-latent paradigm, such that the vector-valued response variable is composed of both observed and latent entries, thus being able to deal with data contaminated by experimental artifacts that cannot be explained with noise models. The proposed probabilistic formulation could be viewed as a latent-variable augmentation of regression. We devise expectation-maximization (EM) procedures based on a data augmentation strategy which facilitates the maximum-likelihood search over the model parameters. We propose two augmentation schemes and we describe in detail the associated EM inference procedures that may well be viewed as generalizations of a number of EM regression, dimension reduction, and factor analysis algorithms. The proposed framework is validated with both synthetic and real data. We provide experimental evidence that our method outperforms several existing regression techniques

A Comparison of the Machine Learning Algorithm for Evaporation Duct Estimation

In this research, a comparison of the relevance vector machine (RVM), least square support vector machine (LSSVM) and the radial basis function neural network (RBFNN) for evaporation duct estimation are presented. The parabolic equation model is adopted as the forward propagation model, and which is used to establish the training database between the radar sea clutter power and the evaporation duct height. The comparison of the RVM, LSSVM and RBFNN for evaporation duct estimation are investigated via the experimental and the simulation studies, and the statistical analysis method is employed to analyze the performance of the three machine learning algorithms in the simulation study. The analysis demonstrate that the M profile of RBFNN estimation has a relatively good match to the measured profile for the experimental study; for the simulation study, the LSSVM is the most precise one among the three machine learning algorithms, besides, the performance of RVM is basically identical to the RBFNN

Hyper-Spectral Image Analysis with Partially-Latent Regression and Spatial Markov Dependencies

Hyper-spectral data can be analyzed to recover physical properties at large planetary scales. This involves resolving inverse problems which can be addressed within machine learning, with the advantage that, once a relationship between physical parameters and spectra has been established in a data-driven fashion, the learned relationship can be used to estimate physical parameters for new hyper-spectral observations. Within this framework, we propose a spatially-constrained and partially-latent regression method which maps high-dimensional inputs (hyper-spectral images) onto low-dimensional responses (physical parameters such as the local chemical composition of the soil). The proposed regression model comprises two key features. Firstly, it combines a Gaussian mixture of locally-linear mappings (GLLiM) with a partially-latent response model. While the former makes high-dimensional regression tractable, the latter enables to deal with physical parameters that cannot be observed or, more generally, with data contaminated by experimental artifacts that cannot be explained with noise models. Secondly, spatial constraints are introduced in the model through a Markov random field (MRF) prior which provides a spatial structure to the Gaussian-mixture hidden variables. Experiments conducted on a database composed of remotely sensed observations collected from the Mars planet by the Mars Express orbiter demonstrate the effectiveness of the proposed model.Comment: 12 pages, 4 figures, 3 table

The ROMES method for statistical modeling of reduced-order-model error

This work presents a technique for statistically modeling errors introduced by reduced-order models. The method employs Gaussian-process regression to construct a mapping from a small number of computationally inexpensive `error indicators' to a distribution over the true error. The variance of this distribution can be interpreted as the (epistemic) uncertainty introduced by the reduced-order model. To model normed errors, the method employs existing rigorous error bounds and residual norms as indicators; numerical experiments show that the method leads to a near-optimal expected effectivity in contrast to typical error bounds. To model errors in general outputs, the method uses dual-weighted residuals---which are amenable to uncertainty control---as indicators. Experiments illustrate that correcting the reduced-order-model output with this surrogate can improve prediction accuracy by an order of magnitude; this contrasts with existing `multifidelity correction' approaches, which often fail for reduced-order models and suffer from the curse of dimensionality. The proposed error surrogates also lead to a notion of `probabilistic rigor', i.e., the surrogate bounds the error with specified probability

Hybrid Advanced Optimization Methods with Evolutionary Computation Techniques in Energy Forecasting

More accurate and precise energy demand forecasts are required when energy decisions are made in a competitive environment. Particularly in the Big Data era, forecasting models are always based on a complex function combination, and energy data are always complicated. Examples include seasonality, cyclicity, fluctuation, dynamic nonlinearity, and so on. These forecasting models have resulted in an over-reliance on the use of informal judgment and higher expenses when lacking the ability to determine data characteristics and patterns. The hybridization of optimization methods and superior evolutionary algorithms can provide important improvements via good parameter determinations in the optimization process, which is of great assistance to actions taken by energy decision-makers. This book aimed to attract researchers with an interest in the research areas described above. Specifically, it sought contributions to the development of any hybrid optimization methods (e.g., quadratic programming techniques, chaotic mapping, fuzzy inference theory, quantum computing, etc.) with advanced algorithms (e.g., genetic algorithms, ant colony optimization, particle swarm optimization algorithm, etc.) that have superior capabilities over the traditional optimization approaches to overcome some embedded drawbacks, and the application of these advanced hybrid approaches to significantly improve forecasting accuracy

Hybrid Verification for Analog and Mixed-signal Circuits

With increasing design complexity and reliability requirements, analog and mixedsignal (AMS) verification manifests itself as a key bottleneck. While formal methods and machine learning have been proposed for AMS verification, these two types of techniques suffer from their own limitations, with the former being specifically limited by scalability and the latter by inherent errors in learning-based models. We present a new direction in AMS verification by proposing a hybrid formal/machinelearning- based verification technique (HFMV) to combine the best of the two worlds. HFMV builds formalism on the top of a machine learning model to verify AMS circuits efficiently while meeting a user-specified confidence level. Guided by formal checks, HFMV intelligently explores the high-dimensional parameter space of a given design by iteratively improving the machine learning model. As a result, it leads to accurate failure prediction in the case of a failing circuit or a reliable pass decision in the case of a good circuit. Our experimental results demonstrate that the proposed HFMV approach is capable of identifying hard-to-find failures which are completely missed by a huge number of random simulation samples while significantly cutting down training sample size and verification cycle time

Prognostics and Health Management for the Optimization of Marine Hybrid Energy Systems

Decarbonization of marine transport is a key global issue, with the carbon emissions of international shipping projected to increase 23% to 1090 million tonnes by 2035 in comparison to 2015 levels. Optimization of the energy system (especially propulsion system) in these vessels is a complex multi-objective challenge involving economical maintenance, environmental metrics, and energy demand requirements. In this paper, data from instrumented vessels on the River Thames in London, which includes environmental emissions, power demands, journey patterns, and variance in operational patterns from the captain(s) and loading (passenger numbers), is integrated and analyzed through automatic, multi-objective global optimization to create an optimal hybrid propulsion configuration for a hybrid vessel. We propose and analyze a number of computational techniques, both for monitoring and remaining useful lifetime (RUL) estimation of individual energy assets, as well as modeling and optimization of energy use scenarios of a hybrid-powered vessel. Our multi-objective optimization relates to emissions, asset health, and power performance. We show that, irrespective of the battery packs used, our Relevance Vector Machine (RVM) algorithm is able to achieve over 92% accuracy in remaining useful life (RUL) predictions. A k-nearest neighbors algorithm (KNN) is proposed for prognostics of state of charge (SOC) of back-up lead-acid batteries. The classifier achieved an average of 95.5% accuracy in a three-fold cross validation. Utilizing operational data from the vessel, optimal autonomous propulsion strategies are modeled combining the use of battery and diesel engines. The experiment results show that 70% to 80% of fuel saving can be achieved when the diesel engine is operated up to 350 kW. Our methodology has demonstrated the feasibility of combination of artificial intelligence (AI) methods and real world data in decarbonization and optimization of green technologies for maritime propulsion