Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks

We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto-Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM-LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.Comment: 31 page

Invariances of random fields paths, with applications in Gaussian Process Regression

We study pathwise invariances of centred random fields that can be controlled through the covariance. A result involving composition operators is obtained in second-order settings, and we show that various path properties including additivity boil down to invariances of the covariance kernel. These results are extended to a broader class of operators in the Gaussian case, via the Lo\`eve isometry. Several covariance-driven pathwise invariances are illustrated, including fields with symmetric paths, centred paths, harmonic paths, or sparse paths. The proposed approach delivers a number of promising results and perspectives in Gaussian process regression

GPR clutter amplitude processing to detect shallow geological targets

The analysis of clutter in A-scans produced by energy randomly scattered in some specific geological structures, provides information about changes in the shallow sedimentary geology. The A-scans are composed by the coherent energy received from reflections on electromagnetic discontinuities and the incoherent waves from the scattering in small heterogeneities. The reflected waves are attenuated as consequence of absorption, geometrical spreading and losses due to reflections and scattering. Therefore, the amplitude of those waves diminishes and at certain two-way travel times becomes on the same magnitude as the background noise in the radargram, mainly produced by the scattering. The amplitude of the mean background noise is higher when the dispersion of the energy increases. Then, the mean amplitude measured in a properly selected time window is a measurement of the amount of the scattered energy and, therefore, a measurement of the increase of scatterers in the ground. This paper presents a simple processing that allows determining the Mean Amplitude of Incoherent Energy (MAEI) for each A-scan, which is represented in front of the position of the trace. This procedure is tested in a field study, in a city built on a sedimentary basin. The basin is crossed by a large number of hidden subterranean streams and paleochannels. The sedimentary structures due to alluvial deposits produce an amount of the random backscattering of the energy that is measured in a time window. The results are compared along the entire radar line, allowing the location of streams and paleochannels. Numerical models were also used in order to compare the synthetic traces with the field radargrams and to test the proposed processing methodology. The results underscore the amount of the MAEI over the streams and also the existence of a surrounding zone where the amplitude is increasing from the average value to the maximum obtained over the structure. Simulations show that this zone does not correspond to any particular geological change but is consequence of the path of the antenna that receives the scattered energy before arriving to the alluvial depositsPeer ReviewedPostprint (published version

Integrated Bayesian Framework for Remaining Useful Life Prediction.

International audienceIn this paper, a data-driven method for remaining useful life (RUL) prediction is presented. The method learns the relation between acquired sensor data and end of life time (EOL) to predict the RUL. The proposed method extracts monotonic trends from offline sensor signals, which are used to build reference models. From online signals the method represents the uncertainty about the current status, using discrete Bayesian filter. Finally, the method predicts RUL of the monitored component using integrated method based on K-nearest neighbor (k-NN) and Gaussian process regression (GPR). The performance of the algorithm is demonstrated using two real data sets from NASA Ames prognostics data repository. The results show that the algorithm obtain good results for both application

Statistical efficiency of structured cpd estimation applied to Wiener-Hammerstein modeling

Accepted for publication in the Proceedings of the European Signal Processing Conference (EUSIPCO) 2015.International audienceThe computation of a structured canonical polyadic decomposition (CPD) is useful to address several important modeling problems in real-world applications. In this paper, we consider the identification of a nonlinear system by means of a Wiener-Hammerstein model, assuming a high-order Volterra kernel of that system has been previously estimated. Such a kernel, viewed as a tensor, admits a CPD with banded circulant factors which comprise the model parameters. To estimate them, we formulate specialized estimators based on recently proposed algorithms for the computation of structured CPDs. Then, considering the presence of additive white Gaussian noise, we derive a closed-form expression for the Cramer-Rao bound (CRB) associated with this estimation problem. Finally, we assess the statistical performance of the proposed estimators via Monte Carlo simulations, by comparing their mean-square error with the CRB

Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial

On top of machine learning models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Toward this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. Two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage and the remaining useful life prediction of turbofan engines

Automatic Road Survey by Using Vehicle Mounted Point Laser for Local Road Management

In most countries local roads (i.e., urban and rural) form over 80% of the entire road network and constitute the country's largest asset value. In order for local roads to remain fit for purpose and maintain their value, they require periodic maintenance. To make the best use of scarce maintenance resources, road maintenance needs to be preventative which requires the condition of the road to be assessed periodically. Traditional road surveys suffer from the lack of repeatability and reproducibility, are high cost and time consuming. This work proposes a vehicle mounted point laser system for the automated, rapid and inexpensive measurement of a major mode of local road deterioration, namely fretting. Compared to other technologies such as Ground Penetrating Radar (GPR), visual sensors and the Mobile Laser Scanning (MLS) system, the point laser requires less computational power, is less sensitive to the surrounding environment and is of comparatively low cost. A robust approach is proposed which consists of a number of pre-processing algorithms to deal with noise and the effects of the vehicles dynamic motion, and a signal processing algorithm which analyses histograms of the distance from the road surface measured by the laser to account for changes in road texture. Road fretting measured by the proposed system on a variety of roads is compared with fretting determined using a standard visual assessment process. The results indicate that the proposed system can measure road fretting to the levels of detail which are suitable for planning, programming and preparations road management functions

Automatic Road Survey by Using Vehicle Mounted Laser for Road Asset Management

In most countries local roads (i.e., urban and rural) form over 80% of the entire road network and constitute the country's largest asset value. In order for local roads to remain fit for purpose and maintain their value, they require periodic maintenance. To make the best use of scarce maintenance resources, road maintenance needs to be preventative which requires the condition of the road to be assessed periodically. Traditional road surveys suffer from the lack of repeatability and reproducibility, are high cost and time consuming. This work proposes a vehicle mounted point laser system for the automated, rapid and inexpensive measurement of a major mode of local road deterioration, namely fretting. Compared to other technologies such as Ground Penetrating Radar (GPR), visual sensors and the Mobile Laser Scanning (MLS) system, the point laser requires less computational power, is less sensitive to the surrounding environment and is of comparatively low cost. A robust approach is proposed which consists of a number of pre-processing algorithms to deal with noise and the effects of the vehicles dynamic motion, and a signal processing algorithm which analyses histograms of the distance from the road surface measured by the laser to account for changes in road texture. Road fretting measured by the proposed system on a variety of roads is compared with fretting determined using a standard visual assessment process. The results indicate that the proposed system can measure road fretting to the levels of detail which are suitable for planning, programming and preparations road management functions