A Bayesian Augmented-Learning framework for spectral uncertainty quantification of incomplete records of stochastic processes

A novel Bayesian Augmented-Learning framework, quantifying the uncertainty of spectral representations of stochastic processes in the presence of missing data, is developed. The approach combines additional information (prior domain knowledge) of the physical processes with real, yet incomplete, observations. Bayesian deep learning models are trained to learn the underlying stochastic process, probabilistically capturing temporal dynamics, from the physics-based pre-simulated data. An ensemble of time domain reconstructions are provided through recurrent computations using the learned Bayesian models. Models are characterized by the posterior distribution of model parameters, whereby uncertainties over learned models, reconstructions and spectral representations are all quantified. In particular, three recurrent neural network architectures, (namely long short-term memory, or LSTM, LSTM-Autoencoder, LSTM-Autoencoder with teacher forcing mechanism), which are implemented in a Bayesian framework through stochastic variational inference, are investigated and compared under many missing data scenarios. An example from stochastic dynamics pertaining to the characterization of earthquake-induced stochastic excitations even when the source load data records are incomplete is used to illustrate the framework. Results highlight the superiority of the proposed approach, which adopts additional information, and the versatility of outputting many forms of results in a probabilistic manner

Time-series analysis if data are randomly missing

Applied Science

Artificial neural network approaches and compressive sensing techniques for stochastic process estimation and simulation subject to incomplete data

This research is themed around development of tools for discrete analysis of stochastic processes subject to limited or missing data; more specifically, estimation of stochastic process power spectra from which new process time-histories may be simulated. In this context, the author proposes three novel approaches to power spectrum estimation subject to missing data which comprise the main body of this work. Of particular importance is the fact that all three approaches are adaptable for use in both stationary and evolutionary power spectrum estimation. Numerous arrangements of missing data are tested to simulate a range of possible scenarios to demonstrate the versatility of the proposed methodologies. The first of the three approaches uses an artificial neural network (ANN) based model for stochastic process power spectrum estimation subject to limited / missing data. In this regard, an appropriately defined ANN is utilized to capture the stochastic pattern in the available data in an “average sense”. Next, the extrapolation capabilities of the ANN are exploited for generating realizations of the underlying stochastic process. Finally, power spectrum estimates are derived based on established frequency (e.g. Fourier analysis), or versatile joint time-frequency analysis techniques (e.g. harmonic wavelets) for the cases of stationary and non-stationary stochastic processes, respectively. One of the significant advantages of the approach relates to the fact that no a priori knowledge about the data is assumed. The second approach uses compressive sensing (CS) to solve the same problem. In this setting, further assumptions are imposed on the nature of the underlying process of interest than in the ANN case, in particular that of sparsity in the frequency domain. The advantages being that when compared to ANN, significant improvements in efficiency and accuracy are achieved with increased reliability for larger amounts of missing data. Specifically, first an appropriate basis is selected for expanding the signal recorded in the time domain. As with the ANN approach, Fourier and harmonic wavelet bases are utilized. Next, an L1 norm minimization procedure is performed for obtaining the sparsest representation of the signal in the selected basis. Further, an adaptive basis procedure is introduced that significantly improves results when working with stochastic process record ensembles. The final approach is somewhat different, in that it aims to quantify uncertainty in power spectrum estimation subject to missing data rather than provide deterministic predictions. By relying on relatively relaxed assumptions for the missing data, utilizing fundamental concepts from probability theory, and resorting to Fourier and harmonic wavelets based representations of stationary and non-stationary stochastic processes, respectively, a closed-form expression is derived for the probability density function (PDF) of the power spectrum value corresponding to a specific frequency. Numerical examples demonstrate the large extent to which any given single estimate using deterministic methods, even for small amounts of missing data, may be unrepresentative of the target spectrum. In this regard, this probabilistic approach can be potentially used to bound deterministic estimates, providing specific validation criteria for missing data reconstruction