Temporal Learning in Video Data Using Deep Learning and Gaussian Processes

This paper presents an approach for data-driven modeling of hidden, stationary temporal dynamics in sequential images or videos using deep learning and Bayesian non-parametric techniques. In particular, a deep Convolutional Neural Network (CNN) is used to extract spatial features in an unsupervised fashion from individual images and then, a Gaussian process is used to model the temporal dynamics of the spatial features extracted by the deep CNN. By decomposing the spatial and temporal components and utilizing the strengths of deep learning and Gaussian processes for the respective sub-problems, we are able to construct a model that is able to capture complex spatio-temporal phenomena while using relatively small number of free parameters. The proposed approach is tested on high-speed grey-scale video data obtained of combustion flames in a swirl-stabilized combustor, where certain protocols are used to induce instability in combustion process. The proposed approach is then used to detect and predict the transition of the combustion process from stable to unstable regime. It is demonstrated that the proposed approach is able to detect unstable flame conditions using very few frames from high-speed video. This is useful as early detection of unstable combustion can lead to better control strategies to mitigate instability. Results from the proposed approach are compared and contrasted with several baselines and recent work in this area. The performance of the proposed approach is found to be significantly better in terms of detection accuracy, model complexity and lead-time to detection

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

2022 Review of Data-Driven Plasma Science

Data-driven science and technology offer transformative tools and methods to science. This review article highlights the latest development and progress in the interdisciplinary field of data-driven plasma science (DDPS), i.e., plasma science whose progress is driven strongly by data and data analyses. Plasma is considered to be the most ubiquitous form of observable matter in the universe. Data associated with plasmas can, therefore, cover extremely large spatial and temporal scales, and often provide essential information for other scientific disciplines. Thanks to the latest technological developments, plasma experiments, observations, and computation now produce a large amount of data that can no longer be analyzed or interpreted manually. This trend now necessitates a highly sophisticated use of high-performance computers for data analyses, making artificial intelligence and machine learning vital components of DDPS. This article contains seven primary sections, in addition to the introduction and summary. Following an overview of fundamental data-driven science, five other sections cover widely studied topics of plasma science and technologies, i.e., basic plasma physics and laboratory experiments, magnetic confinement fusion, inertial confinement fusion and high-energy-density physics, space and astronomical plasmas, and plasma technologies for industrial and other applications. The final section before the summary discusses plasma-related databases that could significantly contribute to DDPS. Each primary section starts with a brief introduction to the topic, discusses the state-of-the-art developments in the use of data and/or data-scientific approaches, and presents the summary and outlook. Despite the recent impressive signs of progress, the DDPS is still in its infancy. This article attempts to offer a broad perspective on the development of this field and identify where further innovations are required

Towards a solution of the closure problem for convective atmospheric boundary-layer turbulence

We consider the closure problem for turbulence in the dry convective atmospheric boundary layer (CBL). Transport in the CBL is carried by small scale eddies near the surface and large plumes in the well mixed middle part up to the inversion that separates the CBL from the stably stratified air above. An analytically tractable model based on a multivariate Delta-PDF approach is developed. It is an extension of the model of Gryanik and Hartmann [1] (GH02) that additionally includes a term for background turbulence. Thus an exact solution is derived and all higher order moments (HOMs) are explained by second order moments, correlation coefficients and the skewness. The solution provides a proof of the extended universality hypothesis of GH02 which is the refinement of the Millionshchikov hypothesis (quasi- normality of FOM). This refined hypothesis states that CBL turbulence can be considered as result of a linear interpolation between the Gaussian and the very skewed turbulence regimes. Although the extended universality hypothesis was confirmed by results of field measurements, LES and DNS simulations (see e.g. [2-4]), several questions remained unexplained. These are now answered by the new model including the reasons of the universality of the functional form of the HOMs, the significant scatter of the values of the coefficients and the source of the magic of the linear interpolation. Finally, the closures 61 predicted by the model are tested against measurements and LES data. Some of the other issues of CBL turbulence, e.g. familiar kurtosis-skewness relationships and relation of area coverage parameters of plumes (so called filling factors) with HOM will be discussed also

Physics-guided machine learning for turbulence closure and reduced-order modeling

A recent advance in scientific machine learning has started to show promising results in fluid mechanics. Despite their early success, the application of data-driven methods to turbulent flow simulation is non-trivial due to underlying highly nonlinear multiscale interactions. Here we present novel physics-guided machine learning (PGML) approaches for turbulence closure model discovery and model order reduction of complex multiscale systems. Our turbulence closure model discovery approach is based on exploiting big data without relying on underlying turbulence physics and learning from physical constraints. Specifically, we propose a frame invariant neural network model that can incorporate physical symmetries as inductive biases and illustrates its stable performance in the coarse-grid simulation without any kind of post-processing of the predicted subgrid-scale closure model. The frame invariant SGS model guarantees desired physical constraints without the need for any regularization terms and ultimately generalizes to different initial conditions and Reynolds numbers. To achieve data-efficient training and improved generalization, we propose a concatenated neural network with an uncertainty quantification mechanism that leverages information from hierarchies of models. The concatenated neural network is based on embedding information from cheap to evaluate low-fidelity approximations into the certain hidden layer of the neural network both during training and deployment. This framework is demonstrated for a range of problems, including turbulent boundary layer reconstruction, and reduced-order modeling of the vortex merging process. Furthermore, we investigate the seamless integration of sparse and noisy observations into non-intrusive reduced-order models, and hybrid models where the dynamical core of the system is modeled using the known governing equations, and the subgrid-scale processes are modeled using a deep learning model. To summarize, this work builds a bridge between extensive physics-based theories and data-driven modeling paradigms and paves the way for using hybrid physics-informed learning algorithms to generate predictive technologies for turbulent fluid flows