Benchmarking Deep Learning Architectures for Predicting Readmission to the ICU and Describing Patients-at-Risk

Objective: To compare different deep learning architectures for predicting the risk of readmission within 30 days of discharge from the intensive care unit (ICU). The interpretability of attention-based models is leveraged to describe patients-at-risk. Methods: Several deep learning architectures making use of attention mechanisms, recurrent layers, neural ordinary differential equations (ODEs), and medical concept embeddings with time-aware attention were trained using publicly available electronic medical record data (MIMIC-III) associated with 45,298 ICU stays for 33,150 patients. Bayesian inference was used to compute the posterior over weights of an attention-based model. Odds ratios associated with an increased risk of readmission were computed for static variables. Diagnoses, procedures, medications, and vital signs were ranked according to the associated risk of readmission. Results: A recurrent neural network, with time dynamics of code embeddings computed by neural ODEs, achieved the highest average precision of 0.331 (AUROC: 0.739, F1-Score: 0.372). Predictive accuracy was comparable across neural network architectures. Groups of patients at risk included those suffering from infectious complications, with chronic or progressive conditions, and for whom standard medical care was not suitable. Conclusions: Attention-based networks may be preferable to recurrent networks if an interpretable model is required, at only marginal cost in predictive accuracy

Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems

Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than standard feed-forward neural networks, recurrent neural networks, or convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering. In this work, we review such methods as well as their extensions for parametric studies and for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications

Deep Learning algorithms for solving high dimensional nonlinear Backward Stochastic Differential Equations

We study deep learning-based schemes for solving high dimensional nonlinear backward stochastic differential equations (BSDEs). First we show how to improve the performances of the proposed scheme in [W. E and J. Han and A. Jentzen, Commun. Math. Stat., 5 (2017), pp.349-380] regarding computational time by using a single neural network architecture instead of the stacked deep neural networks. Furthermore, those schemes can be stuck in poor local minima or diverges, especially for a complex solution structure and longer terminal time. To solve this problem, we investigate to reformulate the problem by including local losses and exploit the Long Short Term Memory (LSTM) networks which are a type of recurrent neural networks (RNN). Finally, in order to study numerical convergence and thus illustrate the improved performances with the proposed methods, we provide numerical results for several 100-dimensional nonlinear BSDEs including nonlinear pricing problems in finance.Comment: 21 pages, 5 figures, 16 table

Artificial intelligence methods for oil and gas reservoir development: Current progresses and perspectives

Artificial neural networks have been widely applied in reservoir engineering. As a powerful tool, it changes the way to find solutions in reservoir simulation profoundly. Deep learning networks exhibit robust learning capabilities, enabling them not only to detect patterns in data, but also uncover underlying physical principles, incorporate prior knowledge of physics, and solve complex partial differential equations. This work presents the latest research advancements in the field of petroleum reservoir engineering, covering three key research directions based on artificial neural networks: data-driven methods, physics driven artificial neural network partial differential equation solver, and data and physics jointly driven methods. In addition, a wide range of neural network architectures are reviewed, including fully connected neural networks, convolutional neural networks, recurrent neural networks, and so on. The basic principles of these methods and their limitations in practical applications are also outlined. The future trends of artificial intelligence methods for oil and gas reservoir development are further discussed. The large language models are the most advanced neural networks so far, it is expected to be applied in reservoir simulation to predict the development performance.Document Type: PerspectiveCited as: Xue, L., Li, D., Dou, H. Artificial intelligence methods for oil and gas reservoir development: Current progresses and perspectives. Advances in Geo-Energy Research, 2023, 10(1): 65-70. https://doi.org/10.46690/ager.2023.10.0

Control of a modified double inverted pendulum using machine learning based model predictive control

Abstract: A machine learning-based controller (MLC) has been developed for a modified double inverted pendulum on a cart (MDIPC). First, the governing differential equations of the system are derived using the Lagrangian method. Then, a dataset is generated to train and test the machine learning-based models of the plant. Different types of machine learning models such as artificial neural networks (ANN), deep neural networks (DNN), long-short-term memory neural networks (LSTM), gated recurrent unit (GRU), and recurrent neural networks (RNN) are employed to capture the system’s dynamics. DNN and LSTM are selected due to their superior performance compared to other models. Finally, different variations of the Model Predictive Controller (MPC) are designed, and their performance is evaluated in terms of running time and tracking error. The proposed control methods are shown to have an advantage over the conventional nonlinear and linear model predictive control methods in simulation.Communication présentée lors du congrès international tenu conjointement par Canadian Society for Mechanical Engineering (CSME) et Computational Fluid Dynamics Society of Canada (CFD Canada), à l’Université de Sherbrooke (Québec), du 28 au 31 mai 2023