Linear Memory Networks

Recurrent neural networks can learn complex transduction problems that require maintaining and actively exploiting a memory of their inputs. Such models traditionally consider memory and input-output functionalities indissolubly entangled. We introduce a novel recurrent architecture based on the conceptual separation between the functional input-output transformation and the memory mechanism, showing how they can be implemented through different neural components. By building on such conceptualization, we introduce the Linear Memory Network, a recurrent model comprising a feedforward neural network, realizing the non-linear functional transformation, and a linear autoencoder for sequences, implementing the memory component. The resulting architecture can be efficiently trained by building on closed-form solutions to linear optimization problems. Further, by exploiting equivalence results between feedforward and recurrent neural networks we devise a pretraining schema for the proposed architecture. Experiments on polyphonic music datasets show competitive results against gated recurrent networks and other state of the art models

Equivalence results between feedforward and recurrent neural networks for sequences

In the context of sequence processing, we study the relationship between single-layer feedforward neural networks,that have simultaneous access to all items composing a sequence, and single-layer recurrent neural networks which access information one step at a time.We treat both linear and nonlinear networks, describing a constructive procedure, based on linear autoencoders for sequences, that given a feedforward neural network shows how to define a recurrent neural network that implements the same function in time. Upper bounds on the required number of hidden units for the recurrent network as a function of some features of the feedforward network are given. By separating the functional from the memory component, the proposed procedure suggests new efficient learning as well as interpretation procedures for recurrent neural networks

Dynamic data driven investigation of petrophysical and geomechanical properties for reservoir formation evaluation

Petrophysical and geomechanical properties of the formation such as Young’s modulus, bulk modulus, shear modulus, Poisson’s ratio, and porosity provide characteristic description of the hydrocarbon reservoir. It is well-established that static geomechanical properties are good representatives of reservoir formations; however, they are non-continuous along the wellbore, expensive and determining these properties may lead to formation damage. Dynamic geomechanical formation properties from acoustic measurements offer a continuous and non-destructive means to provide a characteristic description of the reservoir formation. In the absence of reliable acoustic measurements of the formation, such as sonic logs, the estimation of the dynamic geomechanical properties becomes challenging. Several techniques like empirical, analytical and intelligent systems have been used to approximate the property estimates. These techniques can also be used to approximate acoustic measurements thus enable dynamic estimation of geomechanical properties. This study intends to explore methodologies and models to dynamically estimate geomechanical properties in the absence of some or all acoustic measurements of the formation. The present work focused on developing empirical and intelligent systems like artificial neural networks (ANN), Gaussian processes (GP), and recurrent neural networks (RNN) to determine the dynamic geomechanical properties. The developed models serve as a cost-effective, reliable, efficient, and robust methods, offering dyanmic geomechanical analysis of the formation. This thesis has five main contributions: (a) a new data-driven empirical model of estimating static Young’s modulus from dynamic Young’s modulus, (b) a new data-driven ANN model for sonic well log prediction, (c) a new data-driven GP model for shear wave transit time prediction, (d) a new dynamic data-driven RNN model for sonic well log reproduction, and (e) an assessment on the ANN as a reliable sonic logging tool