SOM-VAE: Interpretable Discrete Representation Learning on Time Series

High-dimensional time series are common in many domains. Since human cognition is not optimized to work well in high-dimensional spaces, these areas could benefit from interpretable low-dimensional representations. However, most representation learning algorithms for time series data are difficult to interpret. This is due to non-intuitive mappings from data features to salient properties of the representation and non-smoothness over time. To address this problem, we propose a new representation learning framework building on ideas from interpretable discrete dimensionality reduction and deep generative modeling. This framework allows us to learn discrete representations of time series, which give rise to smooth and interpretable embeddings with superior clustering performance. We introduce a new way to overcome the non-differentiability in discrete representation learning and present a gradient-based version of the traditional self-organizing map algorithm that is more performant than the original. Furthermore, to allow for a probabilistic interpretation of our method, we integrate a Markov model in the representation space. This model uncovers the temporal transition structure, improves clustering performance even further and provides additional explanatory insights as well as a natural representation of uncertainty. We evaluate our model in terms of clustering performance and interpretability on static (Fashion-)MNIST data, a time series of linearly interpolated (Fashion-)MNIST images, a chaotic Lorenz attractor system with two macro states, as well as on a challenging real world medical time series application on the eICU data set. Our learned representations compare favorably with competitor methods and facilitate downstream tasks on the real world data.Comment: Accepted for publication at the Seventh International Conference on Learning Representations (ICLR 2019

Modeling Financial Time Series with Artificial Neural Networks

Financial time series convey the decisions and actions of a population of human actors over time. Econometric and regressive models have been developed in the past decades for analyzing these time series. More recently, biologically inspired artificial neural network models have been shown to overcome some of the main challenges of traditional techniques by better exploiting the non-linear, non-stationary, and oscillatory nature of noisy, chaotic human interactions. This review paper explores the options, benefits, and weaknesses of the various forms of artificial neural networks as compared with regression techniques in the field of financial time series analysis.CELEST, a National Science Foundation Science of Learning Center (SBE-0354378); SyNAPSE program of the Defense Advanced Research Project Agency (HR001109-03-0001

Neural Networks for Complex Data

Artificial neural networks are simple and efficient machine learning tools. Defined originally in the traditional setting of simple vector data, neural network models have evolved to address more and more difficulties of complex real world problems, ranging from time evolving data to sophisticated data structures such as graphs and functions. This paper summarizes advances on those themes from the last decade, with a focus on results obtained by members of the SAMM team of Universit\'e Paris

Analysis of Data Clusters Obtained by Self-Organizing Methods

The self-organizing methods were used for the investigation of financial market. As an example we consider data time-series of Dow Jones index for the years 2002-2003 (R. Mantegna, cond-mat/9802256). In order to reveal new structures in stock market behavior of the companies drawing up Dow Jones index we apply SOM (Self-Organizing Maps) and GMDH (Group Method of Data Handling) algorithms. Using SOM techniques we obtain SOM-maps that establish a new relationship in market structure. Analysis of the obtained clusters was made by GMDH.Comment: 10 pages, 4 figure

Learning the Pseudoinverse Solution to Network Weights

The last decade has seen the parallel emergence in computational neuroscience and machine learning of neural network structures which spread the input signal randomly to a higher dimensional space; perform a nonlinear activation; and then solve for a regression or classification output by means of a mathematical pseudoinverse operation. In the field of neuromorphic engineering, these methods are increasingly popular for synthesizing biologically plausible neural networks, but the "learning method" - computation of the pseudoinverse by singular value decomposition - is problematic both for biological plausibility and because it is not an online or an adaptive method. We present an online or incremental method of computing the pseudoinverse, which we argue is biologically plausible as a learning method, and which can be made adaptable for non-stationary data streams. The method is significantly more memory-efficient than the conventional computation of pseudoinverses by singular value decomposition.Comment: 13 pages, 3 figures; in submission to Neural Network

Forecasting of financial data: a novel fuzzy logic neural network based on error-correction concept and statistics

First, this paper investigates the effect of good and bad news on volatility in the BUX return time series using asymmetric ARCH models. Then, the accuracy of forecasting models based on statistical (stochastic), machine learning methods, and soft/granular RBF network is investigated. To forecast the high-frequency financial data, we apply statistical ARMA and asymmetric GARCH-class models. A novel RBF network architecture is proposed based on incorporation of an error-correction mechanism, which improves forecasting ability of feed-forward neural networks. These proposed modelling approaches and SVM models are applied to predict the high-frequency time series of the BUX stock index. We found that it is possible to enhance forecast accuracy and achieve significant risk reduction in managerial decision making by applying intelligent forecasting models based on latest information technologies. On the other hand, we showed that statistical GARCH-class models can identify the presence of leverage effects, and react to the good and bad news.Web of Science421049

Time Series Forecasting: Obtaining Long Term Trends with Self-Organizing Maps

à la suite de la conférence ANNPR, Florence 2003International audienceKohonen self-organisation maps are a well know classification tool, commonly used in a wide variety of problems, but with limited applications in time series forecasting context. In this paper, we propose a forecasting method specifically designed for multi-dimensional long-term trends prediction, with a double application of the Kohonen algorithm. Practical applications of the method are also presented

SOME REMARKS ON THE SELF-ORGANIZING FEATURE MAPS

Joint Research on Environmental Science and Technology for the Eart

Integrated characterisation of mud-rich overburden sediment sequences using limited log and seismic data: Application to seal risk

Muds and mudstones are the most abundant sediments in sedimentary basins and can control fluid migration and pressure. In petroleum systems, they can also act as source, reservoir or seal rocks. More recently, the sealing properties of mudstones have been used for nuclear waste storage and geological CO2 sequestration. Despite the growing importance of mudstones, their geological modelling is poorly understood and clear quantitative studies are needed to address 3D lithology and flow properties distribution within these sediments. The key issues in this respect are the high degree of heterogeneity in mudstones and the alteration of lithology and flow properties with time and depth. In addition, there are often very limited field data (log and seismic), with lower quality within these sediments, which makes the common geostatistical modelling practices ineffective. In this study we assess/capture quantitatively the flow-important characteristics of heterogeneous mud-rich sequences based on limited conventional log and post-stack seismic data in a deep offshore West African case study. Additionally, we develop a practical technique of log-seismic integration at the cross-well scale to translate 3D seismic attributes into lithology probabilities. The final products are probabilistic multiattribute transforms at different resolutions which allow prediction of lithologies away from wells while keeping the important sub-seismic stratigraphic and structural flow features. As a key result, we introduced a seismically-driven risk attribute (so-called Seal Risk Factor "SRF") which showed robust correspondence to the lithologies within the seismic volume. High seismic SRFs were often a good approximation for volumes containing a higher percentage of coarser-grained and distorted sediments, and vice versa. We believe that this is the first attempt at quantitative, integrated characterisation of mud-rich overburden sediment sequences using log and seismic data. Its application on modern seismic surveys can save days of processing/mapping time and can reduce exploration risk by basing decisions on seal texture and lithology probabilities