1,112 research outputs found

    Deep learning for time series classification: a review

    Get PDF
    Time Series Classification (TSC) is an important and challenging problem in data mining. With the increase of time series data availability, hundreds of TSC algorithms have been proposed. Among these methods, only a few have considered Deep Neural Networks (DNNs) to perform this task. This is surprising as deep learning has seen very successful applications in the last years. DNNs have indeed revolutionized the field of computer vision especially with the advent of novel deeper architectures such as Residual and Convolutional Neural Networks. Apart from images, sequential data such as text and audio can also be processed with DNNs to reach state-of-the-art performance for document classification and speech recognition. In this article, we study the current state-of-the-art performance of deep learning algorithms for TSC by presenting an empirical study of the most recent DNN architectures for TSC. We give an overview of the most successful deep learning applications in various time series domains under a unified taxonomy of DNNs for TSC. We also provide an open source deep learning framework to the TSC community where we implemented each of the compared approaches and evaluated them on a univariate TSC benchmark (the UCR/UEA archive) and 12 multivariate time series datasets. By training 8,730 deep learning models on 97 time series datasets, we propose the most exhaustive study of DNNs for TSC to date.Comment: Accepted at Data Mining and Knowledge Discover

    Selective Trajectory Memory Network andits application in Vehicle DestinationPrediction

    Get PDF
    학위논문 (석사)-- 서울대학교 대학원 : 공과대학 산업공학과, 2019. 2. Cho, Sungzoon.Predicting efficiently the final destinations of moving vehicles can be of significant usefulness for several applications. Many probabilistic methods have been developed to address it but often include heavy feature engineering and do not generalize well to new datasets. To face these limitations, Deep-Learning models present the advantage of automating processing steps and can therefore be easily adapted to new input data. De Brébisson et al. proposed clustering based deep-learning approaches to solve it in the specific case of the prediction of Taxis destinations with remarkable performances, alongside with a proposition of a novel architecture inspired by Memory-Networks used in Natural Language Processing, and requiring no preliminary clustering. A large room for improvement was however left for the latter approach : the necessity of a relevant selection function retrieving historical trajectories similar to partial trips to predict was indeed outlined by the authors. In this work we propose to use the Segment-Path distance, introduced by Besse et al. in former works on trajectory clustering, to come up with an improved architecture of this memory model. A review of several Memory Networks architecture and their applications in time-series prediction is provided to give an overview of the different structural alternatives existing for the design of our model architecture. Finally, our model is confronted to individual car data and we propose a personalized user-by-user prediction of destinations. We discuss the suitability and limits of the type of model in this specific problem and conclude that the promising obtained results are penalized by infrequent destinations cases inducing noise whose effect could be reduced by turning our approach into a classification problem.Abstract i Contents List of Tables vi List of Figures viii Chapter 1 Introduction 1 1.1 Motivations, background . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Problem Description : destination forecasting problem . . . . . . . . 2 1.2.1 General context . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.2 Specific problem tackled . . . . . . . . . . . . . . . . . . . . . 2 1.3 Existing models and methods . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Research Motivation and Contributions . . . . . . . . . . . . . . . . 6 1.5 Organization of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . 7 Chapter 2 Related works 8 2.1 Artificial neural network models for trajectory prediction . . . . . . 8 2.1.1 Encoding and clustering approach . . . . . . . . . . . . . . . 8 2.1.2 "Memory network" model for taxi trajectory prediction . . . 11 2.2 Memory networks and applications . . . . . . . . . . . . . . . . . . . 13 2.2.1 MemNN models . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2.2 End-to-end memory networks (MemN2N) . . . . . . . . . . . 16 2.2.3 Memory networks for multi-dimensional time-series forecasting (MTNnet) . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.3 Analogies and comparisons between the memory models introduced . 19 2.4 Distances measures for vehicle trajectories . . . . . . . . . . . . . . . 22 2.4.1 Segment-Path Distance (SPD) . . . . . . . . . . . . . . . . . 23 2.5 Personalized predictions on car manufacturer data . . . . . . . . . . 26 2.5.1 Problem approach and redefinition . . . . . . . . . . . . . . . 26 2.5.2 Method and model . . . . . . . . . . . . . . . . . . . . . . . . 27 Chapter 3 Proposed Model 28 3.1 Overall architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.2 Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.3 Memory storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.4 Trajectory encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.4.1 Encoding architecture . . . . . . . . . . . . . . . . . . . . . . 30 3.4.2 Metadata and embedding . . . . . . . . . . . . . . . . . . . . 31 3.4.3 Distinctions between encoders, weight-sharing . . . . . . . . . 31 3.5 Memory selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.5.1 Attention mechanism . . . . . . . . . . . . . . . . . . . . . . 32 3.5.2 Data used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.6 Query-memory association . . . . . . . . . . . . . . . . . . . . . . . . 33 3.7 Final prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Chapter 4 Experiments 35 4.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.2 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.2.1 Variability and predictability . . . . . . . . . . . . . . . . . . 36 4.2.2 Considered vehicles . . . . . . . . . . . . . . . . . . . . . . . . 37 4.3 Experimental settings . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.3.1 Training and testing set . . . . . . . . . . . . . . . . . . . . . 39 4.3.2 Test methodology and parameters . . . . . . . . . . . . . . . 40 4.3.3 Baseline model : simple encoding . . . . . . . . . . . . . . . . 42 4.4 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.4.1 General results . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.4.2 Factors of influence on models performances . . . . . . . . . . 45 4.4.3 Case studies : 5 example vehicles analysis . . . . . . . . . . . 49 4.4.4 Baseline model . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.5 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Chapter 5 Conclusion 56 5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.2 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Bibliography 58 감사의 글 62Maste

    Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction

    Get PDF
    This paper reports the results of the NN3 competition, which is a replication of the M3 competition with an extension of the competition towards neural network (NN) and computational intelligence (CI) methods, in order to assess what progress has been made in the 10 years since the M3 competition. Two masked subsets of the M3 monthly industry data, containing 111 and 11 empirical time series respectively, were chosen, controlling for multiple data conditions of time series length (short/long), data patterns (seasonal/non-seasonal) and forecasting horizons (short/medium/long). The relative forecasting accuracy was assessed using the metrics from the M3, together with later extensions of scaled measures, and non-parametric statistical tests. The NN3 competition attracted 59 submissions from NN, CI and statistics, making it the largest CI competition on time series data. Its main findings include: (a) only one NN outperformed the damped trend using the sMAPE, but more contenders outperformed the AutomatANN of the M3; (b) ensembles of CI approaches performed very well, better than combinations of statistical methods; (c) a novel, complex statistical method outperformed all statistical and Cl benchmarks; and (d) for the most difficult subset of short and seasonal series, a methodology employing echo state neural networks outperformed all others. The NN3 results highlight the ability of NN to handle complex data, including short and seasonal time series, beyond prior expectations, and thus identify multiple avenues for future research. (C) 2011 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved

    American Option Pricing using Self-Attention GRU and Shapley Value Interpretation

    Full text link
    Options, serving as a crucial financial instrument, are used by investors to manage and mitigate their investment risks within the securities market. Precisely predicting the present price of an option enables investors to make informed and efficient decisions. In this paper, we propose a machine learning method for forecasting the prices of SPY (ETF) option based on gated recurrent unit (GRU) and self-attention mechanism. We first partitioned the raw dataset into 15 subsets according to moneyness and days to maturity criteria. For each subset, we matched the corresponding U.S. government bond rates and Implied Volatility Indices. This segmentation allows for a more insightful exploration of the impacts of risk-free rates and underlying volatility on option pricing. Next, we built four different machine learning models, including multilayer perceptron (MLP), long short-term memory (LSTM), self-attention LSTM, and self-attention GRU in comparison to the traditional binomial model. The empirical result shows that self-attention GRU with historical data outperforms other models due to its ability to capture complex temporal dependencies and leverage the contextual information embedded in the historical data. Finally, in order to unveil the "black box" of artificial intelligence, we employed the SHapley Additive exPlanations (SHAP) method to interpret and analyze the prediction results of the self-attention GRU model with historical data. This provides insights into the significance and contributions of different input features on the pricing of American-style options.Comment: Working pape
    corecore