Learning to Detect Violent Videos using Convolutional Long Short-Term Memory

Developing a technique for the automatic analysis of surveillance videos in order to identify the presence of violence is of broad interest. In this work, we propose a deep neural network for the purpose of recognizing violent videos. A convolutional neural network is used to extract frame level features from a video. The frame level features are then aggregated using a variant of the long short term memory that uses convolutional gates. The convolutional neural network along with the convolutional long short term memory is capable of capturing localized spatio-temporal features which enables the analysis of local motion taking place in the video. We also propose to use adjacent frame differences as the input to the model thereby forcing it to encode the changes occurring in the video. The performance of the proposed feature extraction pipeline is evaluated on three standard benchmark datasets in terms of recognition accuracy. Comparison of the results obtained with the state of the art techniques revealed the promising capability of the proposed method in recognizing violent videos.Comment: Accepted in International Conference on Advanced Video and Signal based Surveillance(AVSS 2017

Anomaly Detection Using Predictive Convolutional Long Short-Term Memory Units

Automating the segmentation of anomalous activities within long video sequences is complicated by the ambiguity of how such events are defined. This thesis approaches the problem by learning generative models with which meaningful sequences can be identified in videos using limited supervision. We propose two types of end-to-end trainable Convolutional Long Short-Term Memory (Conv-LSTM) networks that are able to predict the subsequent video sequence from a given input. The first is an encoder decoder based model that learns spatio-temporal features from stacked non-overlapping image patches, and the second is an autoencoder based model that utilizes max-pooling layers to learn an abstraction of the entire image. The networks learn to model “normal” activities from usual events. Regularity scores are derived from the reconstruction errors of a set of predictions with abnormal video sequences yielding lower regularity scores, as they diverge further from the actual sequence with time. The models utilize a composite structure and examine the effects of “conditioning” to learn more meaningful representations. The best model is chosen based on the reconstruction and prediction accuracies. The Conv-LSTM models are evaluated both qualitatively and quantitatively, demonstrating competitive results on multiple anomaly detection datasets. Conv-LSTM units are shown to provide competitive results for modeling and predicting learned events when compared to state-to-the-art methods

Reconstructing Secondary Data based on Air Quality, Meteorological and Traffic Data Considering Spatiotemporal Components

This paper introduces the reconstructed dataset along with procedures to implement air quality prediction, which consists of air quality, meteorological and traffic data over time, and their monitoring stations and measurement points. Given the fact that those monitoring stations and measurement points are located in different places, it is important to incorporate their time series data into a spatiotemporal dimension. The output can be used as input for various predictive analyses, in particular, we used the reconstructed dataset as input for grid-based (Convolutional Long Short-Term Memory and Bidirectional Convolutional Long Short-Term Memory) and graph-based (Attention Temporal Graph Convolutional Network) machine learning algorithms. The raw dataset is obtained from the Open Data portal of the Madrid City Council

Convolutional Long Short-Term Memory (C-LSTM) For Multi Product Prediction

The retail company PT Terang Abadi Raya has a solid commitment to supporting distributors of LED lights and electrical equipment who have joined them, helping to spread their products widely in various regions. To face increasingly intense market competition, it is essential to produce high-quality products to win the competition and meet consumer demands. To achieve this, efficient production planning is necessary. The Convolutional Long Short-Term Memory (C-LSTM) method is used in this study to forecast product sales at PT Terang Abadi Raya. The research results show that C-LSTM has the potential to predict sales effectively. Evaluation is conducted using Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE). The calculations reveal that the smallest values are obtained at epoch 10, with an MAE of 0.1051 and a MAPE of 22% in the testing data. For the cable data, the smallest values are found at epoch 100, with an MAE of 0.0602 and a MAPE of 44% in the testing data. The Long Short-Term Memory (LSTM) method with ten neurons produces the most minor errors during training

Temporal Performance Prediction for Deep Convolutional Long Short-Term Memory Networks

Quantifying predictive uncertainty of deep semantic segmentation networks is essential in safety-critical tasks. In applications like autonomous driving, where video data is available, convolutional long short-term memory networks are capable of not only providing semantic segmentations but also predicting the segmentations of the next timesteps. These models use cell states to broadcast information from previous data by taking a time series of inputs to predict one or even further steps into the future. We present a temporal postprocessing method which estimates the prediction performance of convolutional long short-term memory networks by either predicting the intersection over union of predicted and ground truth segments or classifying between intersection over union being equal to zero or greater than zero. To this end, we create temporal cell state-based input metrics per segment and investigate different models for the estimation of the predictive quality based on these metrics. We further study the influence of the number of considered cell states for the proposed metrics.Comment: 14 pages, 4 figures, this work is related to arXiv:1811.00648 and arXiv:1911.0507

A Convolutional Long Short-Term Memory Neural Network Based Prediction Model

In recent years, the market demand for online car-hailing service has expanded dramatically. To satisfy the daily travel needs, it is important to predict the supply and demand of online car-hailing in an accurate manner, and make active scheduling based on the predicted gap between supply and demand. This paper puts forward a novel supply and demand prediction model for online carhailing, which combines the merits of convolutional neural network (CNN) and long short-term memory (LSTM). The proposed model was named convolutional LSTM (C-LSTM). Next, the original data on online car-hailing were processed, and the key features that affect the supply and demand prediction were extracted. After that, the C-LSTM was optimized by the AdaBound algorithm during the training process. Finally, the superiority of the C-LSTM in predicting online car-hailing supply and demand was proved through contrastive experiments

Short-Term Forecasting of Passenger Demand under On-Demand Ride Services: A Spatio-Temporal Deep Learning Approach

Short-term passenger demand forecasting is of great importance to the on-demand ride service platform, which can incentivize vacant cars moving from over-supply regions to over-demand regions. The spatial dependences, temporal dependences, and exogenous dependences need to be considered simultaneously, however, which makes short-term passenger demand forecasting challenging. We propose a novel deep learning (DL) approach, named the fusion convolutional long short-term memory network (FCL-Net), to address these three dependences within one end-to-end learning architecture. The model is stacked and fused by multiple convolutional long short-term memory (LSTM) layers, standard LSTM layers, and convolutional layers. The fusion of convolutional techniques and the LSTM network enables the proposed DL approach to better capture the spatio-temporal characteristics and correlations of explanatory variables. A tailored spatially aggregated random forest is employed to rank the importance of the explanatory variables. The ranking is then used for feature selection. The proposed DL approach is applied to the short-term forecasting of passenger demand under an on-demand ride service platform in Hangzhou, China. Experimental results, validated on real-world data provided by DiDi Chuxing, show that the FCL-Net achieves better predictive performance than traditional approaches including both classical time-series prediction models and neural network based algorithms (e.g., artificial neural network and LSTM). This paper is one of the first DL studies to forecast the short-term passenger demand of an on-demand ride service platform by examining the spatio-temporal correlations.Comment: 39 pages, 10 figure