3 research outputs found
Fast-Slow Recurrent Neural Networks
Processing sequential data of variable length is a major challenge in a wide
range of applications, such as speech recognition, language modeling,
generative image modeling and machine translation. Here, we address this
challenge by proposing a novel recurrent neural network (RNN) architecture, the
Fast-Slow RNN (FS-RNN). The FS-RNN incorporates the strengths of both
multiscale RNNs and deep transition RNNs as it processes sequential data on
different timescales and learns complex transition functions from one time step
to the next. We evaluate the FS-RNN on two character level language modeling
data sets, Penn Treebank and Hutter Prize Wikipedia, where we improve state of
the art results to and bits-per-character (BPC), respectively. In
addition, an ensemble of two FS-RNNs achieves BPC on Hutter Prize
Wikipedia outperforming the best known compression algorithm with respect to
the BPC measure. We also present an empirical investigation of the learning and
network dynamics of the FS-RNN, which explains the improved performance
compared to other RNN architectures. Our approach is general as any kind of RNN
cell is a possible building block for the FS-RNN architecture, and thus can be
flexibly applied to different tasks.Comment: Corrected minor typos in Figure 1 and Zoneout citatio
Hierarchical Attention-Based Recurrent Highway Networks for Time Series Prediction
Time series prediction has been studied in a variety of domains. However, it
is still challenging to predict future series given historical observations and
past exogenous data. Existing methods either fail to consider the interactions
among different components of exogenous variables which may affect the
prediction accuracy, or cannot model the correlations between exogenous data
and target data. Besides, the inherent temporal dynamics of exogenous data are
also related to the target series prediction, and thus should be considered as
well. To address these issues, we propose an end-to-end deep learning model,
i.e., Hierarchical attention-based Recurrent Highway Network (HRHN), which
incorporates spatio-temporal feature extraction of exogenous variables and
temporal dynamics modeling of target variables into a single framework.
Moreover, by introducing the hierarchical attention mechanism, HRHN can
adaptively select the relevant exogenous features in different semantic levels.
We carry out comprehensive empirical evaluations with various methods over
several datasets, and show that HRHN outperforms the state of the arts in time
series prediction, especially in capturing sudden changes and sudden
oscillations of time series
Real time expert system for anomaly detection of aerators based on computer vision technology and existing surveillance cameras
Aerators are essential and crucial auxiliary devices in intensive culture,
especially in industrial culture in China. The traditional methods cannot
accurately detect abnormal condition of aerators in time. Surveillance cameras
are widely used as visual perception modules of the Internet of Things, and
then using these widely existing surveillance cameras to realize real-time
anomaly detection of aerators is a cost-free and easy-to-promote method.
However, it is difficult to develop such an expert system due to some technical
and applied challenges, e.g., illumination, occlusion, complex background, etc.
To tackle these aforementioned challenges, we propose a real-time expert system
based on computer vision technology and existing surveillance cameras for
anomaly detection of aerators, which consists of two modules, i.e., object
region detection and working state detection. First, it is difficult to detect
the working state for some small object regions in whole images, and the time
complexity of global feature comparison is also high, so we present an object
region detection method based on the region proposal idea. Moreover, we propose
a novel algorithm called reference frame Kanade-Lucas-Tomasi (RF-KLT) algorithm
for motion feature extraction in fixed regions. Then, we present a dimension
reduction method of time series for establishing a feature dataset with obvious
boundaries between classes. Finally, we use machine learning algorithms to
build the feature classifier. The experimental results in both the actual video
dataset and the augmented video dataset show that the accuracy for detecting
object region and working state of aerators is 100% and 99.9% respectively, and
the detection speed is 77-333 frames per second (FPS) according to the
different types of surveillance cameras.Comment: 17 figure