2 research outputs found
Investigating Recurrent Neural Network Memory Structures using Neuro-Evolution
This paper presents a new algorithm, Evolutionary eXploration of Augmenting
Memory Models (EXAMM), which is capable of evolving recurrent neural networks
(RNNs) using a wide variety of memory structures, such as Delta-RNN, GRU, LSTM,
MGU and UGRNN cells. EXAMM evolved RNNs to perform prediction of large-scale,
real world time series data from the aviation and power industries. These data
sets consist of very long time series (thousands of readings), each with a
large number of potentially correlated and dependent parameters. Four different
parameters were selected for prediction and EXAMM runs were performed using
each memory cell type alone, each cell type with feed forward nodes, and with
all possible memory cell types. Evolved RNN performance was measured using
repeated k-fold cross validation, resulting in 1210 EXAMM runs which evolved
2,420,000 RNNs in 12,100 CPU hours on a high performance computing cluster.
Generalization of the evolved RNNs was examined statistically, providing
interesting findings that can help refine the RNN memory cell design as well as
inform future neuro-evolution algorithms development.Comment: Some corrections to language, title fi
Nonlinear Transform Coding
We review a class of methods that can be collected under the name nonlinear
transform coding (NTC), which over the past few years have become competitive
with the best linear transform codecs for images, and have superseded them in
terms of rate--distortion performance under established perceptual quality
metrics such as MS-SSIM. We assess the empirical rate--distortion performance
of NTC with the help of simple example sources, for which the optimal
performance of a vector quantizer is easier to estimate than with natural data
sources. To this end, we introduce a novel variant of entropy-constrained
vector quantization. We provide an analysis of various forms of stochastic
optimization techniques for NTC models; review architectures of transforms
based on artificial neural networks, as well as learned entropy models; and
provide a direct comparison of a number of methods to parameterize the
rate--distortion trade-off of nonlinear transforms, introducing a new one