68,329 research outputs found
On the relationship between predictive coding and backpropagation
Artificial neural networks are often interpreted as abstract models of
biological neuronal networks, but they are typically trained using the
biologically unrealistic backpropagation algorithm and its variants. Predictive
coding has been offered as a potentially more biologically realistic
alternative to backpropagation for training neural networks. In this
manuscript, I review and extend recent work on the mathematical relationship
between predictive coding and backpropagation for training feedforward
artificial neural networks on supervised learning tasks. I discuss some
implications of these results for the interpretation of predictive coding and
deep neural networks as models of biological learning and I describe a
repository of functions, Torch2PC, for performing predictive coding with
PyTorch neural network models
Deep Predictive Coding Neural Network for RF Anomaly Detection in Wireless Networks
Intrusion detection has become one of the most critical tasks in a wireless
network to prevent service outages that can take long to fix. The sheer variety
of anomalous events necessitates adopting cognitive anomaly detection methods
instead of the traditional signature-based detection techniques. This paper
proposes an anomaly detection methodology for wireless systems that is based on
monitoring and analyzing radio frequency (RF) spectrum activities. Our
detection technique leverages an existing solution for the video prediction
problem, and uses it on image sequences generated from monitoring the wireless
spectrum. The deep predictive coding network is trained with images
corresponding to the normal behavior of the system, and whenever there is an
anomaly, its detection is triggered by the deviation between the actual and
predicted behavior. For our analysis, we use the images generated from the
time-frequency spectrograms and spectral correlation functions of the received
RF signal. We test our technique on a dataset which contains anomalies such as
jamming, chirping of transmitters, spectrum hijacking, and node failure, and
evaluate its performance using standard classifier metrics: detection ratio,
and false alarm rate. Simulation results demonstrate that the proposed
methodology effectively detects many unforeseen anomalous events in real time.
We discuss the applications, which encompass industrial IoT, autonomous vehicle
control and mission-critical communications services.Comment: 7 pages, 7 figures, Communications Workshop ICC'1
Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy
Predictive coding provides a computational paradigm for modeling perceptual processing as the construction of representations accounting for causes of sensory inputs. Here, we developed a scalable, deep network architecture for predictive coding that is trained using a gated Hebbian learning rule and mimics the feedforward and feedback connectivity of the cortex. After training on image datasets, the models formed latent representations in higher areas that allowed reconstruction of the original images. We analyzed low- and high-level properties such as orientation selectivity, object selectivity and sparseness of neuronal populations in the model. As reported experimentally, image selectivity increased systematically across ascending areas in the model hierarchy. Depending on the strength of regularization factors, sparseness also increased from lower to higher areas. The results suggest a rationale as to why experimental results on sparseness across the cortical hierarchy have been inconsistent. Finally, representations for different object classes became more distinguishable from lower to higher areas. Thus, deep neural networks trained using a gated Hebbian formulation of predictive coding can reproduce several properties associated with neuronal responses along the visual cortical hierarchy
- …