A VHDL model of a digi-neocognitron neural network for VLSI

Optical character recognition is useful in many aspects of business. However, the use of conventional computers to provide a solution to this problem has not been very effective. Over the past two decades, researchers have utilized artificial neural networks for optical character recognition with considerable success. One such neural network is the neocognitron, a real-valued, multi-layered hierarchical network that simulates the human visual system. The neocognitron was shown to have the capability for pattern recognition despite variations in size, shape or the presence of deformations from the trained patterns. Unfortunately, the neocognitron is an analog network which prevents it from taking full advantage of the many advances in VLSI technology. Major advances in VLSI technology have been in the digital medium. Therefore, it appears necessary to adapt the neocognitron to an efficient digital neural network if it is to be implemented in VLSI. Recent research has shown that through preprocessing approximations and definition of new model functions, the neocognitron is well suited for implementation in digital VLSI. This thesis uses this methodology to implement a large scale digital neocognitron model. The new model, the digi-neocognitron, uses supervised learning and is trained to recognize ten handwritten numerals with widths of one pixel. The development of the neocognitron and the digi-neocognitron software models, and a comparison of their performance will be discussed. This is followed by the development and simulation of the digital model using the VHSIC Hardware Description Language (VHDL). The VHDL model is used to demonstrate the functionality of the hardware model and to aid in its design. The model functions of the digi-neocognitron are then implemented and simulated for a 1.2 micrometers CMOS process

Further Parameters Estimation of Neocognitron Neural Network Modification with FFT Convolution

This paper presents further development of an improved version of the neocognitron algorithm introduced by Fukushima. Some comparisons of other symbol recognition methods based on the neocognitron neural network are also performed, which led to the proposal of several modifications — namely, layer dimension adjustment, threshold function and connection Gaussian kernel estimation. The width and height are taken into account independently in order to improve the recognition of patterns of slightly different dimensions. The learning and recognition calculations are performed as FFT convolutions in order to utilize external specialized computing system. Finally, more detailed results of the neocognitron performance evaluation are provided

How Does Our Visual System Achieve Shift and Size Invariance?

The question of shift and size invariance in the primate visual system is discussed. After a short review of the relevant neurobiology and psychophysics, a more detailed analysis of computational models is given. The two main types of networks considered are the dynamic routing circuit model and invariant feature networks, such as the neocognitron. Some specific open questions in context of these models are raised and possible solutions discussed

Deep Neural Networks - A Brief History

Introduction to deep neural networks and their history.Comment: 14 pages, 14 figure

Visualizing classification of natural video sequences using sparse, hierarchical models of cortex.

Recent work on hierarchical models of visual cortex has reported state-of-the-art accuracy on whole-scene labeling using natural still imagery. This raises the question of whether the reported accuracy may be due to the sophisticated, non-biological back-end supervised classifiers typically used (support vector machines) and/or the limited number of images used in these experiments. In particular, is the model classifying features from the object or the background? Previous work (Landecker, Brumby, et al., COSYNE 2010) proposed tracing the spatial support of a classifier’s decision back through a hierarchical cortical model to determine which parts of the image contributed to the classification, compared to the positions of objects in the scene. In this way, we can go beyond standard measures of accuracy to provide tools for visualizing and analyzing high-level object classification. We now describe new work exploring the extension of these ideas to detection of objects in video sequences of natural scenes

Empiricism without Magic: Transformational Abstraction in Deep Convolutional Neural Networks

In artificial intelligence, recent research has demonstrated the remarkable potential of Deep Convolutional Neural Networks (DCNNs), which seem to exceed state-of-the-art performance in new domains weekly, especially on the sorts of very difficult perceptual discrimination tasks that skeptics thought would remain beyond the reach of artificial intelligence. However, it has proven difficult to explain why DCNNs perform so well. In philosophy of mind, empiricists have long suggested that complex cognition is based on information derived from sensory experience, often appealing to a faculty of abstraction. Rationalists have frequently complained, however, that empiricists never adequately explained how this faculty of abstraction actually works. In this paper, I tie these two questions together, to the mutual benefit of both disciplines. I argue that the architectural features that distinguish DCNNs from earlier neural networks allow them to implement a form of hierarchical processing that I call “transformational abstraction”. Transformational abstraction iteratively converts sensory-based representations of category exemplars into new formats that are increasingly tolerant to “nuisance variation” in input. Reflecting upon the way that DCNNs leverage a combination of linear and non-linear processing to efficiently accomplish this feat allows us to understand how the brain is capable of bi-directional travel between exemplars and abstractions, addressing longstanding problems in empiricist philosophy of mind. I end by considering the prospects for future research on DCNNs, arguing that rather than simply implementing 80s connectionism with more brute-force computation, transformational abstraction counts as a qualitatively distinct form of processing ripe with philosophical and psychological significance, because it is significantly better suited to depict the generic mechanism responsible for this important kind of psychological processing in the brain