A VLSI-design of the minimum entropy neuron

One of the most interesting domains of feedforward networks is the processing of sensor signals. There do exist some networks which extract most of the information by implementing the maximum entropy principle for Gaussian sources. This is done by transforming input patterns to the base of eigenvectors of the input autocorrelation matrix with the biggest eigenvalues. The basic building block of these networks is the linear neuron, learning with the Oja learning rule. Nevertheless, some researchers in pattern recognition theory claim that for pattern recognition and classification clustering transformations are needed which reduce the intra-class entropy. This leads to stable, reliable features and is implemented for Gaussian sources by a linear transformation using the eigenvectors with the smallest eigenvalues. In another paper (Brause 1992) it is shown that the basic building block for such a transformation can be implemented by a linear neuron using an Anti-Hebb rule and restricted weights. This paper shows the analog VLSI design for such a building block, using standard modules of multiplication and addition. The most tedious problem in this VLSI-application is the design of an analog vector normalization circuitry. It can be shown that the standard approaches of weight summation will not give the convergence to the eigenvectors for a proper feature transformation. To avoid this problem, our design differs significantly from the standard approaches by computing the real Euclidean norm. Keywords: minimum entropy, principal component analysis, VLSI, neural networks, surface approximation, cluster transformation, weight normalization circuit

A survey of visual preprocessing and shape representation techniques

Many recent theories and methods proposed for visual preprocessing and shape representation are summarized. The survey brings together research from the fields of biology, psychology, computer science, electrical engineering, and most recently, neural networks. It was motivated by the need to preprocess images for a sparse distributed memory (SDM), but the techniques presented may also prove useful for applying other associative memories to visual pattern recognition. The material of this survey is divided into three sections: an overview of biological visual processing; methods of preprocessing (extracting parts of shape, texture, motion, and depth); and shape representation and recognition (form invariance, primitives and structural descriptions, and theories of attention)

Estimation of the parameters of a boundary contour system using psychophysical hyperacuity experiments

Dissertation (Ph.D.)--Boston UniversityVisual hyperacuity enables observers to make accurate judgments of the relative positions of stimuli when the differences are smaller than the size of a single cone in the fovea. Because hyperacuity can serve as a gauge for precisely measuring characteristics of the visual system, it can provide stringent tests for models of the visual system. A variant of the Boundary Contour System (BCS) model is here used to clarify previously unexplained psychophysical hyperacuity results involving contrast polarity, stimulus separation, and sinusoidal masking gratings. Two-dot alignment thresholds were studied by Levi & Waugh (1996) by varying the gap between the dots, with same and opposite contrast polarity with respect to the background, and also with and without band-limited sinusoidal grating masks of different orientations. They found that when the gap between the dots is small (6 arcmin), different patterns of misalignment thresholds are obtained for the same and different contrast polarity conditions. However, when the gap is large (24 arcmin), the same pattern of thresholds was obtained irrespective of contrast polarity. The simulations presented here replicate these findings, producing the same pattern of results when varying the gap between the dots, with same and opposite contrast polarity with respect to the background, and also with and without sinusoidal grating masks of different orientations. The vision model used (BCS) is able to produce these patterns because of its inherent processing using contrast insensitivity, spatial and oriented competition, and long-range completion layers. A novel aspect of the model is the use of sampled field processing, which simplifies the model's equations. Modified Hebbian learning and a neural decision module are proposed as mechanisms that link the vision model's outputs to a decision criterion. All model parts have plausible neurobiological correlates. In addition, psychophysical hyperacuity experiments served to map the limits of inhibitory spatial interactions. The results show that inhibition occurs even when only half of the split flanking line of Badcock & Westheimer (1985b) is used, suggesting that subthreshold activity in units representing the line extends beyond the end of the line. Furthermore, strong inhibition was observed with a flanking illusory line grating

Modeling the Contributions of the Exocytotic Machinery and Receptor Desensitization to Short- and Long-Term Plasticity of Synapses Between Neocortical Pyramidal Neurons

Short-term synaptic depression (STD) refers to the progressive decrease in synaptic efficacy during a spike train. This decrease may be explained in terms of presynaptic and postsynaptic processes, such as a decrease in the probability of transmitter release, and postsynaptic receptor desensitization. STD may be very strong, and is release-dependent in neocortical pyramid-pyramid synapses. Using a stochastic synapse model, we suggest that the main source of depression in these synapses is the step of vesicle priming, while vesicle depletion and postsynaptic receptor desensitization are proposed to play a lesser role. Our results suggest that vesicle priming may explain not only the release-dependent nature of STD, but also the observation that an average of about one vesicle per active zone is released in central synapses, without positing forced univesicular release. We propose that the latter phenomenon is due to a low priming probability. Our results also explain the effect of paired pre- and postsynaptic activity on STD. In neocortical pyramid-pyramid synapses pairing induces a form of long-term potentiation that has been described as a redistribution of synaptic efficacy (RSE). We propose that RSE is due to a pairing-induced increase in the probability that a primed vesicle will undergo release in response to a presynaptic action potential. This increase may be due to an increased Ca^2+ influx through voltage-gated Ca^2+ channels, or to an increased sensitivity of primed vesicles to this influx. The results were obtained by constraining the model with experimentally observed levels of release probability and other synaptic variables.Defense Advanced Research Projects Agency and the Office of Naval Research (N00014-95-l-0409); Office of Naval Research (N00014-95-l-0657)