ART Neural Networks: Distributed Coding and ARTMAP Applications

ART (Adaptive Resonance Theory) neural networks for fast, stable learning and prediction have been applied in a variety of areas. Applications include airplane design and manufacturing, automatic target recognition, financial forecasting, machine tool monitoring, digital circuit design, chemical analysis, and robot vision. Supervised ART architectures, called ARTMAP systems, feature internal control mechanisms that create stable recognition categories of optimal size by maximizing code compression while minimizing predictive error in an on-line setting. Special-purpose requirements of various application domains have led to a number of ARTMAP variants, including fuzzy ARTMAP, ART-EMAP, Gaussian ARTMAP, and distributed ARTMAP. ARTMAP has been used for a variety of applications, including computer-assisted medical diagnosis. Medical databases present many of the challenges found in general information management settings where speed, efficiency, ease of use, and accuracy are at a premium. A direct goal of improved computer-assisted medicine is to help deliver quality emergency care in situations that may be less than ideal. Working with these problems has stimulated a number of ART architecture developments, including ARTMAP-IC [1]. This paper describes a recent collaborative effort, using a new cardiac care database for system development, has brought together medical statisticians and clinicians at the New England Medical Center with researchers developing expert systems and neural networks, in order to create a hybrid method for medical diagnosis. The paper also considers new neural network architectures, including distributed ART {dART), a real-time model of parallel distributed pattern learning that permits fast as well as slow adaptation, without catastrophic forgetting. Local synaptic computations in the dART model quantitatively match the paradoxical phenomenon of Markram-Tsodyks [2] redistribution of synaptic efficacy, as a consequence of global system hypotheses.Office of Naval Research (N00014-95-1-0409, N00014-95-1-0657

Bring ART into the ACT

ACT is compared with a particular type of connectionist model that cannot handle symbols and use non-biological operations that cannot learn in real time. This focus continues an unfortunate trend of straw man "debates" in cognitive science. Adaptive Resonance Theory, or ART, neural models of cognition can handle both symbols and sub-symbolic representations, and meets the Newell criteria at least as well as these models.Air Force Office of Scientific Research (F49620-01-1-0397); Office of Naval Research (N00014-01-1-0624

Log-domain implementation of complex dynamics reaction-diffusion neural networks

In this paper, we have identified a second-order reaction-diffusion differential equation able to reproduce through parameter setting different complex spatio-temporal behaviors. We have designed a log-domain hardware that implements the spatially discretized version of the selected reaction-diffusion equation. The logarithmic compression of the state variables allows several decades of variation of these state variables within subthreshold operation of the MOS transistors. Furthermore, as all the equation parameters are implemented as currents, they can be adjusted several decades. As a demonstrator, we have designed a chip containing a linear array of ten second-order dynamics coupled cells. Using this hardware, we have experimentally reproduced two complex spatio-temporal phenomena: the propagation of travelling waves and of trigger waves, as well as isolated oscillatory cells.Gobierno de España TIC1999-0446-C02-02Office of Naval Research (USA

On the design and characterization of femtoampere current-mode circuits

In this paper, we show and validate a reliable circuit design technique based on source voltage shifting for current-mode signal processing down to femtoamperes. The technique involves specific-current extractors and logarithmic current splitters for obtaining on-chip subpicoampere currents. It also uses a special on-chip sawtooth oscillator to monitor and measure currents down to a few femtoamperes. This way, subpicoampere currents are characterized without driving them off chip and requiring expensive instrumentation with complicated low leakage setups. A special current mirror is also introduced for reliably replicating such low currents. As an example, a simple log-domain first-order low-pass filter is Implemented that uses a 100-fF capacitor and a 3.5-fA bias current to achieve a cutoff frequency of 0.5 Hz. A technique for characterizing noise at these currents is also described and verified. Finally, transistor mismatch measurements are provided and discussed. Experimental measurements are shown throughout the paper, obtained from prototypes fabricated in the AMS 0.35-μm three-metal two-poly standard CMOS process.Ministerio de Ciencia y Tecnología TIC-1999-0446-C02-02, FIT-070000-2001-0859, TIC-2000-0406-P4-05, TIC-2002-10878-EEuropean Union IST-2001-3412

Compact low-power calibration mini-DACs for neural arrays with programmable weights

This paper considers the viability of compact low-resolution low-power mini digital-to-analog converters (mini-DACs) for use in large arrays of neural type cells, where programmable weights are required. Transistors are biased in weak inversion in order to yield small currents and low power consumptions, a necessity when building large size arrays. One important drawback of weak inversion operation is poor matching between transistors. The resulting effective precision of a fabricated array of 50 DACs turned out to be 47% (1.1 bits), due to transistor mismatch. However, it is possible to combine them two by two in order to build calibrated DACs, thus compensating for inter-DAC mismatch. It is shown experimentally that the precision can be improved easily by a factor of 10 (4.8% or 4.4 bits), which makes these DACs viable for low-resolution applications such as massive arrays of neural processing circuits. A design methodology is provided, and illustrated through examples, to obtain calibrated mini-DACs of a given target precision. As an example application, we show simulation results of using this technique to calibrate an array of digitally controlled integrate-and-fire neurons.Gobierno de España TIC1999-0446-C02-02, TIC2000-0406-P4-05, FIT-07000/2002/921, TIC2002-10878-EEuropean Union IST- 2001-3412

ART/SOFM: A Hybrid Approach to the TSP

We present a new method of solving large scale travelling salesman problem (TSP) instances using a combination of adaptive resonance theory (ART) and self organizing feature maps (SOFM). We divide our algorithm into three phases: phase one uses ART to form clusters of cities; phase two uses a novel modification of the traditional SOFM algorithm to solve a slight variant of the TSP in each cluster of cities; and phase three uses another version of the SOFM to link all the clusters. The experimental results show that our algorithm finds approximate solutions which are about 13% longer than those reported by the chained Lin Kernighan method for problem sizes of 14,000 citie

Exploring the landscapes of "computing": digital, neuromorphic, unconventional -- and beyond

The acceleration race of digital computing technologies seems to be steering toward impasses -- technological, economical and environmental -- a condition that has spurred research efforts in alternative, "neuromorphic" (brain-like) computing technologies. Furthermore, since decades the idea of exploiting nonlinear physical phenomena "directly" for non-digital computing has been explored under names like "unconventional computing", "natural computing", "physical computing", or "in-materio computing". This has been taking place in niches which are small compared to other sectors of computer science. In this paper I stake out the grounds of how a general concept of "computing" can be developed which comprises digital, neuromorphic, unconventional and possible future "computing" paradigms. The main contribution of this paper is a wide-scope survey of existing formal conceptualizations of "computing". The survey inspects approaches rooted in three different kinds of background mathematics: discrete-symbolic formalisms, probabilistic modeling, and dynamical-systems oriented views. It turns out that different choices of background mathematics lead to decisively different understandings of what "computing" is. Across all of this diversity, a unifying coordinate system for theorizing about "computing" can be distilled. Within these coordinates I locate anchor points for a foundational formal theory of a future computing-engineering discipline that includes, but will reach beyond, digital and neuromorphic computing.Comment: An extended and carefully revised version of this manuscript has now (March 2021) been published as "Toward a generalized theory comprising digital, neuromorphic, and unconventional computing" in the new open-access journal Neuromorphic Computing and Engineerin

Theory and computation of electromagnetic fields and thermomechanical structure interaction for systems undergoing large deformations

For an accurate description of electromagneto-thermomechanical systems, electromagnetic fields need to be described in a Eulerian frame, whereby the thermomechanics is solved in a Lagrangean frame. It is possible to map the Eulerian frame to the current placement of the matter and the Lagrangean frame to a reference placement. We present a rigorous and thermodynamically consistent derivation of governing equations for fully coupled electromagneto-thermomechanical systems properly handling finite deformations. A clear separation of the different frames is necessary. There are various attempts to formulate electromagnetism in the Lagrangean frame, or even to compute all fields in the current placement. Both formulations are challenging and heavily discussed in the literature. In this work, we propose another solution scheme that exploits the capabilities of advanced computational tools. Instead of amending the formulation, we can solve thermomechanics in the Lagrangean frame and electromagnetism in the Eulerian frame and manage the interaction between the fields. The approach is similar to its analog in fluid structure interaction, but more challenging because the field equations in electromagnetism must also be solved within the solid body while following their own different set of transformation rules. We additionally present a mesh-morphing algorithm necessary to accommodate finite deformations to solve the electromagnetic fields outside of the material body. We illustrate the use of the new formulation by developing an open-source implementation using the FEniCS package and applying this implementation to several engineering problems in electromagnetic structure interaction undergoing large deformations

Delay dynamics of neuromorphic optoelectronic nanoscale resonators: Perspectives and applications

With the recent exponential growth of applications using artificial intelligence (AI), the development of efficient and ultrafast brain-like (neuromorphic) systems is crucial for future information and communication technologies. While the implementation of AI systems using computer algorithms of neural networks is emerging rapidly, scientists are just taking the very first steps in the development of the hardware elements of an artificial brain, specifically neuromorphic microchips. In this review article, we present the current state of the art of neuromorphic photonic circuits based on solid-state optoelectronic oscillators formed by nanoscale double barrier quantum well resonant tunneling diodes. We address, both experimentally and theoretically, the key dynamic properties of recently developed artificial solid-state neuron microchips with delayed perturbations and describe their role in the study of neural activity and regenerative memory. This review covers our recent research work on excitable and delay dynamic characteristics of both single and autaptic (delayed) artificial neurons including all-or-none response, spike-based data encoding, storage, signal regeneration and signal healing. Furthermore, the neural responses of these neuromorphic microchips display all the signatures of extended spatio-temporal localized structures (LSs) of light, which are reviewed here in detail. By taking advantage of the dissipative nature of LSs, we demonstrate potential applications in optical data reconfiguration and clock and timing at high-speeds and with short transients. The results reviewed in this article are a key enabler for the development of high-performance optoelectronic devices in future high-speed brain-inspired optical memories and neuromorphic computing. (C) 2017 Author(s).Fundacao para a Ciencia e a Tecnologia (FCT) [UID/Multi/00631/2013]European Structural and Investment Funds (FEEI) through the Competitiveness and Internationalization Operational Program - COMPETE 2020National Funds through FCT [ALG-01-0145-FEDER-016432/POCI-01-0145-FEDER-016432]European Commission under the project iBROW [645369]project COMBINA [TEC2015-65212-C3-3-PAEI/FEDER UE]Ramon y Cajal fellowshipinfo:eu-repo/semantics/publishedVersio