On microelectronic self-learning cognitive chip systems

After a brief review of machine learning techniques and applications, this Ph.D. thesis examines several approaches for implementing machine learning architectures and algorithms into hardware within our laboratory. From this interdisciplinary background support, we have motivations for novel approaches that we intend to follow as an objective of innovative hardware implementations of dynamically self-reconfigurable logic for enhanced self-adaptive, self-(re)organizing and eventually self-assembling machine learning systems, while developing this new particular area of research. And after reviewing some relevant background of robotic control methods followed by most recent advanced cognitive controllers, this Ph.D. thesis suggests that amongst many well-known ways of designing operational technologies, the design methodologies of those leading-edge high-tech devices such as cognitive chips that may well lead to intelligent machines exhibiting conscious phenomena should crucially be restricted to extremely well defined constraints. Roboticists also need those as specifications to help decide upfront on otherwise infinitely free hardware/software design details. In addition and most importantly, we propose these specifications as methodological guidelines tightly related to ethics and the nowadays well-identified workings of the human body and of its psyche

Autonomously Reconfigurable Artificial Neural Network on a Chip

Artificial neural network (ANN), an established bio-inspired computing paradigm, has proved very effective in a variety of real-world problems and particularly useful for various emerging biomedical applications using specialized ANN hardware. Unfortunately, these ANN-based systems are increasingly vulnerable to both transient and permanent faults due to unrelenting advances in CMOS technology scaling, which sometimes can be catastrophic. The considerable resource and energy consumption and the lack of dynamic adaptability make conventional fault-tolerant techniques unsuitable for future portable medical solutions. Inspired by the self-healing and self-recovery mechanisms of human nervous system, this research seeks to address reliability issues of ANN-based hardware by proposing an Autonomously Reconfigurable Artificial Neural Network (ARANN) architectural framework. Leveraging the homogeneous structural characteristics of neural networks, ARANN is capable of adapting its structures and operations, both algorithmically and microarchitecturally, to react to unexpected neuron failures. Specifically, we propose three key techniques --- Distributed ANN, Decoupled Virtual-to-Physical Neuron Mapping, and Dual-Layer Synchronization --- to achieve cost-effective structural adaptation and ensure accurate system recovery. Moreover, an ARANN-enabled self-optimizing workflow is presented to adaptively explore a "Pareto-optimal" neural network structure for a given application, on the fly. Implemented and demonstrated on a Virtex-5 FPGA, ARANN can cover and adapt 93% chip area (neurons) with less than 1% chip overhead and O(n) reconfiguration latency. A detailed performance analysis has been completed based on various recovery scenarios

Localist representation can improve efficiency for detection and counting

Almost all representations have both distributed and localist aspects, depending upon what properties of the data are being considered. With noisy data, features represented in a localist way can be detected very efficiently, and in binary representations they can be counted more efficiently than those represented in a distributed way. Brains operate in noisy environments, so the localist representation of behaviourally important events is advantageous, and fits what has been found experimentally. Distributed representations require more neurons to perform as efficiently, but they do have greater versatility

An investigation into implicit emotional memory with concurrent midazolam amnesia following colonoscopy

The aim of this study was to investigate whether implicit emotional memory could be demonstrated in patients undergoing a colonoscopy with midazolam sedation. It was hypothesised that the distress associated with a noxious non-surgical procedure would facilitate the conditioning of neologisms designed to readily associate with the negative experience of colonoscopy. It was further hypothesised that mood (in particular high levels of anxiety), personality (specifically introverted and neurotic patients) and objectively rated peri-operative behavioural distress (especially high ratings of distress) may increase the likelihood of implicit emotional memory formation.The study design was a prospective randomised pre and post repeated measures double blind trial including comparison between three groups. Measurement took place at three different time points both pre- and post-surgical procedure (i.e. immediately before and after the colonoscopy and up to one week post procedure). Also the group that participants were allocated to was randomised and unknown to both the researcher and the participant. The measures used were the Hospital Anxiety and Depression Scale, the Eysenck Personality Inventory, the Behavioural Distress Scale, free recall as a measure of explicit memory for intra-operative events and skin conductance response change as a measure of implicit emotional memory for intra-operatively presented neologisms.The investigation failed to find any statistically significant evidence for implicit memory of neologisms presented during colonoscopy, as detected by skin conductance response, or any differences between those participants presented with 'emotive' and 'neutral' neologisms. There was also no significant effect of mood. personality or behavioural distress on this hypothesised interaction.A potentially unrepresentative and relatively small sample. plus some limitations of methodology. implementation and interpretation are discussed with reference to other research and literature related to the field of interest. Finally. some suggestions are made concerning the direction of future research

An investigation into the neural substrates of virtue to determine the key place of virtues in human moral development

Virtues, as described by Aristotle and Aquinas, are understood as dispositions of character to behave in habitual, specific, positive ways; virtue is a critical requirement for human flourishing. From the perspective of Aristotelian-Thomistic anthropology which offers an integrated vision of the material and the rational in the human person, I seek to identify the neural bases for the development and exercise of moral virtue. First I review current neuroscientific knowledge of the capacity of the brain to structure according to experience, to facilitate behaviours, to regulate emotional responses and support goal election. Then, having identified characteristics of moral virtue in the light of the distinctions between cardinal virtues, I propose neural substrates by mapping neuroscientific knowledge to these characteristics. I then investigate the relationship between virtue, including its neurobiological features, and human flourishing. This process allows a contemporary and evidence-based corroboration for a model of moral development based on growth in virtue as understood by Aristotle and Aquinas, and a demonstration of a biological aptitude and predisposition for the development of virtue. Conclusions are drawn with respect to science, ethics, and parenting

DESIGN AND IMPLEMENTATION OF INTELLIGENT MONITORING SYSTEMS FOR THERMAL POWER PLANT BOILER TRIPS

Steam boilers represent the main equipment in the power plant. Some boiler trips may lead to an entire shutdown of the plant, which is economically burdensome. An early detection and diagnosis of the boiler trips is crucial to maintain normal and safe operational conditions of the plant. Numbers of methodologies have been proposed in the literature for fault diagnosis of power plants. However, rapid deployment of these methodologies is difficult to be achieved due to certain inherent limitations such as system inability to learn or a dynamically improve the system performance and the brittleness of the system beyond its domain of expertise. As a potential solution to these problems, two artificial intelligent monitoring systems specialized in boiler trips have been proposed and coded within the MA TLAB environment in the present work. The training and validation of the two systems have been performed using real operational data which was captured from the plant integrated acquisition system of JANAMANJUNG coal-fired power plant. An integrated plant data preparation framework for seven boiler trips with related operational variables, has been proposed for the training and validation of the proposed artificial intelligent systems. The feedforward neural network methodology has been adopted as a major computational intelligent tool in both systems. The root mean square error has been widely used as a performance indicator of the proposed systems. The first intelligent monitoring system represents the use of the pure artificial neural network system for boiler trip detection. The final architecture for this system has been explored after investigation of various main neural network topology combinations which include one and two hidden layers, one to ten neurons for each hidden layer, three types of activation function, and four types of multidimensional minimization training algorithms. It has been found that there was no general neural network topology combination that can be applied for all boiler trips. All seven boiler trips under consideration had been detected by the proposed systems before or at the same time as the plant control system. The second intelligent monitoring system represents mergmg of genetic algorithms and artificial neural networks as a hybrid intelligent system. For this hybrid intelligent system, the selection of appropriate variables from hundreds of boiler operation variables with optimal neural network topology combinations to monitor boiler trips was a major concern. The encoding and optimization process using genetic algorithms has been applied successfully. A slightly lower root mean square error was observed in the second system which reveals that the hybrid intelligent system performed better than the pure neural network system. Also, the optimal selection of the most influencing variables was performed successfully by the hybrid intelligent system. The proposed artificial intelligent systems could be adopted on-line as a reliable controller of the thermal power plant boiler

Insect neuroethology of reinforcement learning

Historically, reinforcement learning is a branch of machine learning founded on observations of how animals learn. This involved collaboration between the fields of biology and artificial intelligence that was beneficial to both fields, creating smarter artificial agents and improving the understanding of how biological systems function. The evolution of reinforcement learning during the past few years was rapid but substantially diverged from providing insights into how biological systems work, opening a gap between reinforcement learning and biology. In an attempt to close this gap, this thesis studied the insect neuroethology of reinforcement learning, that is, the neural circuits that underlie reinforcement-learning-related behaviours in insects. The goal was to extract a biologically plausible plasticity function from insect-neuronal data, use this to explain biological findings and compare it to more standard reinforcement learning models. Consequently, a novel dopaminergic plasticity rule was developed to approximate the function of dopamine as the plasticity mechanism between neurons in the insect brain. This allowed a range of observed learning phenomena to happen in parallel, like memory depression, potentiation, recovery, and saturation. In addition, by using anatomical data of connections between neurons in the mushroom body neuropils of the insect brain, the neural incentive circuit of dopaminergic and output neurons was also explored. This, together with the dopaminergic plasticity rule, allowed for dynamic collaboration amongst parallel memory functions, such as acquisition, transfer, and forgetting. When tested on olfactory conditioning paradigms, the model reproduced the observed changes in the activity of the identified neurons in fruit flies. It also replicated the observed behaviour of the animals and it allowed for flexible behavioural control. Inspired by the visual navigation system of desert ants, the model was further challenged in the visual place recognition task. Although a relatively simple encoding of the olfactory information was sufficient to explain odour learning, a more sophisticated encoding of the visual input was required to increase the separability among the visual inputs and enable visual place recognition. Signal whitening and sparse combinatorial encoding were sufficient to boost the performance of the system in this task. The incentive circuit enabled the encoding of increasing familiarity along a known route, which dropped proportionally to the distance of the animal from that route. Finally, the proposed model was challenged in delayed reinforcement tasks, suggesting that it might take the role of an adaptive critic in the context of reinforcement learning

NASA Space Engineering Research Center Symposium on VLSI Design

The NASA Space Engineering Research Center (SERC) is proud to offer, at its second symposium on VLSI design, presentations by an outstanding set of individuals from national laboratories and the electronics industry. These featured speakers share insights into next generation advances that will serve as a basis for future VLSI design. Questions of reliability in the space environment along with new directions in CAD and design are addressed by the featured speakers

An instruction systolic array architecture for multiple neural network types

Modern electronic systems, especially sensor and imaging systems, are beginning to incorporate their own neural network subsystems. In order for these neural systems to learn in real-time they must be implemented using VLSI technology, with as much of the learning processes incorporated on-chip as is possible. The majority of current VLSI implementations literally implement a series of neural processing cells, which can be connected together in an arbitrary fashion. Many do not perform the entire neural learning process on-chip, instead relying on other external systems to carry out part of the computation requirements of the algorithm. The work presented here utilises two dimensional instruction systolic arrays in an attempt to define a general neural architecture which is closer to the biological basis of neural networks - it is the synapses themselves, rather than the neurons, that have dedicated processing units. A unified architecture is described which can be programmed at the microcode level in order to facilitate the processing of multiple neural network types. An essential part of neural network processing is the neuron activation function, which can range from a sequential algorithm to a discrete mathematical expression. The architecture presented can easily carry out the sequential functions, and introduces a fast method of mathematical approximation for the more complex functions. This can be evaluated on-chip, thus implementing the entire neural process within a single system. VHDL circuit descriptions for the chip have been generated, and the systolic processing algorithms and associated microcode instruction set for three different neural paradigms have been designed. A software simulator of the architecture has been written, giving results for several common applications in the field