Deep Neural Networks - A Brief History

Introduction to deep neural networks and their history.Comment: 14 pages, 14 figure

Ethics of Artificial Intelligence Demarcations

In this paper we present a set of key demarcations, particularly important when discussing ethical and societal issues of current AI research and applications. Properly distinguishing issues and concerns related to Artificial General Intelligence and weak AI, between symbolic and connectionist AI, AI methods, data and applications are prerequisites for an informed debate. Such demarcations would not only facilitate much-needed discussions on ethics on current AI technologies and research. In addition sufficiently establishing such demarcations would also enhance knowledge-sharing and support rigor in interdisciplinary research between technical and social sciences.Comment: Proceedings of the Norwegian AI Symposium 2019 (NAIS 2019), Trondheim, Norwa

Sequence learning in Associative Neuronal-Astrocytic Network

The neuronal paradigm of studying the brain has left us with limitations in both our understanding of how neurons process information to achieve biological intelligence and how such knowledge may be translated into artificial intelligence and its most brain-derived branch, neuromorphic computing. Overturning our fundamental assumptions of how the brain works, the recent exploration of astrocytes is revealing that these long-neglected brain cells dynamically regulate learning by interacting with neuronal activity at the synaptic level. Following recent experimental evidence, we designed an associative, Hopfield-type, neuronal-astrocytic network and analyzed the dynamics of the interaction between neurons and astrocytes. We show that astrocytes were sufficient to trigger transitions between learned memories in the neuronal component of the network. Further, we mathematically derived the timing of the transitions that was governed by the dynamics of the calcium-dependent slow-currents in the astrocytic processes. Overall, we provide a brain-morphic mechanism for sequence learning that is inspired by, and aligns with, recent experimental findings. To evaluate our model, we emulated astrocytic atrophy and showed that memory recall becomes significantly impaired after a critical point of affected astrocytes was reached. This brain-inspired and brain-validated approach supports our ongoing efforts to incorporate non-neuronal computing elements in neuromorphic information processing.Comment: 8 pages, 5 figure

Boolean Dynamics with Random Couplings

This paper reviews a class of generic dissipative dynamical systems called N-K models. In these models, the dynamics of N elements, defined as Boolean variables, develop step by step, clocked by a discrete time variable. Each of the N Boolean elements at a given time is given a value which depends upon K elements in the previous time step. We review the work of many authors on the behavior of the models, looking particularly at the structure and lengths of their cycles, the sizes of their basins of attraction, and the flow of information through the systems. In the limit of infinite N, there is a phase transition between a chaotic and an ordered phase, with a critical phase in between. We argue that the behavior of this system depends significantly on the topology of the network connections. If the elements are placed upon a lattice with dimension d, the system shows correlations related to the standard percolation or directed percolation phase transition on such a lattice. On the other hand, a very different behavior is seen in the Kauffman net in which all spins are equally likely to be coupled to a given spin. In this situation, coupling loops are mostly suppressed, and the behavior of the system is much more like that of a mean field theory. We also describe possible applications of the models to, for example, genetic networks, cell differentiation, evolution, democracy in social systems and neural networks.Comment: 69 pages, 16 figures, Submitted to Springer Applied Mathematical Sciences Serie

Boron isotopes in foraminifera : systematics, biomineralisation, and CO2 reconstruction

Funding: Fellowship from University of St Andrews, $100 (pending) from Richard Zeebe, UK NERC grants NE/N003861/1 and NE/N011716/1.The boron isotope composition of foraminifera provides a powerful tracer for CO2 change over geological time. This proxy is based on the equilibrium of boron and its isotopes in seawater, which is a function of pH. However while the chemical principles underlying this proxy are well understood, its reliability has previously been questioned, due to the difficulty of boron isotope (δ11B) analysis on foraminferal samples and questions regarding calibrations between δ11B and pH. This chapter reviews the current state of the δ11B-pH proxy in foraminfera, including the pioneering studies that established this proxy’s potential, and the recent work that has improved understanding of boron isotope systematics in foraminifera and applied this tracer to the geological record. The theoretical background of the δ11B-pH proxy is introduced, including an accurate formulation of the boron isotope mass balance equations. Sample preparation and analysis procedures are then reviewed, with discussion of sample cleaning, the potential influence of diagenesis, and the strengths and weaknesses of boron purification by column chromatography versus microsublimation, and analysis by NTIMS versus MC-ICPMS. The systematics of boron isotopes in foraminifera are discussed in detail, including results from benthic and planktic taxa, and models of boron incorporation, fractionation, and biomineralisation. Benthic taxa from the deep ocean have δ11B within error of borate ion at seawater pH. This is most easily explained by simple incorporation of borate ion at the pH of seawater. Planktic foraminifera have δ11B close to borate ion, but with minor offsets. These may be driven by physiological influences on the foraminiferal microenvironment; a novel explanation is also suggested for the reduced δ11B-pH sensitivities observed in culture, based on variable calcification rates. Biomineralisation influences on boron isotopes are then explored, addressing the apparently contradictory observations that foraminifera manipulate pH during chamber formation yet their δ11B appears to record the pH of ambient seawater. Potential solutions include the influences of magnesium-removal and carbon concentration, and the possibility that pH elevation is most pronounced during initial chamber formation under favourable environmental conditions. The steps required to reconstruct pH and pCO2 from δ11B are then reviewed, including the influence of seawater chemistry on boron equilibrium, the evolution of seawater δ11B, and the influence of second carbonate system parameters on δ11B-based reconstructions of pCO2. Applications of foraminiferal δ11B to the geological record are highlighted, including studies that trace CO2 storage and release during recent ice ages, and reconstructions of pCO2 over the Cenozoic. Relevant computer codes and data associated with this article are made available online.Publisher PDFPeer reviewe

An Assessment of Students’ Satisfaction in Higher Education

Student’s Satisfaction (SS) with a particular subject may impact the learning process, being the figure of attentiveness of the utmost importance over time, and also a very difficult undertaking to accomplish. To go forward with such exercise, a workable methodology for problem solving had to be built and tested. It is based on a thermodynamic approach to Knowledge Representation and Reasoning, which is the ultimate goal of SS assessment when working on a particular topic

A Reconfigurable Architecture for Implementing Locally Connected Neural Arrays

Moore’s law is rapidly approaching a long-predicted decline, and with it the performance gains of conventional processors are becoming ever more marginal. Cognitive computing systems based on neural networks have the potential to provide a solution to the decline of Moore’s law. Identifying common traits in neural systems can lead to the design of more efficient, robust and adaptable processors. Despite the potentials, large-scale neural systems remain difficult to implement due to constraints on scalability. Here we introduce a new hardware architecture for implementing locally connected neural networks that can model biological systems with a high level of scalability. We validate our architecture using a full model of the locomotion system of the Caenorhabditis elegans. Further, we show that our proposed architecture archives a nine-fold increase in clock speed over existing hardware models. Importantly the clock speed for our architecture is found to be independent of system size, providing an unparalleled level of scalability. Our approach can be applied to the modelling of large neural networks, with greater performance, easier configuration and a high level of scalability

Explicit Logic Circuits Discriminate Neural States

The magnitude and apparent complexity of the brain's connectivity have left explicit networks largely unexplored. As a result, the relationship between the organization of synaptic connections and how the brain processes information is poorly understood. A recently proposed retinal network that produces neural correlates of color vision is refined and extended here to a family of general logic circuits. For any combination of high and low activity in any set of neurons, one of the logic circuits can receive input from the neurons and activate a single output neuron whenever the input neurons have the given activity state. The strength of the output neuron's response is a measure of the difference between the smallest of the high inputs and the largest of the low inputs. The networks generate correlates of known psychophysical phenomena. These results follow directly from the most cost-effective architectures for specific logic circuits and the minimal cellular capabilities of excitation and inhibition. The networks function dynamically, making their operation consistent with the speed of most brain functions. The networks show that well-known psychophysical phenomena do not require extraordinarily complex brain structures, and that a single network architecture can produce apparently disparate phenomena in different sensory systems

Nature inspired meta-heuristic algorithms for deep learning: recent progress and novel perspective

Deep learning is presently attracting extra ordinary attention from both the industry and the academia. The application of deep learning in computer vision has recently gain popularity. The optimization of deep learning models through nature inspired algorithms is a subject of debate in computer science. The application areas of the hybrid of natured inspired algorithms and deep learning architecture includes: machine vision and learning, image processing, data science, autonomous vehicles, medical image analysis, biometrics, etc. In this paper, we present recent progress on the application of nature inspired algorithms in deep learning. The survey pointed out recent development issues, strengths, weaknesses and prospects for future research. A new taxonomy is created based on natured inspired algorithms for deep learning. The trend of the publications in this domain is depicted; it shows the research area is growing but slowly. The deep learning architectures not exploit by the nature inspired algorithms for optimization are unveiled. We believed that the survey can facilitate synergy between the nature inspired algorithms and deep learning research communities. As such, massive attention can be expected in a near future