Analysis and synthesis of neural networks

The brain has long attracted the interest of researchers. Some tasks, such as pattern recognition and optimization, have proven to be exceptionally difficult for conventional computing systems to perform, but are executed by the brain almost effortlessly. Due to the large number of neurons and interconnections, it has proven impossible to model the brain's architecture exactly. Researchers have instead created modest networks of artificial neural elements which can perform some interesting functions, such as generalization, pattern recognition, and the solution of certain optimization problems. These artificial neural networks do not work in exactly the same manner as the brain but they are proving useful in numerous practical applications. In this thesis the analysis, design, and hardware implementation of an artificial neural network is presented. In order to fully appreciate the architecture of the neural network described, a discussion of the properties of biological neurons and a brief history of neural network research is included. This is followed by a detailed discussion of the design, implementation, and testing of the network. The network under consideration is a general purpose time-multiplexed neural network suitable for VLSI implementation. A non-multiplexed neural network implementation generally requires full connectivity of the neural elements. This approach consumes too much chip area. The multiplexed approach taken in this thesis offers substantial area savings over the non-multiplexed approach

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Reservoir computing (RC), first applied to temporal signal processing, is a recurrent neural network in which neurons are randomly connected. Once initialized, the connection strengths remain unchanged. Such a simple structure turns RC into a non-linear dynamical system that maps low-dimensional inputs into a high-dimensional space. The model's rich dynamics, linear separability, and memory capacity then enable a simple linear readout to generate adequate responses for various applications. RC spans areas far beyond machine learning, since it has been shown that the complex dynamics can be realized in various physical hardware implementations and biological devices. This yields greater flexibility and shorter computation time. Moreover, the neuronal responses triggered by the model's dynamics shed light on understanding brain mechanisms that also exploit similar dynamical processes. While the literature on RC is vast and fragmented, here we conduct a unified review of RC's recent developments from machine learning to physics, biology, and neuroscience. We first review the early RC models, and then survey the state-of-the-art models and their applications. We further introduce studies on modeling the brain's mechanisms by RC. Finally, we offer new perspectives on RC development, including reservoir design, coding frameworks unification, physical RC implementations, and interaction between RC, cognitive neuroscience and evolution.Comment: 51 pages, 19 figures, IEEE Acces

組合せ最適化問題のための測定フィードバック型コヒーレント・イジングマシンの実現と評価

学位の種別: 課程博士審査委員会委員 : （主査）東京大学教授 合原 一幸, 東京大学教授 岩田 覚, 東京大学准教授 平田 祥人, 東京大学准教授 大西 立顕, 東京大学准教授 鈴木 大慈University of Tokyo(東京大学

Hybrid Analog-Digital Co-Processing for Scientific Computation

In the past 10 years computer architecture research has moved to more heterogeneity and less adherence to conventional abstractions. Scientists and engineers hold an unshakable belief that computing holds keys to unlocking humanity's Grand Challenges. Acting on that belief they have looked deeper into computer architecture to find specialized support for their applications. Likewise, computer architects have looked deeper into circuits and devices in search of untapped performance and efficiency. The lines between computer architecture layers---applications, algorithms, architectures, microarchitectures, circuits and devices---have blurred. Against this backdrop, a menagerie of computer architectures are on the horizon, ones that forgo basic assumptions about computer hardware, and require new thinking of how such hardware supports problems and algorithms. This thesis is about revisiting hybrid analog-digital computing in support of diverse modern workloads. Hybrid computing had extensive applications in early computing history, and has been revisited for small-scale applications in embedded systems. But architectural support for using hybrid computing in modern workloads, at scale and with high accuracy solutions, has been lacking. I demonstrate solving a variety of scientific computing problems, including stochastic ODEs, partial differential equations, linear algebra, and nonlinear systems of equations, as case studies in hybrid computing. I solve these problems on a system of multiple prototype analog accelerator chips built by a team at Columbia University. On that team I made contributions toward programming the chips, building the digital interface, and validating the chips' functionality. The analog accelerator chip is intended for use in conjunction with a conventional digital host computer. The appeal and motivation for using an analog accelerator is efficiency and performance, but it comes with limitations in accuracy and problem sizes that we have to work around. The first problem is how to do problems in this unconventional computation model. Scientific computing phrases problems as differential equations and algebraic equations. Differential equations are a continuous view of the world, while algebraic equations are a discrete one. Prior work in analog computing mostly focused on differential equations; algebraic equations played a minor role in prior work in analog computing. The secret to using the analog accelerator to support modern workloads on conventional computers is that these two viewpoints are interchangeable. The algebraic equations that underlie most workloads can be solved as differential equations, and differential equations are naturally solvable in the analog accelerator chip. A hybrid analog-digital computer architecture can focus on solving linear and nonlinear algebra problems to support many workloads. The second problem is how to get accurate solutions using hybrid analog-digital computing. The reason that the analog computation model gives less accurate solutions is it gives up representing numbers as digital binary numbers, and instead uses the full range of analog voltage and current to represent real numbers. Prior work has established that encoding data in analog signals gives an energy efficiency advantage as long as the analog data precision is limited. While the analog accelerator alone may be useful for energy-constrained applications where inputs and outputs are imprecise, we are more interested in using analog in conjunction with digital for precise solutions. This thesis gives novel insight that the trick to do so is to solve nonlinear problems where low-precision guesses are useful for conventional digital algorithms. The third problem is how to solve large problems using hybrid analog-digital computing. The reason the analog computation model can't handle large problems is it gives up step-by-step discrete-time operation, instead allowing variables to evolve smoothly in continuous time. To make that happen the analog accelerator works by chaining hardware for mathematical operations end-to-end. During computation analog data flows through the hardware with no overheads in control logic and memory accesses. The downside is then the needed hardware size grows alongside problem sizes. While scientific computing researchers have for a long time split large problems into smaller subproblems to fit in digital computer constraints, this thesis is a first attempt to consider these divide-and-conquer algorithms as an essential tool in using the analog model of computation. As we enter the post-Moore’s law era of computing, unconventional architectures will offer specialized models of computation that uniquely support specific problem types. Two prominent examples are deep neural networks and quantum computers. Recent trends in computer science research show these unconventional architectures will soon have broad adoption. In this thesis I show another specialized, unconventional architecture is to use analog accelerators to solve problems in scientific computing. Computer architecture researchers will discover other important models of computation in the future. This thesis is an example of the discovery process, implementation, and evaluation of how an unconventional architecture supports specialized workloads

Nonlinear process modeling of pH neutralization process in CSTR using,selective combination of multiple neural Networks.

pH control problem is very important in many chemical and biological systems and especially in waste treatment plants. The neutralization is very fast and occurs as a result of a simple reaction. However, from the control point of view it is very difficult problem to handle because of its high nonlinearity due to the varying gain and varying dynamics with respect to the operating point. Masalah pengawalan pH adalah amat penting dalam kebanyakan proses kimia mahupun biologi terutamanya dalam sistem rawatan air sisa. Dalam sistem ini, proses peneutralan berlaku begitu pantas dan hanya disebabkan oleh tindakbalas yang ringkas

Technology 2003: The Fourth National Technology Transfer Conference and Exposition, volume 2

Proceedings from symposia of the Technology 2003 Conference and Exposition, Dec. 7-9, 1993, Anaheim, CA, are presented. Volume 2 features papers on artificial intelligence, CAD&E, computer hardware, computer software, information management, photonics, robotics, test and measurement, video and imaging, and virtual reality/simulation

Biologically Inspired Vision and Control for an Autonomous Flying Vehicle

This thesis makes a number of new contributions to control and sensing for unmanned vehicles. I begin by developing a non-linear simulation of a small unmanned helicopter and then proceed to develop new algorithms for control and sensing using the simulation. The work is field-tested in successful flight trials of biologically inspired vision and neural network control for an unstable rotorcraft. The techniques are more robust and more easily implemented on a small flying vehicle than previously attempted methods. ¶ ..

Functional grammar and genre analysis : a description of the language of learned and popular articles

There has been a growing interest in the form and function of academic English, especially among teachers of English as a Foreign Language. `Academic' English, however, covers a variety of genres, including specialist and non-specialist writings across a range of disciplines. Little is known about the linguistic similarities and differences among these genres. This thesis aims to add to the study of academic English by investigating learned and popular articles in the fields of biology, computing and history. The descriptive framework is based mainly on Halliday's functional grammar, although reference is made to other linguistic theories, such as Winter's clause relations. Eighteen articles from the three fields were selected, nine learned articles and nine corresponding popular articles. Extracts from these articles form the small corpus analysed. After an introductory chapter, the second chapter reviews the nature of theme in English, and performs a thematic analysis on the corpus. The third chapter reviews the ideational function of language, and investigates how the language of the corpus articles represents reality. The fourth chapter reviews the interpersonal function of language and investigates this aspect of the corpus. The penultimate chapter comments on discourse patterns in the articles. The conclusion suggests that the similarities and differences between learned and popular articles, and between science and the humanities, are a result of systematic functional variation among genres