4,592 research outputs found
Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective
On metrics of density and power efficiency, neuromorphic technologies have
the potential to surpass mainstream computing technologies in tasks where
real-time functionality, adaptability, and autonomy are essential. While
algorithmic advances in neuromorphic computing are proceeding successfully, the
potential of memristors to improve neuromorphic computing have not yet born
fruit, primarily because they are often used as a drop-in replacement to
conventional memory. However, interdisciplinary approaches anchored in machine
learning theory suggest that multifactor plasticity rules matching neural and
synaptic dynamics to the device capabilities can take better advantage of
memristor dynamics and its stochasticity. Furthermore, such plasticity rules
generally show much higher performance than that of classical Spike Time
Dependent Plasticity (STDP) rules. This chapter reviews the recent development
in learning with spiking neural network models and their possible
implementation with memristor-based hardware
Approaches to the implementation of binary relation inference network.
by C.W. Tong.Thesis (M.Phil.)--Chinese University of Hong Kong, 1994.Includes bibliographical references (leaves 96-98).Chapter 1 --- Introduction --- p.1Chapter 1.1 --- The Availability of Parallel Processing Machines --- p.2Chapter 1.1.1 --- Neural Networks --- p.5Chapter 1.2 --- Parallel Processing in the Continuous-Time Domain --- p.6Chapter 1.3 --- Binary Relation Inference Network --- p.10Chapter 2 --- Binary Relation Inference Network --- p.12Chapter 2.1 --- Binary Relation Inference Network --- p.12Chapter 2.1.1 --- Network Structure --- p.14Chapter 2.2 --- Shortest Path Problem --- p.17Chapter 2.2.1 --- Problem Statement --- p.17Chapter 2.2.2 --- A Binary Relation Inference Network Solution --- p.18Chapter 3 --- A Binary Relation Inference Network Prototype --- p.21Chapter 3.1 --- The Prototype --- p.22Chapter 3.1.1 --- The Network --- p.22Chapter 3.1.2 --- Computational Element --- p.22Chapter 3.1.3 --- Network Response Time --- p.27Chapter 3.2 --- Improving Response --- p.29Chapter 3.2.1 --- Removing Feedback --- p.29Chapter 3.2.2 --- Selecting Minimum with Diodes --- p.30Chapter 3.3 --- Speeding Up the Network Response --- p.33Chapter 3.4 --- Conclusion --- p.35Chapter 4 --- VLSI Building Blocks --- p.36Chapter 4.1 --- The Site --- p.37Chapter 4.2 --- The Unit --- p.40Chapter 4.2.1 --- A Minimum Finding Circuit --- p.40Chapter 4.2.2 --- A Tri-state Comparator --- p.44Chapter 4.3 --- The Computational Element --- p.45Chapter 4.3.1 --- Network Performances --- p.46Chapter 4.4 --- Discussion --- p.47Chapter 5 --- A VLSI Chip --- p.48Chapter 5.1 --- Spatial Configuration --- p.49Chapter 5.2 --- Layout --- p.50Chapter 5.2.1 --- Computational Elements --- p.50Chapter 5.2.2 --- The Network --- p.52Chapter 5.2.3 --- I/O Requirements --- p.53Chapter 5.2.4 --- Optional Modules --- p.53Chapter 5.3 --- A Scalable Design --- p.54Chapter 6 --- The Inverse Shortest Paths Problem --- p.57Chapter 6.1 --- Problem Statement --- p.59Chapter 6.2 --- The Embedded Approach --- p.63Chapter 6.2.1 --- The Formulation --- p.63Chapter 6.2.2 --- The Algorithm --- p.65Chapter 6.3 --- Implementation Results --- p.66Chapter 6.4 --- Other Implementations --- p.67Chapter 6.4.1 --- Sequential Machine --- p.67Chapter 6.4.2 --- Parallel Machine --- p.68Chapter 6.5 --- Discussion --- p.68Chapter 7 --- Closed Semiring Optimization Circuits --- p.71Chapter 7.1 --- Transitive Closure Problem --- p.72Chapter 7.1.1 --- Problem Statement --- p.72Chapter 7.1.2 --- Inference Network Solutions --- p.73Chapter 7.2 --- Closed Semirings --- p.76Chapter 7.3 --- Closed Semirings and the Binary Relation Inference Network --- p.79Chapter 7.3.1 --- Minimum Spanning Tree --- p.80Chapter 7.3.2 --- VLSI Implementation --- p.84Chapter 7.4 --- Conclusion --- p.86Chapter 8 --- Conclusions --- p.87Chapter 8.1 --- Summary of Achievements --- p.87Chapter 8.2 --- Future Work --- p.89Chapter 8.2.1 --- VLSI Fabrication --- p.89Chapter 8.2.2 --- Network Robustness --- p.90Chapter 8.2.3 --- Inference Network Applications --- p.91Chapter 8.2.4 --- Architecture for the Bellman-Ford Algorithm --- p.91Bibliography --- p.92Appendices --- p.99Chapter A --- Detailed Schematic --- p.99Chapter A.1 --- Schematic of the Inference Network Structures --- p.99Chapter A.1.1 --- Unit with Self-Feedback --- p.99Chapter A.1.2 --- Unit with Self-Feedback Removed --- p.100Chapter A.1.3 --- Unit with a Compact Minimizer --- p.100Chapter A.1.4 --- Network Modules --- p.100Chapter A.2 --- Inference Network Interface Circuits --- p.100Chapter B --- Circuit Simulation and Layout Tools --- p.107Chapter B.1 --- Circuit Simulation --- p.107Chapter B.2 --- VLSI Circuit Design --- p.110Chapter B.3 --- VLSI Circuit Layout --- p.111Chapter C --- The Conjugate-Gradient Descent Algorithm --- p.113Chapter D --- Shortest Path Problem on MasPar --- p.11
Recommended from our members
Physically Equivalent Intelligent Systems for Reasoning Under Uncertainty at Nanoscale
Machines today lack the inherent ability to reason and make decisions, or operate in the presence of uncertainty. Machine-learning methods such as Bayesian Networks (BNs) are widely acknowledged for their ability to uncover relationships and generate causal models for complex interactions. However, their massive computational requirement, when implemented on conventional computers, hinders their usefulness in many critical problem areas e.g., genetic basis of diseases, macro finance, text classification, environment monitoring, etc. We propose a new non-von Neumann technology framework purposefully architected across all layers for solving these problems efficiently through physical equivalence, enabled by emerging nanotechnology. The architecture builds on a probabilistic information representation and multi-domain mixed-signal circuit style, and is tightly coupled to a nanoscale physical layer that spans magnetic and electrical domains. Based on bottom-up device-circuit-architecture simulations, we show up to four orders of magnitude performance improvement (using computational resolution of 0.1) vs. best-of-breed multi-core machines with 100 processors, for BNs with about a million variables. Smaller problem sizes of ~100 variables can be realized at 20 mW power consumption and very low area around a few tenths of a mm2. Our vision is to enable solving complex Bayesian problems in real time, as well as enable intelligence capabilities at a small scale everywhere, ushering in a new era of machine intelligence
On FPGA implementations for bioinformatics, neural prosthetics and reinforcement learning problems.
Mak Sui Tung Terrence.Thesis (M.Phil.)--Chinese University of Hong Kong, 2005.Includes bibliographical references (leaves 132-142).Abstracts in English and Chinese.Abstract --- p.iList of Tables --- p.ivList of Figures --- p.vAcknowledgements --- p.ixChapter 1. --- Introduction --- p.1Chapter 1.1 --- Bioinformatics --- p.1Chapter 1.2 --- Neural Prosthetics --- p.4Chapter 1.3 --- Learning in Uncertainty --- p.5Chapter 1.4 --- The Field Programmable Gate Array (FPGAs) --- p.7Chapter 1.5 --- Scope of the Thesis --- p.10Chapter 2. --- A Hybrid GA-DP Approach for Searching Equivalence Sets --- p.14Chapter 2.1 --- Introduction --- p.16Chapter 2.2 --- Equivalence Set Criterion --- p.18Chapter 2.3 --- Genetic Algorithm and Dynamic Programming --- p.19Chapter 2.3.1 --- Genetic Algorithm Formulation --- p.20Chapter 2.3.2 --- Bounded Mutation --- p.21Chapter 2.3.3 --- Conditioned Crossover --- p.22Chapter 2.3.4 --- Implementation --- p.22Chapter 2.4 --- FPGAs Implementation of GA-DP --- p.24Chapter 2.4.1 --- System Overview --- p.25Chapter 2.4.2 --- Parallel Computation for Transitive Closure --- p.26Chapter 2.4.3 --- Genetic Operation Realization --- p.28Chapter 2.5 --- Discussion --- p.30Chapter 2.6 --- Limitation and Future Work --- p.33Chapter 2.7 --- Conclusion --- p.34Chapter 3. --- An FPGA-based Architecture for Maximum-Likelihood Phylogeny Evaluation --- p.35Chapter 3.1 --- Introduction --- p.36Chapter 3.2 --- Maximum-Likelihood Model --- p.39Chapter 3.3 --- Hardware Mapping for Pruning Algorithm --- p.41Chapter 3.3.1 --- Related Works --- p.41Chapter 3.3.2 --- Number Representation --- p.42Chapter 3.3.3 --- Binary Tree Representation --- p.43Chapter 3.3.4 --- Binary Tree Traversal --- p.45Chapter 3.3.5 --- Maximum-Likelihood Evaluation Algorithm --- p.46Chapter 3.4 --- System Architecture --- p.49Chapter 3.4.1 --- Transition Probability Unit --- p.50Chapter 3.4.2 --- State-Parallel Computation Unit --- p.51Chapter 3.4.3 --- Error Computation --- p.54Chapter 3.5 --- Discussion --- p.56Chapter 3.5.1 --- Hardware Resource Consumption --- p.56Chapter 3.5.2 --- Delay Evaluation --- p.57Chapter 3.6 --- Conclusion --- p.59Chapter 4. --- Field Programmable Gate Array Implementation of Neuronal Ion Channel Dynamics --- p.61Chapter 4.1 --- Introduction --- p.62Chapter 4.2 --- Background --- p.63Chapter 4.2.1 --- Analog VLSI Model for Hebbian Synapse --- p.63Chapter 4.2.2 --- A Unifying Model of Bi-directional Synaptic Plasticity --- p.64Chapter 4.2.3 --- Non-NMDA Receptor Channel Regulation --- p.65Chapter 4.3 --- FPGAs Implementation --- p.65Chapter 4.3.1 --- FPGA Design Flow --- p.65Chapter 4.3.2 --- Digital Model of NMD A and AMPA receptors --- p.65Chapter 4.3.3 --- Synapse Modification --- p.67Chapter 4.4 --- Results --- p.68Chapter 4.4.1 --- Simulation Results --- p.68Chapter 4.5 --- Discussion --- p.70Chapter 4.6 --- Conclusion --- p.71Chapter 5. --- Continuous-Time and Discrete-Time Inference Networks for Distributed Dynamic Programming --- p.72Chapter 5.1 --- Introduction --- p.74Chapter 5.2 --- Background --- p.77Chapter 5.2.1 --- Markov decision process (MDPs) --- p.78Chapter 5.2.2 --- Learning in the MDPs --- p.80Chapter 5.2.3 --- Bellman Optimal Criterion --- p.80Chapter 5.2.4 --- Value Iteration --- p.81Chapter 5.3 --- A Computational Framework for Continuous-Time Inference Network --- p.82Chapter 5.3.1 --- Binary Relation Inference Network --- p.83Chapter 5.3.2 --- Binary Relation Inference Network for MDPs --- p.85Chapter 5.3.3 --- Continuous-Time Inference Network for MDPs --- p.87Chapter 5.4 --- Convergence Consideration --- p.88Chapter 5.5 --- Numerical Simulation --- p.90Chapter 5.5.1 --- Example 1: Random Walk --- p.90Chapter 5.5.2 --- Example 2: Random Walk on a Grid --- p.94Chapter 5.5.3 --- Example 3: Stochastic Shortest Path Problem --- p.97Chapter 5.5.4 --- Relationships Between λ and γ --- p.99Chapter 5.6 --- Discrete-Time Inference Network --- p.100Chapter 5.6.1 --- Results --- p.101Chapter 5.7 --- Conclusion --- p.102Chapter 6. --- On Distributed g-Learning Network --- p.104Chapter 6.1 --- Introduction --- p.105Chapter 6.2 --- Distributed Q-Learniing Network --- p.108Chapter 6.2.1 --- Distributed Q-Learning Network --- p.109Chapter 6.2.2 --- Q-Learning Network Architecture --- p.111Chapter 6.3 --- Experimental Results --- p.114Chapter 6.3.1 --- Random Walk --- p.114Chapter 6.3.2 --- The Shortest Path Problem --- p.116Chapter 6.4 --- Discussion --- p.120Chapter 6.4.1 --- Related Work --- p.121Chapter 6.5 --- FPGAs Implementation --- p.122Chapter 6.5.1 --- Distributed Registering Approach --- p.123Chapter 6.5.2 --- Serial BRAM Storing Approach --- p.124Chapter 6.5.3 --- Comparison --- p.125Chapter 6.5.4 --- Discussion --- p.127Chapter 6.6 --- Conclusion --- p.128Chapter 7. --- Summary --- p.129Bibliography --- p.132AppendixChapter A. --- Simplified Floating-Point Arithmetic --- p.143Chapter B. --- "Logarithm, Exponential and Division Implementation" --- p.144Chapter B.1 --- Introduction --- p.144Chapter B.2 --- Approximation Scheme --- p.145Chapter B.2.1 --- Logarithm --- p.145Chapter B.2.2 --- Exponentiation --- p.147Chapter B.2.3 --- Division --- p.148Chapter C. --- Analog VLSI Implementation --- p.150Chapter C.1 --- Site Function --- p.150Chapter C.1.1 --- Multiplication Cell --- p.150Chapter C.2 --- The Unit Function --- p.153Chapter C.3 --- The Inference Network Computation --- p.154Chapter C.4 --- Layout --- p.157Chapter C.5 --- Fabrication --- p.159Chapter C.5.1 --- Testing and Characterization --- p.16
Hierarchical Temporal Memory using Memristor Networks: A Survey
This paper presents a survey of the currently available hardware designs for
implementation of the human cortex inspired algorithm, Hierarchical Temporal
Memory (HTM). In this review, we focus on the state of the art advances of
memristive HTM implementation and related HTM applications. With the advent of
edge computing, HTM can be a potential algorithm to implement on-chip near
sensor data processing. The comparison of analog memristive circuit
implementations with the digital and mixed-signal solutions are provided. The
advantages of memristive HTM over digital implementations against performance
metrics such as processing speed, reduced on-chip area and power dissipation
are discussed. The limitations and open problems concerning the memristive HTM,
such as the design scalability, sneak currents, leakage, parasitic effects,
lack of the analog learning circuits implementations and unreliability of the
memristive devices integrated with CMOS circuits are also discussed
Applications and implementation of neuro-connectionist architectures.
by H.S. Ng.Thesis (M.Phil.)--Chinese University of Hong Kong, 1996.Includes bibliographical references (leaves 91-97).Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Introduction --- p.1Chapter 1.2 --- Neuro-connectionist Network --- p.2Chapter 2 --- Related Works --- p.5Chapter 2.1 --- Introduction --- p.5Chapter 2.1.1 --- Kruskal's Algorithm --- p.5Chapter 2.1.2 --- Prim's algorithm --- p.6Chapter 2.1.3 --- Sollin's algorithm --- p.7Chapter 2.1.4 --- Bellman-Ford algorithm --- p.8Chapter 2.1.5 --- Floyd-Warshall algorithm --- p.9Chapter 3 --- Binary Relation Inference Network and Path Problems --- p.11Chapter 3.1 --- Introduction --- p.11Chapter 3.2 --- Topology --- p.12Chapter 3.3 --- Network structure --- p.13Chapter 3.3.1 --- Single-destination BRIN architecture --- p.14Chapter 3.3.2 --- Comparison between all-pair BRIN and single-destination BRIN --- p.18Chapter 3.4 --- Path Problems and BRIN Solution --- p.18Chapter 3.4.1 --- Minimax path problems --- p.18Chapter 3.4.2 --- BRIN solution --- p.19Chapter 4 --- Analog and Voltage-mode Approach --- p.22Chapter 4.1 --- Introduction --- p.22Chapter 4.2 --- Analog implementation --- p.24Chapter 4.3 --- Voltage-mode approach --- p.26Chapter 4.3.1 --- The site function --- p.26Chapter 4.3.2 --- The unit function --- p.28Chapter 4.3.3 --- The computational unit --- p.28Chapter 4.4 --- Conclusion --- p.29Chapter 5 --- Current-mode Approach --- p.32Chapter 5.1 --- Introduction --- p.32Chapter 5.2 --- Current-mode approach for analog VLSI Implementation --- p.33Chapter 5.2.1 --- Site and Unit output function --- p.33Chapter 5.2.2 --- Computational unit --- p.34Chapter 5.2.3 --- A complete network --- p.35Chapter 5.3 --- Conclusion --- p.37Chapter 6 --- Neural Network Compensation for Optimization Circuit --- p.40Chapter 6.1 --- Introduction --- p.40Chapter 6.2 --- A Neuro-connectionist Architecture for error correction --- p.41Chapter 6.2.1 --- Linear Relationship --- p.42Chapter 6.2.2 --- Output Deviation of Computational Unit --- p.44Chapter 6.3 --- Experimental Results --- p.46Chapter 6.3.1 --- Training Phase --- p.46Chapter 6.3.2 --- Generalization Phase --- p.48Chapter 6.4 --- Conclusion --- p.50Chapter 7 --- Precision-limited Analog Neural Network Compensation --- p.51Chapter 7.1 --- Introduction --- p.51Chapter 7.2 --- Analog Neural Network hardware --- p.53Chapter 7.3 --- Integration of analog neural network compensation of connectionist net- work for general path problems --- p.54Chapter 7.4 --- Experimental Results --- p.55Chapter 7.4.1 --- Convergence time --- p.56Chapter 7.4.2 --- The accuracy of the system --- p.57Chapter 7.5 --- Conclusion --- p.58Chapter 8 --- Transitive Closure Problems --- p.60Chapter 8.1 --- Introduction --- p.60Chapter 8.2 --- Different ways of implementation of BRIN for transitive closure --- p.61Chapter 8.2.1 --- Digital Implementation --- p.61Chapter 8.2.2 --- Analog Implementation --- p.61Chapter 8.3 --- Transitive Closure Problem --- p.63Chapter 8.3.1 --- A special case of maximum spanning tree problem --- p.64Chapter 8.3.2 --- Analog approach solution for transitive closure problem --- p.65Chapter 8.3.3 --- Current-mode approach solution for transitive closure problem --- p.67Chapter 8.4 --- Comparisons between the different forms of implementation of BRIN for transitive closure --- p.71Chapter 8.4.1 --- Convergence Time --- p.71Chapter 8.4.2 --- Circuit complexity --- p.72Chapter 8.5 --- Discussion --- p.73Chapter 9 --- Critical path problems --- p.74Chapter 9.1 --- Introduction --- p.74Chapter 9.2 --- Problem statement and single-destination BRIN solution --- p.75Chapter 9.3 --- Analog implementation --- p.76Chapter 9.3.1 --- Separated building block --- p.78Chapter 9.3.2 --- Combined building block --- p.79Chapter 9.4 --- Current-mode approach --- p.80Chapter 9.4.1 --- "Site function, unit output function and a completed network" --- p.80Chapter 9.5 --- Conclusion --- p.83Chapter 10 --- Conclusions --- p.85Chapter 10.1 --- Summary of Achievements --- p.85Chapter 10.2 --- Future development --- p.88Chapter 10.2.1 --- Application for financial problems --- p.88Chapter 10.2.2 --- Fabrication of VLSI Implementation --- p.88Chapter 10.2.3 --- Actual prototyping of Analog Integrated Circuits for critical path and transitive closure problems --- p.89Chapter 10.2.4 --- Other implementation platform --- p.89Chapter 10.2.5 --- On-line update of routing table inside the router for network com- munication using BRIN --- p.89Chapter 10.2.6 --- Other BRIN's applications --- p.90Bibliography --- p.9
Applications and implementation of neuro-connectionist architectures.
by H.S. Ng.Thesis (M.Phil.)--Chinese University of Hong Kong, 1996.Includes bibliographical references (leaves 91-97).Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Introduction --- p.1Chapter 1.2 --- Neuro-connectionist Network --- p.2Chapter 2 --- Related Works --- p.5Chapter 2.1 --- Introduction --- p.5Chapter 2.1.1 --- Kruskal's Algorithm --- p.5Chapter 2.1.2 --- Prim's algorithm --- p.6Chapter 2.1.3 --- Sollin's algorithm --- p.7Chapter 2.1.4 --- Bellman-Ford algorithm --- p.8Chapter 2.1.5 --- Floyd-Warshall algorithm --- p.9Chapter 3 --- Binary Relation Inference Network and Path Problems --- p.11Chapter 3.1 --- Introduction --- p.11Chapter 3.2 --- Topology --- p.12Chapter 3.3 --- Network structure --- p.13Chapter 3.3.1 --- Single-destination BRIN architecture --- p.14Chapter 3.3.2 --- Comparison between all-pair BRIN and single-destination BRIN --- p.18Chapter 3.4 --- Path Problems and BRIN Solution --- p.18Chapter 3.4.1 --- Minimax path problems --- p.18Chapter 3.4.2 --- BRIN solution --- p.19Chapter 4 --- Analog and Voltage-mode Approach --- p.22Chapter 4.1 --- Introduction --- p.22Chapter 4.2 --- Analog implementation --- p.24Chapter 4.3 --- Voltage-mode approach --- p.26Chapter 4.3.1 --- The site function --- p.26Chapter 4.3.2 --- The unit function --- p.28Chapter 4.3.3 --- The computational unit --- p.28Chapter 4.4 --- Conclusion --- p.29Chapter 5 --- Current-mode Approach --- p.32Chapter 5.1 --- Introduction --- p.32Chapter 5.2 --- Current-mode approach for analog VLSI Implementation --- p.33Chapter 5.2.1 --- Site and Unit output function --- p.33Chapter 5.2.2 --- Computational unit --- p.34Chapter 5.2.3 --- A complete network --- p.35Chapter 5.3 --- Conclusion --- p.37Chapter 6 --- Neural Network Compensation for Optimization Circuit --- p.40Chapter 6.1 --- Introduction --- p.40Chapter 6.2 --- A Neuro-connectionist Architecture for error correction --- p.41Chapter 6.2.1 --- Linear Relationship --- p.42Chapter 6.2.2 --- Output Deviation of Computational Unit --- p.44Chapter 6.3 --- Experimental Results --- p.46Chapter 6.3.1 --- Training Phase --- p.46Chapter 6.3.2 --- Generalization Phase --- p.48Chapter 6.4 --- Conclusion --- p.50Chapter 7 --- Precision-limited Analog Neural Network Compensation --- p.51Chapter 7.1 --- Introduction --- p.51Chapter 7.2 --- Analog Neural Network hardware --- p.53Chapter 7.3 --- Integration of analog neural network compensation of connectionist net- work for general path problems --- p.54Chapter 7.4 --- Experimental Results --- p.55Chapter 7.4.1 --- Convergence time --- p.56Chapter 7.4.2 --- The accuracy of the system --- p.57Chapter 7.5 --- Conclusion --- p.58Chapter 8 --- Transitive Closure Problems --- p.60Chapter 8.1 --- Introduction --- p.60Chapter 8.2 --- Different ways of implementation of BRIN for transitive closure --- p.61Chapter 8.2.1 --- Digital Implementation --- p.61Chapter 8.2.2 --- Analog Implementation --- p.61Chapter 8.3 --- Transitive Closure Problem --- p.63Chapter 8.3.1 --- A special case of maximum spanning tree problem --- p.64Chapter 8.3.2 --- Analog approach solution for transitive closure problem --- p.65Chapter 8.3.3 --- Current-mode approach solution for transitive closure problem --- p.67Chapter 8.4 --- Comparisons between the different forms of implementation of BRIN for transitive closure --- p.71Chapter 8.4.1 --- Convergence Time --- p.71Chapter 8.4.2 --- Circuit complexity --- p.72Chapter 8.5 --- Discussion --- p.73Chapter 9 --- Critical path problems --- p.74Chapter 9.1 --- Introduction --- p.74Chapter 9.2 --- Problem statement and single-destination BRIN solution --- p.75Chapter 9.3 --- Analog implementation --- p.76Chapter 9.3.1 --- Separated building block --- p.78Chapter 9.3.2 --- Combined building block --- p.79Chapter 9.4 --- Current-mode approach --- p.80Chapter 9.4.1 --- "Site function, unit output function and a completed network" --- p.80Chapter 9.5 --- Conclusion --- p.83Chapter 10 --- Conclusions --- p.85Chapter 10.1 --- Summary of Achievements --- p.85Chapter 10.2 --- Future development --- p.88Chapter 10.2.1 --- Application for financial problems --- p.88Chapter 10.2.2 --- Fabrication of VLSI Implementation --- p.88Chapter 10.2.3 --- Actual prototyping of Analog Integrated Circuits for critical path and transitive closure problems --- p.89Chapter 10.2.4 --- Other implementation platform --- p.89Chapter 10.2.5 --- On-line update of routing table inside the router for network com- munication using BRIN --- p.89Chapter 10.2.6 --- Other BRIN's applications --- p.90Bibliography --- p.9
Large-Scale Optical Neural Networks based on Photoelectric Multiplication
Recent success in deep neural networks has generated strong interest in
hardware accelerators to improve speed and energy consumption. This paper
presents a new type of photonic accelerator based on coherent detection that is
scalable to large () networks and can be operated at high (GHz)
speeds and very low (sub-aJ) energies per multiply-and-accumulate (MAC), using
the massive spatial multiplexing enabled by standard free-space optical
components. In contrast to previous approaches, both weights and inputs are
optically encoded so that the network can be reprogrammed and trained on the
fly. Simulations of the network using models for digit- and
image-classification reveal a "standard quantum limit" for optical neural
networks, set by photodetector shot noise. This bound, which can be as low as
50 zJ/MAC, suggests performance below the thermodynamic (Landauer) limit for
digital irreversible computation is theoretically possible in this device. The
proposed accelerator can implement both fully-connected and convolutional
networks. We also present a scheme for back-propagation and training that can
be performed in the same hardware. This architecture will enable a new class of
ultra-low-energy processors for deep learning.Comment: Text: 10 pages, 5 figures, 1 table. Supplementary: 8 pages, 5,
figures, 2 table
- …