468 research outputs found
BPLight-CNN: A Photonics-based Backpropagation Accelerator for Deep Learning
Training deep learning networks involves continuous weight updates across the
various layers of the deep network while using a backpropagation algorithm
(BP). This results in expensive computation overheads during training.
Consequently, most deep learning accelerators today employ pre-trained weights
and focus only on improving the design of the inference phase. The recent trend
is to build a complete deep learning accelerator by incorporating the training
module. Such efforts require an ultra-fast chip architecture for executing the
BP algorithm. In this article, we propose a novel photonics-based
backpropagation accelerator for high performance deep learning training. We
present the design for a convolutional neural network, BPLight-CNN, which
incorporates the silicon photonics-based backpropagation accelerator.
BPLight-CNN is a first-of-its-kind photonic and memristor-based CNN
architecture for end-to-end training and prediction. We evaluate BPLight-CNN
using a photonic CAD framework (IPKISS) on deep learning benchmark models
including LeNet and VGG-Net. The proposed design achieves (i) at least 34x
speedup, 34x improvement in computational efficiency, and 38.5x energy savings,
during training; and (ii) 29x speedup, 31x improvement in computational
efficiency, and 38.7x improvement in energy savings, during inference compared
to the state-of-the-art designs. All these comparisons are done at a 16-bit
resolution; and BPLight-CNN achieves these improvements at a cost of
approximately 6% lower accuracy compared to the state-of-the-art
InP photonic integrated multi-layer neural networks:Architecture and performance analysis
We demonstrate the use of a wavelength converter, based on cross-gain modulation in a semiconductor optical amplifier (SOA), as a nonlinear function co-integrated within an all-optical neuron realized with SOA and wavelength-division multiplexing technology. We investigate the impact of fully monolithically integrated linear and nonlinear functions on the all-optical neuron output with respect to the number of synapses/neuron and data rate. Results suggest that the number of inputs can scale up to 64 while guaranteeing a large input power dynamic range of 36 dB with neglectable error introduction. We also investigate the performance of its nonlinear transfer function by tuning the total input power and data rate: The monolithically integrated neuron performs about 10% better in accuracy than the corresponding hybrid device for the same data rate. These all-optical neurons are then used to simulate a 64:64:10 two-layer photonic deep neural network for handwritten digit classification, which shows an 89.5% best-case accuracy at 10 GS/s. Moreover, we analyze the energy consumption for synaptic operation, considering the full end-to-end system, which includes the transceivers, the optical neural network, and the electrical control part. This investigation shows that when the number of synapses/neuron is >18, the energy per operation is <20 pJ (6 times higher than when considering only the optical engine). The computation speed of this two-layer all-optical neural network system is 47 TMAC/s, 2.5 times faster than state-of-the-art graphics processing units, while the energy efficiency is 12 pJ/MAC, 2 times better. This result underlines the importance of scaling photonic integrated neural networks on chip
Scalability Analysis of the SOA-based All-optical Deep Neural Network
In this work we propose a noise model to investigate the scaling of the SOA-based all-optical deep neural networks regarding the number of WDM inputs and the cascading layers. The model is validated experimentally by emulating the OSNR evolution of the all-optical neuron. The results show that our all-optical neuron structure can be interconnected to establish a 16-input/neuron 16-neuron/layer 10-layer all-optical neural network with minor accuracy degradation for image classification
Photonic Neural Networks and Optics-informed Deep Learning Fundamentals
The recent explosive compute growth, mainly fueled by the boost of AI and
DNNs, is currently instigating the demand for a novel computing paradigm that
can overcome the insurmountable barriers imposed by conventional electronic
computing architectures. PNNs implemented on silicon integration platforms
stand out as a promising candidate to endow NN hardware, offering the potential
for energy efficient and ultra-fast computations through the utilization of the
unique primitives of photonics i.e. energy efficiency, THz bandwidth and
low-latency. Thus far, several demonstrations have revealed the huge potential
of PNNs in performing both linear and non-linear NN operations at unparalleled
speed and energy consumption metrics. Transforming this potential into a
tangible reality for DL applications requires, however, a deep understanding of
the basic PNN principles, requirements and challenges across all constituent
architectural, technological and training aspects. In this tutorial, we,
initially, review the principles of DNNs along with their fundamental building
blocks, analyzing also the key mathematical operations needed for their
computation in a photonic hardware. Then, we investigate, through an intuitive
mathematical analysis, the interdependence of bit precision and energy
efficiency in analog photonic circuitry, discussing the opportunities and
challenges of PNNs. Followingly, a performance overview of PNN architectures,
weight technologies and activation functions is presented, summarizing their
impact in speed, scalability and power consumption. Finally, we provide an
holistic overview of the optics-informed NN training framework that
incorporates the physical properties of photonic building blocks into the
training process in order to improve the NN classification accuracy and
effectively elevate neuromorphic photonic hardware into high-performance DL
computational settings
Toward optical signal processing using photonic reservoir computing
We propose photonic reservoir computing as a new approach to optical signal processing in the context of large scale pattern recognition problems. Photonic reservoir computing is a photonic implementation of the recently proposed reservoir computing concept, where the dynamics of a network of nonlinear elements are exploited to perform general signal processing tasks. In our proposed photonic implementation, we employ a network of coupled Semiconductor Optical Amplifiers (SOA) as the basic building blocks for the reservoir. Although they differ in many key respects from traditional software-based hyperbolic tangent reservoirs, we show using simulations that such a photonic reservoir can outperform traditional reservoirs on a benchmark classification task. Moreover, a photonic implementation offers the promise of massively parallel information processing with low power and high speed. (C) 2008 Optical Society of America
Hardware-algorithm collaborative computing with photonic spiking neuron chip based on integrated Fabry-P\'erot laser with saturable absorber
Photonic neuromorphic computing has emerged as a promising avenue toward
building a low-latency and energy-efficient non-von-Neuman computing system.
Photonic spiking neural network (PSNN) exploits brain-like spatiotemporal
processing to realize high-performance neuromorphic computing. However, the
nonlinear computation of PSNN remains a significant challenging. Here, we
proposed and fabricated a photonic spiking neuron chip based on an integrated
Fabry-P\'erot laser with a saturable absorber (FP-SA) for the first time. The
nonlinear neuron-like dynamics including temporal integration, threshold and
spike generation, refractory period, and cascadability were experimentally
demonstrated, which offers an indispensable fundamental building block to
construct the PSNN hardware. Furthermore, we proposed time-multiplexed spike
encoding to realize functional PSNN far beyond the hardware integration scale
limit. PSNNs with single/cascaded photonic spiking neurons were experimentally
demonstrated to realize hardware-algorithm collaborative computing, showing
capability in performing classification tasks with supervised learning
algorithm, which paves the way for multi-layer PSNN for solving complex tasks.Comment: 10 pages, 8 figure
Nanophotonic reservoir computing with photonic crystal cavities to generate periodic patterns
Reservoir computing (RC) is a technique in machine learning inspired by neural systems. RC has been used successfully to solve complex problems such as signal classification and signal generation. These systems are mainly implemented in software, and thereby they are limited in speed and power efficiency. Several optical and optoelectronic implementations have been demonstrated, in which the system has signals with an amplitude and phase. It is proven that these enrich the dynamics of the system, which is beneficial for the performance. In this paper, we introduce a novel optical architecture based on nanophotonic crystal cavities. This allows us to integrate many neurons on one chip, which, compared with other photonic solutions, closest resembles a classical neural network. Furthermore, the components are passive, which simplifies the design and reduces the power consumption. To assess the performance of this network, we train a photonic network to generate periodic patterns, using an alternative online learning rule called first-order reduced and corrected error. For this, we first train a classical hyperbolic tangent reservoir, but then we vary some of the properties to incorporate typical aspects of a photonics reservoir, such as the use of continuous-time versus discrete-time signals and the use of complex-valued versus real-valued signals. Then, the nanophotonic reservoir is simulated and we explore the role of relevant parameters such as the topology, the phases between the resonators, the number of nodes that are biased and the delay between the resonators. It is important that these parameters are chosen such that no strong self-oscillations occur. Finally, our results show that for a signal generation task a complex-valued, continuous-time nanophotonic reservoir outperforms a classical (i.e., discrete-time, real-valued) leaky hyperbolic tangent reservoir (normalized root-mean-square errors = 0.030 versus NRMSE = 0.127)
- …