Rubik's Optical Neural Networks: Multi-task Learning with Physics-aware Rotation Architecture

Recently, there are increasing efforts on advancing optical neural networks (ONNs), which bring significant advantages for machine learning (ML) in terms of power efficiency, parallelism, and computational speed. With the considerable benefits in computation speed and energy efficiency, there are significant interests in leveraging ONNs into medical sensing, security screening, drug detection, and autonomous driving. However, due to the challenge of implementing reconfigurability, deploying multi-task learning (MTL) algorithms on ONNs requires re-building and duplicating the physical diffractive systems, which significantly degrades the energy and cost efficiency in practical application scenarios. This work presents a novel ONNs architecture, namely, \textit{RubikONNs}, which utilizes the physical properties of optical systems to encode multiple feed-forward functions by physically rotating the hardware similarly to rotating a \textit{Rubik's Cube}. To optimize MTL performance on RubikONNs, two domain-specific physics-aware training algorithms \textit{RotAgg} and \textit{RotSeq} are proposed. Our experimental results demonstrate more than 4$\times$ improvements in energy and cost efficiency with marginal accuracy degradation compared to the state-of-the-art approaches

Perfectly Perform Machine Learning Task with Imperfect Optical Hardware Accelerator

Optical architectures have been emerging as an energy-efficient and high-throughput hardware platform to accelerate computationally intensive general matrix-matrix multiplications (GEMMs) in modern machine learning (ML) algorithms. However, the inevitable imperfection and non-uniformity in large-scale optoelectronic devices prevent the scalable deployment of optical architectures, particularly those with innovative nano-devices. Here, we report an optical ML hardware to accelerate GEMM operations based on cascaded spatial light modulators and present a calibration procedure that enables accurate calculations despite the non-uniformity and imperfection in devices and system. We further characterize the hardware calculation accuracy under different configurations of electrical-optical interfaces. Finally, we deploy the developed optical hardware and calibration procedure to perform a ML task of predicting the intersubband plasmon frequency in single-wall carbon nanotubes. The obtained prediction accuracy from the optical hardware agrees well with that obtained using a general purpose electronic graphic process unit

Artificial Intelligence Accelerators based on Graphene Optoelectronic Devices

Optical and optoelectronic approaches of performing matrix-vector multiplication (MVM) operations have shown the great promise of accelerating machine learning (ML) algorithms with unprecedented performance. The incorporation of nanomaterials into the system can further improve the performance thanks to their extraordinary properties, but the non-uniformity and variation of nanostructures in the macroscopic scale pose severe limitations for large-scale hardware deployment. Here, we report a new optoelectronic architecture consisting of spatial light modulators and photodetector arrays made from graphene to perform MVM. The ultrahigh carrier mobility of graphene, nearly-zero-power-consumption electro-optic control, and extreme parallelism suggest ultrahigh data throughput and ultralow-power consumption. Moreover, we develop a methodology of performing accurate calculations with imperfect components, laying the foundation for scalable systems. Finally, we perform a few representative ML algorithms, including singular value decomposition, support vector machine, and deep neural networks, to show the versatility and generality of our platform

Additional file 1: of Destabilization of Surfactant-Dispersed Carbon Nanotubes by Anions

Experimental Results. (DOCX 636 kb

Supplementary document for Design and fabrication of terahertz dual-plane hologram and extended-depth-of-focus diffractive lens - 5949321.pdf

For 466008 manuscrip

Scientific Computing with Diffractive Optical Neural Networks

Diffractive optical neural networks (DONNs) have been emerging as a high-throughput and energy-efficient hardware platform to perform all-optical machine learning (ML) in machine vision systems. However, the current demonstrated applications of DONNs are largely straightforward image classification tasks, which undermines the prospect of developing and utilizing such hardware for other ML applications. Here, we numerically and experimentally demonstrate the deployment of an all-optical reconfigurable DONNs system for scientific computing, including guiding two-dimensional quantum material synthesis, predicting the properties of nanomaterials and small molecular cancer drugs, predicting the device response of nanopatterned integrated photonic power splitters, and the dynamic stabilization of an inverted pendulum with reinforcement learning. Despite a large variety of input data structures, we develop a universal feature engineering approach to convert categorical input features to the images that can be processed in the DONNs system. Our results open up new opportunities of employing DONNs systems for a broad range of ML applications

Real-time Multi-Task Diffractive Deep Neural Networks via Hardware-Software Co-design

Deep neural networks (DNNs) have substantial computational requirements, which greatly limit their performance in resource-constrained environments. Recently, there are increasing efforts on optical neural networks and optical computing based DNNs hardware, which bring significant advantages for deep learning systems in terms of their power efficiency, parallelism and computational speed. Among them, free-space diffractive deep neural networks (D$^2$NNs) based on the light diffraction, feature millions of neurons in each layer interconnected with neurons in neighboring layers. However, due to the challenge of implementing reconfigurability, deploying different DNNs algorithms requires re-building and duplicating the physical diffractive systems, which significantly degrades the hardware efficiency in practical application scenarios. Thus, this work proposes a novel hardware-software co-design method that enables robust and noise-resilient Multi-task Learning in D$^2$NNs. Our experimental results demonstrate significant improvements in versatility and hardware efficiency, and also demonstrate the robustness of proposed multi-task D$^2$NN architecture under wide noise ranges of all system components. In addition, we propose a domain-specific regularization algorithm for training the proposed multi-task architecture, which can be used to flexibly adjust the desired performance for each task

Macroscopically Aligned Carbon Nanotubes as a Refractory Platform for Hyperbolic Thermal Emitters

Nanophotonic thermal emitters with large photonic density of states (PDOS) have the potential to significantly enhance the efficiency of radiative cooling and waste heat recovery. Because of their nearly infinite PDOS, refractory hyperbolic materials make a promising material platform for thermal emitters. However, it is challenging to achieve a prominent PDOS in existing refractory hyperbolic materials, especially in a broad bandwidth. Here, we demonstrate macroscopically aligned carbon nanotubes as an excellent refractory material platform for hyperbolic nanophotonic devices. Aligned carbon nanotubes are thermally stable up to 1600 °C and exhibit extreme anisotropy: metallic in one direction and insulating in the other two directions. Such extreme anisotropy results in an exceptionally large PDOS over a broadband spectrum range (longer than 4.3 μm) in the mid-infrared, manifesting as strong resonances in deeply subwavelength-sized cavities. We demonstrate polarized, spectrally selective, thermal emission from aligned carbon nanotube films and indefinite cavities of volume as small as ∼λ3/700 operating at 700 °C. These experiments suggest that aligned carbon nanotubes enhance PDOS and hence also thermal photon density by over 2 orders of magnitude, making them a promising refractory nanophotonics platform

LightRidge: An End-to-end Agile Design Framework for Diffractive Optical Neural Networks

To lower the barrier to diffractive optical neural networks (DONNs) design, exploration, and deployment, we propose LightRidge, the first end-to-end optical ML compilation framework, which consists of (1) precise and differentiable optical physics kernels that enable complete explorations of DONNs architectures, (2) optical physics computation kernel acceleration that significantly reduces the runtime cost in training, emulation, and deployment of DONNs, and (3) versatile and flexible optical system modeling and user-friendly domain-specific-language (DSL). As a result, LightRidge framework enables efficient end-to-end design and deployment of DONNs, and significantly reduces the efforts for programming, hardware-software codesign, and chip integration. Our results are experimentally conducted with physical optical systems, where we demonstrate: (1) the optical physics kernels precisely correlated to low-level physics and systems, (2) significant speedups in runtime with physics-aware emulation workloads compared to the state-of-the-art commercial system, (3) effective architectural design space exploration verified by the hardware prototype and on-chip integration case study, and (4) novel DONN design principles including successful demonstrations of advanced image classification and image segmentation task using DONNs architecture and topology

Physics-Guided and Physics-Explainable Recurrent Neural Network for Time Dynamics in Optical Resonances

Understanding the time evolution of physical systems is crucial to revealing fundamental characteristics that are hidden in frequency domain. In optical science, high-quality resonance cavities and enhanced interactions with matters are at the heart of modern quantum technologies. However, capturing their time dynamics in real-world scenarios suffers from long data acquisition and low analysis accuracy due to slow convergence and limited time window. Here, we report a physics-guided and physics-explainable recurrent neural network to precisely forecast the time-domain response of resonance features with the shortest acquired input sequence being 7\% of full length, and to infer corresponding resonance frequencies. The model is trained in a two-step multi-fidelity framework for high-accuracy forecast, where the first step is based on a large amount of low-fidelity physical-model-generated synthetic data and second step involves a small set of high-fidelity application-oriented observational data. Through both simulations and experiments, we demonstrate that the model is universally applicable to a wide range of resonances, including dielectric metasurfaces, graphene plasmonics, and ultrastrongly coupled Landau polaritons, where our model accurately captures small signal features and learns essential physical quantities. The demonstrated machine learning algorithm offers a new way to accelerate the exploration of physical phenomena and the design of devices under resonance-enhanced light-matter interaction