Quantum-inspired computational imaging

Computational imaging combines measurement and computational methods with the aim of forming images even when the measurement conditions are weak, few in number, or highly indirect. The recent surge in quantum-inspired imaging sensors, together with a new wave of algorithms allowing on-chip, scalable and robust data processing, has induced an increase of activity with notable results in the domain of low-light flux imaging and sensing. We provide an overview of the major challenges encountered in low-illumination (e.g., ultrafast) imaging and how these problems have recently been addressed for imaging applications in extreme conditions. These methods provide examples of the future imaging solutions to be developed, for which the best results are expected to arise from an efficient codesign of the sensors and data analysis tools.Y.A. acknowledges support from the UK Royal Academy of Engineering under the Research Fellowship Scheme (RF201617/16/31). S.McL. acknowledges financial support from the UK Engineering and Physical Sciences Research Council (grant EP/J015180/1). V.G. acknowledges support from the U.S. Defense Advanced Research Projects Agency (DARPA) InPho program through U.S. Army Research Office award W911NF-10-1-0404, the U.S. DARPA REVEAL program through contract HR0011-16-C-0030, and U.S. National Science Foundation through grants 1161413 and 1422034. A.H. acknowledges support from U.S. Army Research Office award W911NF-15-1-0479, U.S. Department of the Air Force grant FA8650-15-D-1845, and U.S. Department of Energy National Nuclear Security Administration grant DE-NA0002534. D.F. acknowledges financial support from the UK Engineering and Physical Sciences Research Council (grants EP/M006514/1 and EP/M01326X/1). (RF201617/16/31 - UK Royal Academy of Engineering; EP/J015180/1 - UK Engineering and Physical Sciences Research Council; EP/M006514/1 - UK Engineering and Physical Sciences Research Council; EP/M01326X/1 - UK Engineering and Physical Sciences Research Council; W911NF-10-1-0404 - U.S. Defense Advanced Research Projects Agency (DARPA) InPho program through U.S. Army Research Office; HR0011-16-C-0030 - U.S. DARPA REVEAL program; 1161413 - U.S. National Science Foundation; 1422034 - U.S. National Science Foundation; W911NF-15-1-0479 - U.S. Army Research Office; FA8650-15-D-1845 - U.S. Department of the Air Force; DE-NA0002534 - U.S. Department of Energy National Nuclear Security Administration)Accepted manuscrip

Structured Dictionary Learning and its applications in Neural Recording

Widely utilized in the field of neuroscience, implantable neural recording devices could capture neuron activities with an acquisition rate on the order of megabytes per second. In order to efficiently transmit neural signals through wireless channels, these devices require compression methods that reduce power consumption. Although recent Compressed Sensing (CS) approaches have successfully demonstrated their power, their full potential is yet to be explored, particularly towards exploring a more efficient representation of the neural signals. As a promising solution, sparse representation not only provides better signal compression for bandwidth/storage efficiency, but also leads to faster processing algorithms as well as more effective signal separation for classification purpose. However, current sparsity‐based approaches for neural recording are limited due to several critical drawbacks: (i) the lack of an efficient data‐driven representation to fully capture the characteristics of specific neural signal; (ii) most existing methods do not fully explore the prior knowledge of neural signals (e.g., labels), while such information is often known; and (iii) the capability to encode discriminative information into the representation to promote classification. Using neural recording as a case study, this dissertation presents new theoretical ideas and mathematical frameworks on structured dictionary learning with applications in compression and classification. Start with a single task setup, we provide theoretical proofs to show the benefits of using structured sparsity in dictionary learning. Then we provide various novel models for the representation of a single measurement, as well as multiple measurements where signals exhibit both with‐in class similarity as well as with‐in class difference. Under the assumption that the label information of the neural signal is known, the proposed models minimize the data fidelity term together with the structured sparsity terms to drive for more discriminative representation. We demonstrate that this is particularly essential in neural recording since it can further improve the compression ratio, classification accuracy and help deal with non‐ideal scenarios such as co-occurrences of neuron firings. Fast and efficient algorithms based on Bayesian inference and alternative direction method are proposed. Extensive experiments are conducted on both neural recording applications as well as some other classification task, such as image classification

High-speed holographic imaging using compressed sensing and phase retrieval

Digital in-line holography serves as a useful encoder for spatial information. This allows three-dimensional reconstruction from a two-dimensional image. This is applicable to the tasks of fast motion capture, particle tracking etc. Sampling high resolution holograms yields a spatiotemporal tradeoff. We spatially subsample holograms to increase temporal resolution. We demonstrate this idea with two subsampling techniques, periodic and uniformly random sampling. The implementation includes an on-chip setup for periodic subsampling and a DMD (Digital Micromirror Device) -based setup for pixel-wise random subsampling. The on-chip setup enables direct increase of up to 20 in camera frame rate. Alternatively, the DMD-based setup encodes temporal information as high-speed mask patterns, and projects these masks within a single exposure (coded exposure). This way, the frame rate is improved to the level of the DMD with a temporal gain of 10. The reconstruction of subsampled data using the aforementioned setups is achieved in two ways. We examine and compare two iterative reconstruction methods. One is an error reduction phase retrieval and the other is sparsity-based compressed sensing algorithm. Both methods show strong capability of reconstructing complex object fields. We present both simulations and real experiments. In the lab, we image and reconstruct structure and movement of static polystyrene microspheres, microscopic moving peranema, macroscopic fast moving fur and glitters

1994 Science Information Management and Data Compression Workshop

This document is the proceedings from the 'Science Information Management and Data Compression Workshop,' which was held on September 26-27, 1994, at the NASA Goddard Space Flight Center, Greenbelt, Maryland. The Workshop explored promising computational approaches for handling the collection, ingestion, archival and retrieval of large quantities of data in future Earth and space science missions. It consisted of eleven presentations covering a range of information management and data compression approaches that are being or have been integrated into actual or prototypical Earth or space science data information systems, or that hold promise for such an application. The workshop was organized by James C. Tilton and Robert F. Cromp of the NASA Goddard Space Flight Center

Ultrafast Radiographic Imaging and Tracking: An overview of instruments, methods, data, and applications

Ultrafast radiographic imaging and tracking (U-RadIT) use state-of-the-art ionizing particle and light sources to experimentally study sub-nanosecond dynamic processes in physics, chemistry, biology, geology, materials science and other fields. These processes, fundamental to nuclear fusion energy, advanced manufacturing, green transportation and others, often involve one mole or more atoms, and thus are challenging to compute by using the first principles of quantum physics or other forward models. One of the central problems in U-RadIT is to optimize information yield through, e.g. high-luminosity X-ray and particle sources, efficient imaging and tracking detectors, novel methods to collect data, and large-bandwidth online and offline data processing, regulated by the underlying physics, statistics, and computing power. We review and highlight recent progress in: a.) Detectors; b.) U-RadIT modalities; c.) Data and algorithms; and d.) Applications. Hardware-centric approaches to U-RadIT optimization are constrained by detector material properties, low signal-to-noise ratio, high cost and long development cycles of critical hardware components such as ASICs. Interpretation of experimental data, including comparisons with forward models, is frequently hindered by sparse measurements, model and measurement uncertainties, and noise. Alternatively, U-RadIT makes increasing use of data science and machine learning algorithms, including experimental implementations of compressed sensing. Machine learning and artificial intelligence approaches, refined by physics and materials information, may also contribute significantly to data interpretation, uncertainty quantification and U-RadIT optimization.Comment: 51 pages, 31 figures; Overview of ultrafast radiographic imaging and tracking as a part of ULITIMA 2023 conference, Mar. 13-16,2023, Menlo Park, CA, US

Robust inversion and detection techniques for improved imaging performance

Thesis (Ph.D.)--Boston UniversityIn this thesis we aim to improve the performance of information extraction from imaging systems through three thrusts. First, we develop improved image formation methods for physics-based, complex-valued sensing problems. We propose a regularized inversion method that incorporates prior information about the underlying field into the inversion framework for ultrasound imaging. We use experimental ultrasound data to compute inversion results with the proposed formulation and compare it with conventional inversion techniques to show the robustness of the proposed technique to loss of data. Second, we propose methods that combine inversion and detection in a unified framework to improve imaging performance. This framework is applicable for cases where the underlying field is label-based such that each pixel of the underlying field can only assume values from a discrete, limited set. We consider this unified framework in the context of combinatorial optimization and propose graph-cut based methods that would result in label-based images, thereby eliminating the need for a separate detection step. Finally, we propose a robust method of object detection from microscopic nanoparticle images. In particular, we focus on a portable, low cost interferometric imaging platform and propose robust detection algorithms using tools from computer vision. We model the electromagnetic image formation process and use this model to create an enhanced detection technique. The effectiveness of the proposed technique is demonstrated using manually labeled ground-truth data. In addition, we extend these tools to develop a detection based autofocusing algorithm tailored for the high numerical aperture interferometric microscope

Image compression and energy harvesting for energy constrained sensors

Title from PDF of title page, viewed on June 21, 2013Dissertation advisor: Walter D. Leon-SalasVitaIncludes bibliographic references (pages 176-[187])Thesis (Ph.D.)--School of Computing and Engineering. University of Missouri--Kansas City, 2013The advances in complementary metal-oxide-semiconductor (CMOS) technology have led to the integration of all components of electronic system into a single integrated circuit. Ultra-low power circuit techniques have reduced the power consumption of circuits. Moreover, solar cells with improved efficiency can be integrated on chip to harvest energy from sunlight. As a result of all the above, a new class of miniaturized electronic systems known as self-powered system on a chip has emerged. There is an increasing research interest in the area of self-powered devices which provide cost-effective solutions especially when these devices are used in the areas that changing or replacing batteries is too costly. Therefore, image compression and energy harvesting are studied in this dissertation. The integration of energy harvesting, image compression, and an image sensor on the same chip provides the energy source to charge a battery, reduces the data rate, and improves the performance of wireless image sensors. Integrated circuits of image compression, solar energy harvesting, and image sensors are studied, designed, and analyzed in this work. In this dissertation, a hybrid image sensor that can perform the tasks of sensing and energy harvesting is presented. Photodiodes of hybrid image sensor can be programmed as image sensors or energy harvesting cells. The hybrid image sensor can harvest energy in between frames, in sleep mode, and even when it is taking images. When sensing images and harvesting energy are both needed at the same time, some pixels have to work as sensing pixels, and the others have to work as solar cells. Since some pixels are devoted to harvest energy, the resolution of the image will be reduced. To preserve the resolution or to keep the fair resolution when a lot of energy collection is needed, image reconstruction algorithms and compressive sensing theory provide solutions to achieve a good image quality. On the other hand, when the battery has enough charge, image compression comes into the picture. Multiresolution decomposition image compression provides a way to compress image data in order to reduce the energy need from data transmission. The solution provided in this dissertation not only harvests energy but also saves energy resulting long lasting wireless sensors. The problem was first studied at the system level to identify the best system-level configuration which was then implemented on silicon. As a proof of concept, a 32 x 32 array of hybrid image sensor, a 32 x 32 array of image sensor with multiresolution decomposition compression, and a compressive sensing converter have been designed and fabricated in a standard 0.5 [micrometer] CMOS process. Printed circuit broads also have been designed to test and verify the proposed and fabricated chips. VHDL and Matlab codes were written to generate the proper signals to control, and read out data from chips. Image processing and recovery were carried out in Matlab. DC-DC converters were designed to boost the inherently low voltage output of the photodiodes. The DC-DC converter has also been improved to increase the efficiency of power transformation.Introduction -- Hybrid imager system and circuit design -- Hybrid imager energy harvesting and image acquisition results and discussion -- Detailed description and mathematical analysis for a circuit of energy harvesting using on-chip solar cells -- Multiresolution decomposition for lossless and near-lossless compression -- An incremental [sigma-delta] converter for compressive sensing -- Detailed description of a sigma-delta random demodulator converter architecture for compressive sensing applications -- Conclusion -- Appendix A. Chip pin-out -- Appendix B. Schematics -- Appendix C. Pictures of custom PC

Neural Network Methods for Radiation Detectors and Imaging

Recent advances in image data processing through machine learning and especially deep neural networks (DNNs) allow for new optimization and performance-enhancement schemes for radiation detectors and imaging hardware through data-endowed artificial intelligence. We give an overview of data generation at photon sources, deep learning-based methods for image processing tasks, and hardware solutions for deep learning acceleration. Most existing deep learning approaches are trained offline, typically using large amounts of computational resources. However, once trained, DNNs can achieve fast inference speeds and can be deployed to edge devices. A new trend is edge computing with less energy consumption (hundreds of watts or less) and real-time analysis potential. While popularly used for edge computing, electronic-based hardware accelerators ranging from general purpose processors such as central processing units (CPUs) to application-specific integrated circuits (ASICs) are constantly reaching performance limits in latency, energy consumption, and other physical constraints. These limits give rise to next-generation analog neuromorhpic hardware platforms, such as optical neural networks (ONNs), for high parallel, low latency, and low energy computing to boost deep learning acceleration