Ultra-fast Lensless Computational Imaging through 5D Frequency Analysis of Time-resolved Light Transport

Light transport has been analyzed extensively, in both the primal domain and the frequency domain. Frequency analyses often provide intuition regarding effects introduced by light propagation and interaction with optical elements; such analyses encourage optimal designs of computational cameras that efficiently capture tailored visual information. However, previous analyses have relied on instantaneous propagation of light, so that the measurement of the time dynamics of light–scene interaction, and any resulting information transfer, is precluded. In this paper, we relax the common assumption that the speed of light is infinite. We analyze free space light propagation in the frequency domain considering spatial, temporal, and angular light variation. Using this analysis, we derive analytic expressions for information transfer between these dimensions and show how this transfer can be exploited for designing a new lensless imaging system. With our frequency analysis, we also derive performance bounds for the proposed computational camera architecture and provide a mathematical framework that will also be useful for future ultra-fast computational imaging systems.MIT Media Lab ConsortiumNatural Sciences and Engineering Research Council of Canad

Relativistic Effects for Time-Resolved Light Transport

We present a real-time framework which allows interactive visualization of relativistic effects for time-resolved light transport. We leverage data from two different sources: real-world data acquired with an effective exposure time of less than 2 picoseconds, using an ultra-fast imaging technique termed femto-photography, and a transient renderer based on ray-tracing. We explore the effects of time dilation, light aberration, frequency shift and radiance accumulation by modifying existing models of these relativistic effects to take into account the time-resolved nature of light propagation. Unlike previous works, we do not impose limiting constraints in the visualization, allowing the virtual camera to explore freely a reconstructed 3D scene depicting dynamic illumination. Moreover, we consider not only linear motion, but also acceleration and rotation of the camera. We further introduce, for the first time, a pinhole camera model into our relativistic rendering framework, and account for subsequent changes in focal length and field of view as the camera moves through the scene

Low-cost portable microscopy systems for biomedical imaging and healthcare applications

In recent years, the development of low-cost portable microscopes (LPMs) has opened new possibilities for disease detection and biomedical research, especially in resource-limited areas. Despite these advancements, the majority of existing LPMs are hampered by sophisticated optical and mechanical designs, require extensive post-data analysis, and are often tailored for specific biomedical applications, limiting their broader utility. Furthermore, creating an optical-sectioning microscope that is both compact and cost effective presents a significant challenge. Addressing these critical gaps, this PhD study aims to: (1) develop a universally applicable LPM featuring a simplified mechanical and optical design for real-time biomedical imaging analysis, and (2) design a novel, smartphone-based optical sectioning microscope that is both compact and affordable. These objectives are driven by the need to enhance accessibility to quality diagnostic tools in varied settings, promising a significant leap forward in the democratization of biomedical imaging technologies. With 3D printing, optimised optical design, and AI techniques, we can develop LPM’s real time analysis functionality. I conducted a literature review on LPMs and related applications in my study and implemented two low-cost prototype microscopes and one theoretical study. 1) The first project is a portable AI fluorescence microscope based on a webcam and the NVIDIA Jetson Nano (NJN) with real-time analysis functionality. The system was 3D printed, weighing ~250 grams with a size of 145mm × 172 mm × 144 mm (L×W×H) and costing ~$400. It achieves a physical magnification of ×5 and can resolve 228.1 lp/mm USAF features. The system can recognise and count fluorescent beads and human red blood cells (RBCs). 2) I developed a smartphone-based optical sectioning microscope using the HiLo technique. To our knowledge, it is the first smartphone-based HiLo microscope that offers low-cost optical-sectioned widefield imaging. It has a 571.5 μm telecentric scanning range and an 11.7 μm axial resolution. I successfully used it to realize optical sectioning imaging of fluorescent beads. For this system, I developed a new low-cost HiLo microscopy technique using microlens arrays (MLAs) with incoherent light-emitting diode (LED) light sources. I conducted a numerical simulation study assessing the integration of uncoherent LEDs and MLAs for a low-cost HiLo system. The MLA can generate structured illumination in HiLo. How the MLA’s geometry structure and physical parameters affect the image performance were discussed in detail. This PhD thesis explores the advancement of low-cost portable microscopes (LPMs) through the integration of 3D printing, optimized optical design, and artificial intelligence (AI) techniques to enhance their real-time analysis capabilities. The research involved a comprehensive literature review on LPMs and their applications, leading to the development of two innovative prototype LPMs, alongside a theoretical study. These works contribute significantly to the field by not only addressing the technical and financial barriers associated with advanced microscopy but also by laying the groundwork for future innovations in portable and accessible biomedical imaging. Through its focus on simplification, affordability, and practicality, the research holds promise for substantially expanding the reach and impact of diagnostic imaging technologies, especially in those resource-limited areas.In recent years, the development of low-cost portable microscopes (LPMs) has opened new possibilities for disease detection and biomedical research, especially in resource-limited areas. Despite these advancements, the majority of existing LPMs are hampered by sophisticated optical and mechanical designs, require extensive post-data analysis, and are often tailored for specific biomedical applications, limiting their broader utility. Furthermore, creating an optical-sectioning microscope that is both compact and cost effective presents a significant challenge. Addressing these critical gaps, this PhD study aims to: (1) develop a universally applicable LPM featuring a simplified mechanical and optical design for real-time biomedical imaging analysis, and (2) design a novel, smartphone-based optical sectioning microscope that is both compact and affordable. These objectives are driven by the need to enhance accessibility to quality diagnostic tools in varied settings, promising a significant leap forward in the democratization of biomedical imaging technologies. With 3D printing, optimised optical design, and AI techniques, we can develop LPM’s real time analysis functionality. I conducted a literature review on LPMs and related applications in my study and implemented two low-cost prototype microscopes and one theoretical study. 1) The first project is a portable AI fluorescence microscope based on a webcam and the NVIDIA Jetson Nano (NJN) with real-time analysis functionality. The system was 3D printed, weighing ~250 grams with a size of 145mm × 172 mm × 144 mm (L×W×H) and costing ~$400. It achieves a physical magnification of ×5 and can resolve 228.1 lp/mm USAF features. The system can recognise and count fluorescent beads and human red blood cells (RBCs). 2) I developed a smartphone-based optical sectioning microscope using the HiLo technique. To our knowledge, it is the first smartphone-based HiLo microscope that offers low-cost optical-sectioned widefield imaging. It has a 571.5 μm telecentric scanning range and an 11.7 μm axial resolution. I successfully used it to realize optical sectioning imaging of fluorescent beads. For this system, I developed a new low-cost HiLo microscopy technique using microlens arrays (MLAs) with incoherent light-emitting diode (LED) light sources. I conducted a numerical simulation study assessing the integration of uncoherent LEDs and MLAs for a low-cost HiLo system. The MLA can generate structured illumination in HiLo. How the MLA’s geometry structure and physical parameters affect the image performance were discussed in detail. This PhD thesis explores the advancement of low-cost portable microscopes (LPMs) through the integration of 3D printing, optimized optical design, and artificial intelligence (AI) techniques to enhance their real-time analysis capabilities. The research involved a comprehensive literature review on LPMs and their applications, leading to the development of two innovative prototype LPMs, alongside a theoretical study. These works contribute significantly to the field by not only addressing the technical and financial barriers associated with advanced microscopy but also by laying the groundwork for future innovations in portable and accessible biomedical imaging. Through its focus on simplification, affordability, and practicality, the research holds promise for substantially expanding the reach and impact of diagnostic imaging technologies, especially in those resource-limited areas

Omni-Line-of-Sight Imaging for Holistic Shape Reconstruction

We introduce Omni-LOS, a neural computational imaging method for conducting holistic shape reconstruction (HSR) of complex objects utilizing a Single-Photon Avalanche Diode (SPAD)-based time-of-flight sensor. As illustrated in Fig. 1, our method enables new capabilities to reconstruct near-$360^\circ$ surrounding geometry of an object from a single scan spot. In such a scenario, traditional line-of-sight (LOS) imaging methods only see the front part of the object and typically fail to recover the occluded back regions. Inspired by recent advances of non-line-of-sight (NLOS) imaging techniques which have demonstrated great power to reconstruct occluded objects, Omni-LOS marries LOS and NLOS together, leveraging their complementary advantages to jointly recover the holistic shape of the object from a single scan position. The core of our method is to put the object nearby diffuse walls and augment the LOS scan in the front view with the NLOS scans from the surrounding walls, which serve as virtual ``mirrors'' to trap lights toward the object. Instead of separately recovering the LOS and NLOS signals, we adopt an implicit neural network to represent the object, analogous to NeRF and NeTF. While transients are measured along straight rays in LOS but over the spherical wavefronts in NLOS, we derive differentiable ray propagation models to simultaneously model both types of transient measurements so that the NLOS reconstruction also takes into account the direct LOS measurements and vice versa. We further develop a proof-of-concept Omni-LOS hardware prototype for real-world validation. Comprehensive experiments on various wall settings demonstrate that Omni-LOS successfully resolves shape ambiguities caused by occlusions, achieves high-fidelity 3D scan quality, and manages to recover objects of various scales and complexity

Revealing the Invisible: On the Extraction of Latent Information from Generalized Image Data

The desire to reveal the invisible in order to explain the world around us has been a source of impetus for technological and scientific progress throughout human history. Many of the phenomena that directly affect us cannot be sufficiently explained based on the observations using our primary senses alone. Often this is because their originating cause is either too small, too far away, or in other ways obstructed. To put it in other words: it is invisible to us. Without careful observation and experimentation, our models of the world remain inaccurate and research has to be conducted in order to improve our understanding of even the most basic effects. In this thesis, we1 are going to present our solutions to three challenging problems in visual computing, where a surprising amount of information is hidden in generalized image data and cannot easily be extracted by human observation or existing methods. We are able to extract the latent information using non-linear and discrete optimization methods based on physically motivated models and computer graphics methodology, such as ray tracing, real-time transient rendering, and image-based rendering

Needs, trends, and advances in scintillators for radiographic imaging and tomography

Scintillators are important materials for radiographic imaging and tomography (RadIT), when ionizing radiations are used to reveal internal structures of materials. Since its invention by R\"ontgen, RadIT now come in many modalities such as absorption-based X-ray radiography, phase contrast X-ray imaging, coherent X-ray diffractive imaging, high-energy X- and $\gamma-$ray radiography at above 1 MeV, X-ray computed tomography (CT), proton imaging and tomography (IT), neutron IT, positron emission tomography (PET), high-energy electron radiography, muon tomography, etc. Spatial, temporal resolution, sensitivity, and radiation hardness, among others, are common metrics for RadIT performance, which are enabled by, in addition to scintillators, advances in high-luminosity accelerators and high-power lasers, photodetectors especially CMOS pixelated sensor arrays, and lately data science. Medical imaging, nondestructive testing, nuclear safety and safeguards are traditional RadIT applications. Examples of growing or emerging applications include space, additive manufacturing, machine vision, and virtual reality or `metaverse'. Scintillator metrics such as light yield and decay time are correlated to RadIT metrics. More than 160 kinds of scintillators and applications are presented during the SCINT22 conference. New trends include inorganic and organic scintillator heterostructures, liquid phase synthesis of perovskites and $\mu$m-thick films, use of multiphysics models and data science to guide scintillator development, structural innovations such as photonic crystals, nanoscintillators enhanced by the Purcell effect, novel scintillator fibers, and multilayer configurations. Opportunities exist through optimization of RadIT with reduced radiation dose, data-driven measurements, photon/particle counting and tracking methods supplementing time-integrated measurements, and multimodal RadIT.Comment: 45 pages, 43 Figures, SCINT22 conference overvie

The 2017 Magnetism Roadmap

Building upon the success and relevance of the 2014 Magnetism Roadmap, this 2017 Magnetism Roadmap edition follows a similar general layout, even if its focus is naturally shifted, and a different group of experts and, thus, viewpoints are being collected and presented. More importantly, key developments have changed the research landscape in very relevant ways, so that a novel view onto some of the most crucial developments is warranted, and thus, this 2017 Magnetism Roadmap article is a timely endeavour. The change in landscape is hereby not exclusively scientific, but also reflects the magnetism related industrial application portfolio. Specifically, Hard Disk Drive technology, which still dominates digital storage and will continue to do so for many years, if not decades, has now limited its footprint in the scientific and research community, whereas significantly growing interest in magnetism and magnetic materials in relation to energy applications is noticeable, and other technological fields are emerging as well. Also, more and more work is occurring in which complex topologies of magnetically ordered states are being explored, hereby aiming at a technological utilization of the very theoretical concepts that were recognised by the 2016 Nobel Prize in Physics. Given this somewhat shifted scenario, it seemed appropriate to select topics for this Roadmap article that represent the three core pillars of magnetism, namely magnetic materials, magnetic phenomena and associated characterization techniques, as well as applications of magnetism. While many of the contributions in this Roadmap have clearly overlapping relevance in all three fields, their relative focus is mostly associated to one of the three pillars. In this way, the interconnecting roles of having suitable magnetic materials, understanding (and being able to characterize) the underlying physics of their behaviour and utilizing them for applications and devices is well illustrated, thus giving an accurate snapshot of the world of magnetism in 2017. The article consists of 14 sections, each written by an expert in the field and addressing a specific subject on two pages. Evidently, the depth at which each contribution can describe the subject matter is limited and a full review of their statuses, advances, challenges and perspectives cannot be fully accomplished. Also, magnetism, as a vibrant research field, is too diverse, so that a number of areas will not be adequately represented here, leaving space for further Roadmap editions in the future. However, this 2017 Magnetism Roadmap article can provide a frame that will enable the reader to judge where each subject and magnetism research field stands overall today and which directions it might take in the foreseeable future. The first material focused pillar of the 2017 Magnetism Roadmap contains five articles, which address the questions of atomic scale confinement, 2D, curved and topological magnetic materials, as well as materials exhibiting unconventional magnetic phase transitions. The second pillar also has five contributions, which are devoted to advances in magnetic characterization, magneto-optics and magneto-plasmonics, ultrafast magnetization dynamics and magnonic transport. The final and application focused pillar has four contributions, which present non-volatile memory technology, antiferromagnetic spintronics, as well as magnet technology for energy and bio-related applications. As a whole, the 2017 Magnetism Roadmap article, just as with its 2014 predecessor, is intended to act as a reference point and guideline for emerging research directions in modern magnetism

Micro- and Nanofluidics for Bionanoparticle Analysis

Bionanoparticles such as microorganisms and exosomes are recoganized as important targets for clinical applications, food safety, and environmental monitoring. Other nanoscale biological particles, includeing liposomes, micelles, and functionalized polymeric particles are widely used in nanomedicines. The recent deveopment of microfluidic and nanofluidic technologies has enabled the separation and anslysis of these species in a lab-on-a-chip platform, while there are still many challenges to address before these analytical tools can be adopted in practice. For example, the complex matrices within which these species reside in create a high background for their detection. Their small dimension and often low concentration demand creative strategies to amplify the sensing signal and enhance the detection speed. This Special Issue aims to recruit recent discoveries and developments of micro- and nanofluidic strategies for the processing and analysis of biological nanoparticles. The collection of papers will hopefully bring out more innovative ideas and fundamental insights to overcome the hurdles faced in the separation and detection of bionanoparticles

Time resolved and time average imaging of magnetic nano-structures

The ability of a ferromagnet to maintain its magnetic state in the absence of an external magnetic field has made ferromagnetic materials an important subject of study in physics since the end of the 19th century. Moreover, ferromagnetic materials are the cornerstone for data storage systems such as magnetic tapes, magnetic disk drives and magnetic random access memory. The discovery of the Giant Magneto Resistance (GMR) in 1988 suggested that, since the magnetic state of the electrical conductor has an important effect upon the current flow, there may also be an inverse influence of the current upon the magnetization. In this effect, predicted in 1989 [1] by Slonczewski and called Spin Transfer Torque, angular momentum transferred by a spin polarized current can exert a torque on the magnetization of a ferromagnetic material, changing the local magnetization and stimulating the precession of the magnetic moments, generating microwave signals. This provides a new method of manipulating magnetization without applying an external field. Large polarized currents lead to spin transfer effects which are the driving force for the magnetic dynamics of devices known as Spin Transfer Oscillators (STO). In this new kind of nano-device the emission of microwaves is stimulated by a DC electrical current and measured as a change in the output voltage due the GMR effect. The specific characteristics of these devices such as working frequency and DC current ranges, microwave emission linewidth, and maximum emission power among others, are given by the design and size of the device,and the nature of the magnetic oscillations producing the emission. Among the multiple types of STO that now exist , I have focused my research upon three of them: Spin Transfer Vortex Oscillators (STVO), Single Layer Spin Transfer Oscillators (SL-STO) and Orthogonal Pseudo Spin Valves. Within STVOs and SL-STOs we can nucleate what is called a magnetic vortex. A magnetic vortex is a curling of the in-plane of a magnetic layer with its centre pointing out of the magnetization plane. The gyration of this vortex due to STT produces a microwave emission < 1GHz with a greater emission power than that produced by the precession of magnetic moments in STOs. The phase-locked synchronisation of multiple vortices is expected to exhibit enhanced microwaved power and phase stability compared to a single vortex device, providing a solution to the drawbacks of the STO in the low frequency regime. On the other hand, Orthogonal Pseudo Spin Valves promote the nucleation of magnetic dissipative solitons, also called magnetic droplets. This type of magnetic structure has an opposite out of plane magnetization to the layer that contains it. Compared to the microwave emission of magnetic vortices , magnetic droplets have a higher frequency range and emission power. However, their nucleation is subject to large external fields being applied to the sample. In this thesis, I electrically characterized these devices and applied magnetic imaging techniques in order to go further in the understanding of the spatial features and dynamic behaviour of these magnetic structures. It is not possible to acquire this knowledge by only using electrical characterization. Understanding the magnetization dynamics in these devices is crucial for the design of STO based devices while imaging studies are required to prove the existence of these magnetic structures, as in case of the magnetic droplet. In chapter 2 I will introduce the background concepts of magnetism that are relevant to this thesis. I will go from the basics principles of ferromagnetism, its quantum mechanical treatment, and the theory that explain the dynamics of the magnetisation. I will also present the state of the art in experimental research in the field of spin transfer oscillators. My aim is to give the basic background needed to understand the results presented in this thesis. In chapter 3 I will introduce the two main experimental techniques used for imaging the magnetisation of the devices presented: Holography with Extended Reference by Autocorrelation Linear Differential Operator (HERALDO) and Time Resolved Scanning Kerr Microscopy (TRSKM). I will revise the theoretical background concepts and the development of the techniques in order to demostrate the uniqueness of each technique and how they were used in this thesis. It is interesting to note that while MOKE is a well-known and widely-used technique, far fewer laboratories in the world area able to perform time resolved measurements using MOKE, with the University of Exeter being one of them. Furthermore, HERALDO is a novel technique that is used for the first time to image magnetic structures within multilayer systems in this thesis, which is a milestone in the development of the techinque. In chapter 4 I present an investigation of the magnetization dynamics of a SL-STO. Electrical transport measurements provided an initial characterization of the device. We then used HERALDO for the first time to investigate the magnetization dynamics in an intermediate layer of a multilayer stack. We present time averaged measurements of the magnetisation of a magnetic vortex formed underneath a nano contact (NC) positioned on top of the multilayer, using a combination of x-ray holography and x-ray magnetic circular dichroism. In chapter 5 I present the first direct measurement at the time of a magnetic dissipative droplet, using holography with extended reference autocorrelation by linear differential operator (HERALDO). I studied the out of plane magnetisation of the free layer under a NC within an orthogonal pseudo spin salve. In chapter 6 I present and study STVO devices with pairs of NCs of 100 nm diameter and centre-to-centre separation D = 200 to 1100 nm, by a combination of electrical measurements and time-resolved scanning Kerr microscopy (TRSKM). It will be shown that the dynamic behaviour of vortices and anti vortices changes when the distances between the NCs within the devices is changed