Text detection and recognition based on a lensless imaging system

Lensless cameras are characterized by several advantages (e.g., miniaturization, ease of manufacture, and low cost) as compared with conventional cameras. However, they have not been extensively employed due to their poor image clarity and low image resolution, especially for tasks that have high requirements on image quality and details such as text detection and text recognition. To address the problem, a framework of deep-learning-based pipeline structure was built to recognize text with three steps from raw data captured by employing lensless cameras. This pipeline structure consisted of the lensless imaging model U-Net, the text detection model connectionist text proposal network (CTPN), and the text recognition model convolutional recurrent neural network (CRNN). Compared with the method focusing only on image reconstruction, UNet in the pipeline was able to supplement the imaging details by enhancing factors related to character categories in the reconstruction process, so the textual information can be more effectively detected and recognized by CTPN and CRNN with fewer artifacts and high-clarity reconstructed lensless images. By performing experiments on datasets of different complexities, the applicability to text detection and recognition on lensless cameras was verified. This study reasonably demonstrates text detection and recognition tasks in the lensless camera system,and develops a basic method for novel applications

Strain in a silicon-on-insulator nanostructure revealed by 3D x-ray Bragg ptychography

International audienceProgresses in the design of well-defined electronic band structure and dedicated functionalities rely on the high control of complex architectural device nano-scaled structures. This includes the challenging accurate description of strain fields in crystalline structures, which requires non invasive and three-dimensional (3D) imaging methods. Here, we demonstrate in details how x-ray Bragg ptychography can be used to quantify in 3D a displacement field in a lithographically patterned silicon-on-insulator structure. The image of the crystalline properties, which results from the phase retrieval of a coherent intensity data set, is obtained from a well-controlled optimized process, for which all steps are detailed. These results confirm the promising perspectives of 3D Bragg ptychography for the investigation of complex nano-structured crystals in material science

Roadmap on digital holography [Invited]

This Roadmap article on digital holography provides an overview of a vast array of research activities in the field of digital holography. The paper consists of a series of 25 sections from the prominent experts in digital holography presenting various aspects of the field on sensing, 3D imaging and displays, virtual and augmented reality, microscopy, cell identification, tomography, label-free live cell imaging, and other applications. Each section represents the vision of its author to describe the significant progress, potential impact, important developments, and challenging issues in the field of digital holography

Low-cost portable microscopy systems for biomedical imaging and healthcare applications

In recent years, the development of low-cost portable microscopes (LPMs) has opened new possibilities for disease detection and biomedical research, especially in resource-limited areas. Despite these advancements, the majority of existing LPMs are hampered by sophisticated optical and mechanical designs, require extensive post-data analysis, and are often tailored for specific biomedical applications, limiting their broader utility. Furthermore, creating an optical-sectioning microscope that is both compact and cost effective presents a significant challenge. Addressing these critical gaps, this PhD study aims to: (1) develop a universally applicable LPM featuring a simplified mechanical and optical design for real-time biomedical imaging analysis, and (2) design a novel, smartphone-based optical sectioning microscope that is both compact and affordable. These objectives are driven by the need to enhance accessibility to quality diagnostic tools in varied settings, promising a significant leap forward in the democratization of biomedical imaging technologies. With 3D printing, optimised optical design, and AI techniques, we can develop LPM’s real time analysis functionality. I conducted a literature review on LPMs and related applications in my study and implemented two low-cost prototype microscopes and one theoretical study. 1) The first project is a portable AI fluorescence microscope based on a webcam and the NVIDIA Jetson Nano (NJN) with real-time analysis functionality. The system was 3D printed, weighing ~250 grams with a size of 145mm × 172 mm × 144 mm (L×W×H) and costing ~$400. It achieves a physical magnification of ×5 and can resolve 228.1 lp/mm USAF features. The system can recognise and count fluorescent beads and human red blood cells (RBCs). 2) I developed a smartphone-based optical sectioning microscope using the HiLo technique. To our knowledge, it is the first smartphone-based HiLo microscope that offers low-cost optical-sectioned widefield imaging. It has a 571.5 μm telecentric scanning range and an 11.7 μm axial resolution. I successfully used it to realize optical sectioning imaging of fluorescent beads. For this system, I developed a new low-cost HiLo microscopy technique using microlens arrays (MLAs) with incoherent light-emitting diode (LED) light sources. I conducted a numerical simulation study assessing the integration of uncoherent LEDs and MLAs for a low-cost HiLo system. The MLA can generate structured illumination in HiLo. How the MLA’s geometry structure and physical parameters affect the image performance were discussed in detail. This PhD thesis explores the advancement of low-cost portable microscopes (LPMs) through the integration of 3D printing, optimized optical design, and artificial intelligence (AI) techniques to enhance their real-time analysis capabilities. The research involved a comprehensive literature review on LPMs and their applications, leading to the development of two innovative prototype LPMs, alongside a theoretical study. These works contribute significantly to the field by not only addressing the technical and financial barriers associated with advanced microscopy but also by laying the groundwork for future innovations in portable and accessible biomedical imaging. Through its focus on simplification, affordability, and practicality, the research holds promise for substantially expanding the reach and impact of diagnostic imaging technologies, especially in those resource-limited areas.In recent years, the development of low-cost portable microscopes (LPMs) has opened new possibilities for disease detection and biomedical research, especially in resource-limited areas. Despite these advancements, the majority of existing LPMs are hampered by sophisticated optical and mechanical designs, require extensive post-data analysis, and are often tailored for specific biomedical applications, limiting their broader utility. Furthermore, creating an optical-sectioning microscope that is both compact and cost effective presents a significant challenge. Addressing these critical gaps, this PhD study aims to: (1) develop a universally applicable LPM featuring a simplified mechanical and optical design for real-time biomedical imaging analysis, and (2) design a novel, smartphone-based optical sectioning microscope that is both compact and affordable. These objectives are driven by the need to enhance accessibility to quality diagnostic tools in varied settings, promising a significant leap forward in the democratization of biomedical imaging technologies. With 3D printing, optimised optical design, and AI techniques, we can develop LPM’s real time analysis functionality. I conducted a literature review on LPMs and related applications in my study and implemented two low-cost prototype microscopes and one theoretical study. 1) The first project is a portable AI fluorescence microscope based on a webcam and the NVIDIA Jetson Nano (NJN) with real-time analysis functionality. The system was 3D printed, weighing ~250 grams with a size of 145mm × 172 mm × 144 mm (L×W×H) and costing ~$400. It achieves a physical magnification of ×5 and can resolve 228.1 lp/mm USAF features. The system can recognise and count fluorescent beads and human red blood cells (RBCs). 2) I developed a smartphone-based optical sectioning microscope using the HiLo technique. To our knowledge, it is the first smartphone-based HiLo microscope that offers low-cost optical-sectioned widefield imaging. It has a 571.5 μm telecentric scanning range and an 11.7 μm axial resolution. I successfully used it to realize optical sectioning imaging of fluorescent beads. For this system, I developed a new low-cost HiLo microscopy technique using microlens arrays (MLAs) with incoherent light-emitting diode (LED) light sources. I conducted a numerical simulation study assessing the integration of uncoherent LEDs and MLAs for a low-cost HiLo system. The MLA can generate structured illumination in HiLo. How the MLA’s geometry structure and physical parameters affect the image performance were discussed in detail. This PhD thesis explores the advancement of low-cost portable microscopes (LPMs) through the integration of 3D printing, optimized optical design, and artificial intelligence (AI) techniques to enhance their real-time analysis capabilities. The research involved a comprehensive literature review on LPMs and their applications, leading to the development of two innovative prototype LPMs, alongside a theoretical study. These works contribute significantly to the field by not only addressing the technical and financial barriers associated with advanced microscopy but also by laying the groundwork for future innovations in portable and accessible biomedical imaging. Through its focus on simplification, affordability, and practicality, the research holds promise for substantially expanding the reach and impact of diagnostic imaging technologies, especially in those resource-limited areas

High-resolution ptychographic imaging at a seeded free-electron laser source using OAM beams

Electromagnetic waves possessing orbital angular momentum (OAM) are powerful tools for applications in optical communications, new quantum technologies and optical tweezers. Recently, they have attracted growing interest since they can be harnessed to detect peculiar helical dichroic effects in chiral molecular media and in magnetic nanostructures. In this work, we perform single-shot per position ptychography on a nanostructured object at a seeded free-electron laser, using extreme ultraviolet OAM beams of different topological charge order $\ell$ generated with spiral zone plates. By controlling $\ell$, we demonstrate how the structural features of OAM beam profile determine an improvement of about 30% in image resolution with respect to conventional Gaussian beam illumination. This result extends the capabilities of coherent diffraction imaging techniques, and paves the way for achieving time-resolved high-resolution (below 100 nm) microscopy on large area samples.Comment: M. Pancaldi and F. Guzzi contributed equally to this wor

Coherent methods in the X-ray sciences

X-ray sources are developing rapidly and their coherent output is growing extremely rapidly. The increased coherent flux from modern X-ray sources is being matched with an associated rapid development in experimental methods. This article reviews the literature describing the ideas that utilise the increased brilliance from modern X-ray sources. It explores how ideas in coherent X-ray science are leading to developments in other areas, and vice versa. The article describes measurements of coherence properties and uses this discussion as a base from which to describe partially-coherent diffraction and X-ray phase contrast imaging, with its applications in materials science, engineering and medicine. Coherent diffraction imaging methods are reviewed along with associated experiments in materials science. Proposals for experiments to be performed with the new X-ray free-electron-lasers are briefly discussed. The literature on X-ray photon correlation spectroscopy is described and the features it has in common with other coherent X-ray methods are identified. Many of the ideas used in the coherent X-ray literature have their origins in the optical and electron communities and these connections are explored. A review of the areas in which ideas from coherent X-ray methods are contributing to methods for the neutron, electron and optical communities is presented.Comment: A review articel accepted by Advances in Physics. 158 pages, 29 figures, 3 table

Spectroscopic imaging with single acquisition ptychography and a hyperspectral detector

We present a new method of single acquisition spectroscopic imaging with high spatial resolution. The technique is based on the combination of polychromatic synchrotron radiation and ptychographic imaging with a recently developed energy discriminating detector. We demonstrate the feasibility with a Ni-Cu test sample recorded at I13-1 of the Diamond Light Source, UK. The two elements can be clearly distinguished and the Ni absorption edge is identified. The results prove the feasibility of obtaining high-resolution structural and chemical images within a single acquisition using a polychromatic X-ray beam. The capability of resolving the absorption edge applies to a wide range of research areas, such as magnetic domains imaging and element specific investigations in biological, materials, and earth sciences. The method utilises the full available radiation spectrum and is therefore well suited for broadband radiation sources

Design and characterization of advanced diffractive devices for imaging and spectroscopy

Due to the ever-increasing demands of highly integrated optical devices in imaging, spectroscopy, communications, and so on, there is a compelling need to design and characterize novel compact photonic components. The traditional approaches to realizing compact optical devices typically result in large footprints and sizable optical thicknesses. Moreover, they offer few degrees of freedom (DOF), hampering on-demand functionalities, on-chip integration, and scalability. This thesis will address the design and development of ultracompact diffractive devices for imaging and spectroscopy, utilizing advanced machine learning techniques and optimization algorithms. I first present the inverse design of ultracompact dual-focusing lenses and broad-band focusing spectrometers based on adaptive diffractive optical networks (a-DONs), which combine optical diffraction physics and deep learning capabilities for the inverse design of multi-layered diffractive devices. I designed two-layer diffractive devices that can selectively focus incident radiation over well-separated spectral bands at desired distances and also optimized a-DON-based focusing spectrometers with engineered angular dispersion for desired bandwidth and nanometer spectral resolution. Furthermore, I introduced a new approach based on a-DONs for the engineering of diffractive devices with arbitrary k-space, which produces improved imaging performances compared to contour-PSF approaches to lens-less computational imaging. Moreover, my method enables control of sparsity and isotropic k-space in pixelated screens of dielectric scatterers that are compatible with large-scale photolithographic fabrication techniques. Finally, by combining adjoint optimization with the rigorous generalized Mie theory, I developed and characterize functionalized compact devices, which I called "photonic patches," consisting of ~100 dielectric nanocylinders that achieve predefined functionalities such as beam steering, Fresnel zone focusing, local density of states (LDOS) enhancement, etc. My method enables the inverse design of ultracompact focusing spectrometers for on-chip planar integration. Leveraging multiple scattering of light in disordered random media, I additionally demonstrated a novel approach to on-chip spectroscopy driven by high-throughput multifractal (i.e., multiscale) media, resulting in sub-nanometer spectral resolution at the 50×50 µm²-scale footprint