A 25 micron-thin microscope for imaging upconverting nanoparticles with NIR-I and NIR-II illumination.

Rationale: Intraoperative visualization in small surgical cavities and hard-to-access areas are essential requirements for modern, minimally invasive surgeries and demand significant miniaturization. However, current optical imagers require multiple hard-to-miniaturize components including lenses, filters and optical fibers. These components restrict both the form-factor and maneuverability of these imagers, and imagers largely remain stand-alone devices with centimeter-scale dimensions. Methods: We have engineered INSITE (Immunotargeted Nanoparticle Single-Chip Imaging Technology), which integrates the unique optical properties of lanthanide-based alloyed upconverting nanoparticles (aUCNPs) with the time-resolved imaging of a 25-micron thin CMOS-based (complementary metal oxide semiconductor) imager. We have synthesized core/shell aUCNPs of different compositions and imaged their visible emission with INSITE under either NIR-I and NIR-II photoexcitation. We characterized aUCNP imaging with INSITE across both varying aUCNP composition and 980 nm and 1550 nm excitation wavelengths. To demonstrate clinical experimental validity, we also conducted an intratumoral injection into LNCaP prostate tumors in a male nude mouse that was subsequently excised and imaged with INSITE. Results: Under the low illumination fluences compatible with live animal imaging, we measure aUCNP radiative lifetimes of 600 μs - 1.3 ms, which provides strong signal for time-resolved INSITE imaging. Core/shell NaEr0.6Yb0.4F4 aUCNPs show the highest INSITE signal when illuminated at either 980 nm or 1550 nm, with signal from NIR-I excitation about an order of magnitude brighter than from NIR-II excitation. The 55 μm spatial resolution achievable with this approach is demonstrated through imaging of aUCNPs in PDMS (polydimethylsiloxane) micro-wells, showing resolution of micrometer-scale targets with single-pixel precision. INSITE imaging of intratumoral NaEr0.8Yb0.2F4 aUCNPs shows a signal-to-background ratio of 9, limited only by photodiode dark current and electronic noise. Conclusion: This work demonstrates INSITE imaging of aUCNPs in tumors, achieving an imaging platform that is thinned to just a 25 μm-thin, planar form-factor, with both NIR-I and NIR-II excitation. Based on a highly paralleled array structure INSITE is scalable, enabling direct coupling with a wide array of surgical and robotic tools for seamless integration with tissue actuation, resection or ablation

Depth Fields: Extending Light Field Techniques to Time-of-Flight Imaging

A variety of techniques such as light field, structured illumination, and time-of-flight (TOF) are commonly used for depth acquisition in consumer imaging, robotics and many other applications. Unfortunately, each technique suffers from its individual limitations preventing robust depth sensing. In this paper, we explore the strengths and weaknesses of combining light field and time-of-flight imaging, particularly the feasibility of an on-chip implementation as a single hybrid depth sensor. We refer to this combination as depth field imaging. Depth fields combine light field advantages such as synthetic aperture refocusing with TOF imaging advantages such as high depth resolution and coded signal processing to resolve multipath interference. We show applications including synthesizing virtual apertures for TOF imaging, improved depth mapping through partial and scattering occluders, and single frequency TOF phase unwrapping. Utilizing space, angle, and temporal coding, depth fields can improve depth sensing in the wild and generate new insights into the dimensions of light's plenoptic function.Comment: 9 pages, 8 figures, Accepted to 3DV 201

A 72 × 60 Angle-Sensitive SPAD Imaging Array for Lens-less FLIM

We present a 72 × 60, angle-sensitive single photon avalanche diode (A-SPAD) array for lens-less 3D fluorescence lifetime imaging. An A-SPAD pixel consists of (1) a SPAD to provide precise photon arrival time where a time-resolved operation is utilized to avoid stimulus-induced saturation, and (2) integrated diffraction gratings on top of the SPAD to extract incident angles of the incoming light. The combination enables mapping of fluorescent sources with different lifetimes in 3D space down to micrometer scale. Futhermore, the chip presented herein integrates pixel-level counters to reduce output data-rate and to enable a precise timing control. The array is implemented in standard 180 nm complementary metal-oxide-semiconductor (CMOS) technology and characterized without any post-processing

Angle Sensitive Pixels for Lensless Imaging on Spherical Sensors

We propose OrbCam, a lensless architecture for imaging with spherical sensors. Prior work in lensless imager techniques have focused largely on using planar sensors; for such designs, it is important to use a modulation element, e.g. amplitude or phase masks, to construct a invertible imaging system. In contrast, we show that the diversity of pixel orientations on a curved surface is sufficient to improve the conditioning of the mapping between the scene and the sensor. Hence, when imaging on a spherical sensor, all pixels can have the same angular response function such that the lensless imager is comprised of pixels that are identical to each other and differ only in their orientations. We provide the computational tools for the design of the angular response of the pixels in a spherical sensor that leads to well-conditioned and noise-robust measurements. We validate our design in both simulation and a lab prototype. The implications of our design is that the lensless imaging can be enabled easily for curved and flexible surfaces thereby opening up a new set of application domains

Application of CMOS sensors in biology

S vývojem CMOS technologie dochází ke stálému zmenšování velikosti pixelu CMOS obrazových senzorů a ke snižování ceny senzoru, což umožňuje řadu nových využití v oblasti biomedicinského zobrazování. Díky těmto pokrokům dosahují metody bezčočkového zobrazování dostatečného rozlišení pro jejich aplikace namísto klasických optických systémů, s výhodami nižší ceny, zobrazování s rozlišením hloubky, většího zorného pole a vysoké přizpůsobitelnosti. Tato práce představuje řadu bezčočkových zobrazovacích metod a jejich aplikací, popisuje základní teorii holografie, metod holografické rekonstrukce a popisuje postupný návrh bezčočkového digitálního holografického mikroskopu v on-chip konfiguraci.ObhájenoWith the advances in CMOS technology, the pixel size of CMOS image sensors is getting smaller and the sensor price lower, allowing for many applications in biomedical imaging. Following these advances, lensless imaging techniques are reaching sufficient resolution capabilities that enable their use instead of classical lens-based optical systems with the advantage of lower cost, dept-resolved imaging, large Field-of-View and high adaptability. This thesis introduces various lensless imaging methods and their applications, describes the basic holographic theory, reconstruction methods and step-by-step design of a digital lensless holographic microscope in an on-chip configuration

Development of a Nano-Illumination Microscope

[eng] This doctoral thesis proposes and explores a new approach to lensless microscopy, focusing on making high resolution imaging ubiquitous and low cost. A short introduction to microscopy frames the state of current techniques: Abbe’s law limits the resolving power for visible light microscopes with lenses, techniques using UV, X-rays, or electrons are incompatible with live samples and all of them, including super-resolution microscopy methods, are complex devices not suitable for being used in the field as mobile devices. Some lensless microscopy methods try to solve these issues. The microscopy method is named Nano Illumination Microscopy (NIM) because it relies on using nanometric light sources in an ordered array to illuminate a sample placed in close proximity to them, and a photodetector at the other side to measure the amount of light arriving from each LED. In a setup like this, the resolving power is provided by the nano-LEDs and their distribution instead of the sensing devices, as is the case in the other methods. Since the resolving power depends on the pitch of the LED array, this method also opens a path to obtain super-resolution images, depending only on obtaining LED arrays with pitches smaller than Abbe’s limit for the wavelength. After the introduction to microscopy setting the context of the thesis, the thesis continues explaining the main components used to build the microscope: a SPAD camera, designed within the context of this work, and the electronics to control the nano-LED array. The third chapter of this thesis provides an overview of the microscopy method and its fundaments, exploring the requirements and capabilities. Image formation is first introduced with simulations, and this information is then used to build the very first prototype, a microscope capable of forming 8x8 pixel images -since that is the form factor of the LED array used, with LEDs of 5 μm in size (and 10 μm in pitch). The first results from this technique are presented and compared with the simulations, showing the agreement between both, validating the method, and offering insight on building the next prototypes, which will use smaller LEDs in an attempt to study the technological limits. The thesis continues with the work done in search of the limits of the technique, building and testing new improved versions of the microscope and confronting the limitations which arise. Some of those came from the structure of the LED arrays themselves: while nano-LEDs well below the sizes used have been reported, those have been isolated structures or non individually addressable. Selecting exactly which LED will emit is one of the main problems to solve since with increasingly large arrays, the connections required increase exponentially until routing is impossible. The thesis also studies this problem, as the LED arrays were changed in search of the proper solution. This implied moving from a direct addressing strategy, in which each LED was selected individually, towards a matrix-addressing format, in which the LEDs are selected by polarising the appropriate row and columns. The microscopy technique is validated and the more advanced prototypes presented. Images with a maximum resolving power of 800 nm are shown, and the results discussed, since the physical limitations on fabricating the chips limit the maximum resolving power below what was theoretically expected. The thesis also offers a short overview into the future of the Nano Illumination Microscopy technique.[cat] Aquesta tesi doctoral proposa i explora una nova aproximació a la microscopia sense lents, amb la intenció de facilitar l’obtenció d’imatges d’alta resolució amb baix cost i disponible arreu. S’ha batejat aquest mètode de microscòpia com a Microscopia de Nano-Il·luminació (MNI) perquè la imatge es construeix a partir de fonts de llum de mida nanomètrica distribuïdes en una matriu que il·luminen la mostra de forma consecutiva i ordenada. Un sensor a l’altre costat recull la intensitat de llum que arriba de cada LED, creant un mapa de l’objecte observat. Aquest mètode fa que la resolució de les imatges depengui de la mida i distribució dels LEDs, en comptes de la del sensor com és el cas convencionalment, obrint la porta a noves integracions. En la tesi s’ofereix una introducció general a la microscòpia abans d’entrar a detallar els components del microscopi i com s’integren per muntar-lo. A continuació es presenta i s’estudia el funcionament del mètode, començant amb simulacions i seguint amb la construcció del primer prototip de microscopi amb el que s’obtenen les primeres imatges. La tesi procedeix a continuació a investigar els límits actuals de la tècnica de microscòpia, utilitzant noves versions de la matriu de LEDs i estratègies alternatives per intentar superar-ne les complicacions tècniques. Així, s’obtenen imatges amb una resolució de 800 nm i es discuteix la problemàtica d’implementar dispositius que s’aproximin a les expectatives teòriques per la tècnica

A Compact raster lensless microscope based on a microdisplay

Lensless microscopy requires the simplest possible configuration, as it uses only a light source, the sample and an image sensor. The smallest practical microscope is demonstrated here. In contrast to standard lensless microscopy, the object is located near the lighting source. Raster optical microscopy is applied by using a single-pixel detector and a microdisplay. Maximum resolution relies on reduced LED size and the position of the sample respect the microdisplay. Contrarily to other sort of digital lensless holographic microscopes, light backpropagation is not required to reconstruct the images of the sample. In a mm-high microscope, resolutions down to 800 nm have been demonstrated even when measuring with detectors as large as 138 μm × 138 μm, with field of view given by the display size. Dedicated technology would shorten measuring time