Deep Cytometry: Deep learning with Real-time Inference in Cell Sorting and Flow Cytometry

Deep learning has achieved spectacular performance in image and speech recognition and synthesis. It outperforms other machine learning algorithms in problems where large amounts of data are available. In the area of measurement technology, instruments based on the photonic time stretch have established record real-time measurement throughput in spectroscopy, optical coherence tomography, and imaging flow cytometry. These extreme-throughput instruments generate approximately 1 Tbit/s of continuous measurement data and have led to the discovery of rare phenomena in nonlinear and complex systems as well as new types of biomedical instruments. Owing to the abundance of data they generate, time-stretch instruments are a natural fit to deep learning classification. Previously we had shown that high-throughput label-free cell classification with high accuracy can be achieved through a combination of time-stretch microscopy, image processing and feature extraction, followed by deep learning for finding cancer cells in the blood. Such a technology holds promise for early detection of primary cancer or metastasis. Here we describe a new deep learning pipeline, which entirely avoids the slow and computationally costly signal processing and feature extraction steps by a convolutional neural network that directly operates on the measured signals. The improvement in computational efficiency enables low-latency inference and makes this pipeline suitable for cell sorting via deep learning. Our neural network takes less than a few milliseconds to classify the cells, fast enough to provide a decision to a cell sorter for real-time separation of individual target cells. We demonstrate the applicability of our new method in the classification of OT-II white blood cells and SW-480 epithelial cancer cells with more than 95% accuracy in a label-free fashion

Hybrid dispersion laser scanner.

Laser scanning technology is one of the most integral parts of today's scientific research, manufacturing, defense, and biomedicine. In many applications, high-speed scanning capability is essential for scanning a large area in a short time and multi-dimensional sensing of moving objects and dynamical processes with fine temporal resolution. Unfortunately, conventional laser scanners are often too slow, resulting in limited precision and utility. Here we present a new type of laser scanner that offers ∼1,000 times higher scan rates than conventional state-of-the-art scanners. This method employs spatial dispersion of temporally stretched broadband optical pulses onto the target, enabling inertia-free laser scans at unprecedented scan rates of nearly 100 MHz at 800 nm. To show our scanner's broad utility, we use it to demonstrate unique and previously difficult-to-achieve capabilities in imaging, surface vibrometry, and flow cytometry at a record 2D raster scan rate of more than 100 kHz with 27,000 resolvable points

All-passive pixel super-resolution of time-stretch imaging

Based on image encoding in a serial-temporal format, optical time-stretch imaging entails a stringent requirement of state-of-the- art fast data acquisition unit in order to preserve high image resolution at an ultrahigh frame rate --- hampering the widespread utilities of such technology. Here, we propose a pixel super-resolution (pixel-SR) technique tailored for time-stretch imaging that preserves pixel resolution at a relaxed sampling rate. It harnesses the subpixel shifts between image frames inherently introduced by asynchronous digital sampling of the continuous time-stretch imaging process. Precise pixel registration is thus accomplished without any active opto-mechanical subpixel-shift control or other additional hardware. Here, we present the experimental pixel-SR image reconstruction pipeline that restores high-resolution time-stretch images of microparticles and biological cells (phytoplankton) at a relaxed sampling rate (approx. 2--5 GSa/s) --- more than four times lower than the originally required readout rate (20 GSa/s) --- is thus effective for high-throughput label-free, morphology-based cellular classification down to single-cell precision. Upon integration with the high-throughput image processing technology, this pixel-SR time- stretch imaging technique represents a cost-effective and practical solution for large scale cell-based phenotypic screening in biomedical diagnosis and machine vision for quality control in manufacturing.Comment: 17 pages, 8 figure

Real-time observation of dissipative soliton formation in nonlinear polarization rotation mode-locked fibre lasers

Formation of coherent structures and patterns from unstable uniform state or noise is a fundamental physical phenomenon that occurs in various areas of science ranging from biology to astrophysics. Understanding of the underlying mechanisms of such processes can both improve our general interdisciplinary knowledge about complex nonlinear systems and lead to new practical engineering techniques. Modern optics with its high precision measurements offers excellent test-beds for studying complex nonlinear dynamics, though capturing transient rapid formation of optical solitons is technically challenging. Here we unveil the build-up of dissipative soliton in mode-locked fibre lasers using dispersive Fourier transform to measure spectral dynamics and employing autocorrelation analysis to investigate temporal evolution. Numerical simulations corroborate experimental observations, and indicate an underlying universality in the pulse formation. Statistical analysis identifies correlations and dependencies during the build-up phase. Our study may open up possibilities for real-time observation of various nonlinear structures in photonic systems