28 research outputs found
Deep Cytometry: Deep learning with Real-time Inference in Cell Sorting and Flow Cytometry
Deep learning has achieved spectacular performance in image and speech
recognition and synthesis. It outperforms other machine learning algorithms in
problems where large amounts of data are available. In the area of measurement
technology, instruments based on the photonic time stretch have established
record real-time measurement throughput in spectroscopy, optical coherence
tomography, and imaging flow cytometry. These extreme-throughput instruments
generate approximately 1 Tbit/s of continuous measurement data and have led to
the discovery of rare phenomena in nonlinear and complex systems as well as new
types of biomedical instruments. Owing to the abundance of data they generate,
time-stretch instruments are a natural fit to deep learning classification.
Previously we had shown that high-throughput label-free cell classification
with high accuracy can be achieved through a combination of time-stretch
microscopy, image processing and feature extraction, followed by deep learning
for finding cancer cells in the blood. Such a technology holds promise for
early detection of primary cancer or metastasis. Here we describe a new deep
learning pipeline, which entirely avoids the slow and computationally costly
signal processing and feature extraction steps by a convolutional neural
network that directly operates on the measured signals. The improvement in
computational efficiency enables low-latency inference and makes this pipeline
suitable for cell sorting via deep learning. Our neural network takes less than
a few milliseconds to classify the cells, fast enough to provide a decision to
a cell sorter for real-time separation of individual target cells. We
demonstrate the applicability of our new method in the classification of OT-II
white blood cells and SW-480 epithelial cancer cells with more than 95%
accuracy in a label-free fashion
Deep Learning in Label-free Cell Classification.
Label-free cell analysis is essential to personalized genomics, cancer diagnostics, and drug development as it avoids adverse effects of staining reagents on cellular viability and cell signaling. However, currently available label-free cell assays mostly rely only on a single feature and lack sufficient differentiation. Also, the sample size analyzed by these assays is limited due to their low throughput. Here, we integrate feature extraction and deep learning with high-throughput quantitative imaging enabled by photonic time stretch, achieving record high accuracy in label-free cell classification. Our system captures quantitative optical phase and intensity images and extracts multiple biophysical features of individual cells. These biophysical measurements form a hyperdimensional feature space in which supervised learning is performed for cell classification. We compare various learning algorithms including artificial neural network, support vector machine, logistic regression, and a novel deep learning pipeline, which adopts global optimization of receiver operating characteristics. As a validation of the enhanced sensitivity and specificity of our system, we show classification of white blood T-cells against colon cancer cells, as well as lipid accumulating algal strains for biofuel production. This system opens up a new path to data-driven phenotypic diagnosis and better understanding of the heterogeneous gene expressions in cells
High-Speed Imaging and Optical Sensing Systems for Biomedical Applications
High-throughput real-time optical sensing and imaging instruments for capture and analysis of fast phenomena are among the most essential tools for scientific, industrial, military, and most importantly biomedical applications. The key challenge in these instruments is the fundamental trade-off between speed and sensitivity of the measurement system due to the limited signal energy collected in each measurement window. Based on two enabling technologies, namely photonic time-stretch dispersive Fourier transform and optical amplification, we developed several novel high-throughput optical measurement tools for applications such as flow cytometry, vibrometry, and volumetric scanning.We demonstrated optical Raman amplification at about 800 nm wavelength for the first time and extended time-stretch dispersive Fourier transform to this region of electromagnetic spectrum. We used this enabling technology to make an ultrafast three-dimensional laser scanner with about hundred thousand scans per second and an imaging vibrometer with nanometer-scale axial resolution. We also employed our high-speed laser scanner to perform label-free cell screening in flow. One of the fundamental challenges in cell analysis is the undesirable impact of cell labeling on cellular behavior. To eliminate the need for these labels, while keeping the cell classification accuracy high, additional label-free parameters such as precise measurement of the cell protein concentration is required. We introduced a high-accuracy label-free imaging flow cytometer based on simultaneous measurement of morphology and optical path length through the cell at flow speeds as high as a few meters per second. Finally, the ultimate challenge in ultra-high-throughput instrumentation is the storage and analysis of the torrent of generated data. As an example, our imaging flow cytometer generates about ten terabytes of cell images over a course of one hour acquisition, which captures images of every single cell in more than two milliliters of sample e.g. blood. We enabled practical use of these big data volumes by efficient combination of analog preprocessing techniques such as quadrature demodulation with parallel storage and digital post-processing
Designing an Efficient End-to-end Machine Learning Pipeline for Real-time Empty-shelf Detection
On-Shelf Availability (OSA) of products in retail stores is a critical
business criterion in the fast moving consumer goods and retails sector. When a
product is out-of-stock (OOS) and a customer cannot find it on its designed
shelf, this motivates the customer to store-switching or buying nothing, which
causes fall in future sales and demands. Retailers are employing several
approaches to detect empty shelves and ensure high OSA of products; however,
such methods are generally ineffective and infeasible since they are either
manual, expensive or less accurate. Recently machine learning based solutions
have been proposed, but they suffer from high computational cost and low
accuracy problem due to lack of large annotated datasets of on-shelf products.
Here, we present an elegant approach for designing an end-to-end machine
learning (ML) pipeline for real-time empty shelf detection. Considering the
strong dependency between the quality of ML models and the quality of data, we
focus on the importance of proper data collection, cleaning and correct data
annotation before delving into modeling. Since an empty-shelf detection
solution should be computationally-efficient for real-time predictions, we
explore different run-time optimizations to improve the model performance. Our
dataset contains 1000 images, collected and annotated by following well-defined
guidelines. Our low-latency model achieves a mean average F1-score of 68.5%,
and can process up to 67 images/s on Intel Xeon Gold and up to 860 images/s on
an A100 GPU.Comment: 7 figures, 3 tables, 10 page
Artificial intelligence in label-free microscopy: biological cell classification by time stretch
This book introduces time-stretch quantitative phase imaging (TS-QPI), a high-throughput label-free imaging flow cytometer developed for big data acquisition and analysis in phenotypic screening. TS-QPI is able to capture quantitative optical phase and intensity images simultaneously, enabling high-content cell analysis, cancer diagnostics, personalized genomics, and drug development. The authors also demonstrate a complete machine learning pipeline that performs optical phase measurement, image processing, feature extraction, and classification, enabling high-throughput quantitative imaging that achieves record high accuracy in label -free cellular phenotypic screening and opens up a new path to data-driven diagnosis. • Demonstrates how machine learning is used in high-speed microscopy imaging to facilitate medical diagnosis; • Provides a systematic and comprehensive illustration of time stretch technology; • Enables multidisciplinary application, including industrial, biomedical, and artificial intelligence