DeepLight: Robust and unobtrusive real-time screen-camera communication for real-world displays

National Research Foundation (NRF) Singapore under NRF Investigatorship gran

Image processing and understanding based on graph similarity testing: algorithm design and software development

Image processing and understanding is a key task in the human visual system. Among all related topics, content based image retrieval and classification is the most typical and important problem. Successful image retrieval/classification models require an effective fundamental step of image representation and feature extraction. While traditional methods are not capable of capturing all structural information on the image, using graph to represent the image is not only biologically plausible but also has certain advantages. Graphs have been widely used in image related applications. Traditional graph-based image analysis models include pixel-based graph-cut techniques for image segmentation, low-level and high-level image feature extraction based on graph statistics and other related approaches which utilize the idea of graph similarity testing. To compare the images through their graph representations, a graph similarity testing algorithm is essential. Most of the existing graph similarity measurement tools are not designed for generic tasks such as image classification and retrieval, and some other models are either not scalable or not always effective. Graph spectral theory is a powerful analytical tool for capturing and representing structural information of the graph, but to use it on image understanding remains a challenge. In this dissertation, we focus on developing fast and effective image analysis models based on the spectral graph theory and other graph related mathematical tools. We first propose a fast graph similarity testing method based on the idea of the heat content and the mathematical theory of diffusion over manifolds. We then demonstrate the ability of our similarity testing model by comparing random graphs and power law graphs. Based on our graph analysis model, we develop a graph-based image representation and understanding framework. We propose the image heat content feature at first and then discuss several approaches to further improve the model. The first component in our improved framework is a novel graph generation model. The proposed model greatly reduces the size of the traditional pixel-based image graph representation and is shown to still be effective in representing an image. Meanwhile, we propose and discuss several low-level and high-level image features based on spectral graph information, including oscillatory image heat content, weighted eigenvalues and weighted heat content spectrum. Experiments show that the proposed models are invariant to non-structural changes on images and perform well in standard image classification benchmarks. Furthermore, our image features are robust to small distortions and changes of viewpoint. The model is also capable of capturing important image structural information on the image and performs well alone or in combination with other traditional techniques. We then introduce two real world software development projects using graph-based image processing techniques in this dissertation. Finally, we discuss the pros, cons and the intuition of our proposed model by demonstrating the properties of the proposed image feature and the correlation between different image features

Indoor Navigation System for the Visually Impaired with User-centric Graph Representation and Vision Detection Assistance

Independent navigation through unfamiliar indoor spaces is beset with barriers for the visually impaired. Hence, this issue impairs their independence, self-respect and self-reliance. In this thesis I will introduce a new indoor navigation system for the blind and visually impaired that is affordable for both the user and the building owners. Outdoor vehicle navigation technical challenges have been solved using location information provided by Global Positioning Systems (GPS) and maps using Geographical Information Systems (GIS). However, GPS and GIS information is not available for indoor environments making indoor navigation, a challenging technical problem. Moreover, the indoor navigation system needs to be developed with the blind user in mind, i.e., special care needs to be given to vision free user interface. In this project, I design and implement an indoor navigation application for the blind and visually impaired that uses RFID technology and Computer Vision for localization and a navigation map generated automatically based on environmental landmarks by simulating a user’s behavior. The focus of the indoor navigation system is no longer only on the indoor environment itself, but the way the blind users can experience it. This project will try this new idea in solving indoor navigation problems for blind and visually impaired users

AirCode: Unobtrusive Physical Tags for Digital Fabrication

We present AirCode, a technique that allows the user to tag physically fabricated objects with given information. An AirCode tag consists of a group of carefully designed air pockets placed beneath the object surface. These air pockets are easily produced during the fabrication process of the object, without any additional material or postprocessing. Meanwhile, the air pockets affect only the scattering light transport under the surface, and thus are hard to notice to our naked eyes. But, by using a computational imaging method, the tags become detectable. We present a tool that automates the design of air pockets for the user to encode information. AirCode system also allows the user to retrieve the information from captured images via a robust decoding algorithm. We demonstrate our tagging technique with applications for metadata embedding, robotic grasping, as well as conveying object affordances.Comment: ACM UIST 2017 Technical Paper

Keeping track of worm trackers

C. elegans is used extensively as a model system in the neurosciences due to its well defined nervous system. However, the seeming simplicity of this nervous system in anatomical structure and neuronal connectivity, at least compared to higher animals, underlies a rich diversity of behaviors. The usefulness of the worm in genome-wide mutagenesis or RNAi screens, where thousands of strains are assessed for phenotype, emphasizes the need for computational methods for automated parameterization of generated behaviors. In addition, behaviors can be modulated upon external cues like temperature, O2 and CO2 concentrations, mechanosensory and chemosensory inputs. Different machine vision tools have been developed to aid researchers in their efforts to inventory and characterize defined behavioral “outputs”. Here we aim at providing an overview of different worm-tracking packages or video analysis tools designed to quantify different aspects of locomotion such as the occurrence of directional changes (turns, omega bends), curvature of the sinusoidal shape (amplitude, body bend angles) and velocity (speed, backward or forward movement)

A lab-on-a-disc platform enables serial monitoring of individual CTCs associated with tumor progression during EGFR-targeted therapy for patients with NSCLC

Rationale: Unlike traditional biopsy, liquid biopsy, which is a largely non-invasive diagnostic and monitoring tool, can be performed more frequently to better track tumors and mutations over time and to validate the efficiency of a cancer treatment. Circulating tumor cells (CTCs) are considered promising liquid biopsy biomarkers; however, their use in clinical settings is limited by high costs and a low throughput of standard platforms for CTC enumeration and analysis. In this study, we used a label-free, high-throughput method for CTC isolation directly from whole blood of patients using a standalone, clinical setting-friendly platform. Methods: A CTC-based liquid biopsy approach was used to examine the efficacy of therapy and emergent drug resistance via longitudinal monitoring of CTC counts, DNA mutations, and single-cell-level gene expression in a prospective cohort of 40 patients with epidermal growth factor receptor (EGFR)-mutant non-small cell lung cancer. Results: The change ratio of the CTC counts was associated with tumor response, detected by CT scan, while the baseline CTC counts did not show association with progression-free survival or overall survival. We achieved a 100% concordance rate for the detection of EGFR mutation, including emergence of T790M, between tumor tissue and CTCs. More importantly, our data revealed the importance of the analysis of the epithelial/mesenchymal signature of individual pretreatment CTCs to predict drug responsiveness in patients. Conclusion: The fluid-assisted separation technology disc platform enables serial monitoring of CTC counts, DNA mutations, as well as unbiased molecular characterization of individual CTCs associated with tumor progression during targeted therapy

Miniaturizing High Throughput Droplet Assays For Ultrasensitive Molecular Detection On A Portable Platform

Digital droplet assays – in which biological samples are compartmentalized into millions of femtoliter-volume droplets and interrogated individually – have generated enormous enthusiasm for their ability to detect biomarkers with single-molecule sensitivity. These assays have untapped potential for point-of-care diagnostics but are mainly confined to laboratory settings due to the instrumentation necessary to serially generate, control, and measure millions of compartments. To address this challenge, we developed an optofluidic platform that miniaturizes digital assays into a mobile format by parallelizing their operation. This technology has three key innovations: 1. the integration and parallel operation of hundred droplet generators onto a single chip that operates \u3e100x faster than a single droplet generator. 2. the fluorescence detection of droplets at \u3e100x faster than conventional in-flow detection using time-domain encoded mobile-phone imaging, and 3. the integration of on-chip delay lines and sample processing to allow serum-to-answer device operation. By using this time-domain modulation with cloud computing, we overcome the low framerate of digital imaging, and achieve throughputs of one million droplets per second. To demonstrate the power of this approach, we performed a duplex digital enzyme-linked immunosorbent assay (ELISA) in serum to show a 1000x improvement over standard ELISA and matching that of the existing laboratory-based gold standard digital ELISA system. This work has broad potential for ultrasensitive, highly multiplexed detection, in a mobile format. Building on our initial demonstration, we explored the following: (i) we demonstrated that the platform can be extended to \u3e100x multiplexing by using time-domain encoded light sources to detect color-coded beads that each correspond to a unique assay, (ii) we demonstrated that the platform can be extended to the detection of nucleic acid by implementing polymerase chain reaction, and (iii) we demonstrated that sensitivity can be improved with a nanoparticle-enhanced ELISA. Clinical applications can be expanded to measure numerous biomarkers simultaneously such as surface markers, proteins, and nucleic acids. Ultimately, by building a robust device, suitable for low-cost implementation with ultrasensitive capabilities, this platform can be used as a tool to quantify numerous medical conditions and help physicians choose optimal treatment strategies to enable personalized medicine in a cost-effective manner