Driver Distraction Identification with an Ensemble of Convolutional Neural Networks

The World Health Organization (WHO) reported 1.25 million deaths yearly due to road traffic accidents worldwide and the number has been continuously increasing over the last few years. Nearly fifth of these accidents are caused by distracted drivers. Existing work of distracted driver detection is concerned with a small set of distractions (mostly, cell phone usage). Unreliable ad-hoc methods are often used.In this paper, we present the first publicly available dataset for driver distraction identification with more distraction postures than existing alternatives. In addition, we propose a reliable deep learning-based solution that achieves a 90% accuracy. The system consists of a genetically-weighted ensemble of convolutional neural networks, we show that a weighted ensemble of classifiers using a genetic algorithm yields in a better classification confidence. We also study the effect of different visual elements in distraction detection by means of face and hand localizations, and skin segmentation. Finally, we present a thinned version of our ensemble that could achieve 84.64% classification accuracy and operate in a real-time environment.Comment: arXiv admin note: substantial text overlap with arXiv:1706.0949

Augmenting biological pathway extraction with synthetic data and active learning

The corpus of biomedical literature is growing rapidly as many papers are recorded in PubMed every day. These papers often contain high-quality biological pathways in their figures/text, which are great resources for studying biological mechanisms and precision medicine. However, it can take a long time for many of these works to be put into practical use as each paper's contributions need to be curated by experts. This, often lengthy, process causes professional practice to lag behind research. To speed up this process, I helped develop a pipeline that integrates NLP and object detection processing to extract gene relationships reported in articles' figures and text. This pipeline was able to extract such relationships with high precision and recall on a small, annotated set. However, extending this pipeline for improved generalization and new settings was limited by the number of high-quality annotations available. Such labeled data is very time consuming to collect and traditional augmentations were observed to generate diminishing returns. To address this shortcoming, I developed an approach for generating purely synthetic data for object detection on biological pathway diagrams based on a set of rules and domain knowledge. Our method iteratively generates each pathway relationship uniquely and is demonstrated to improve the generalization of our object detection model significantly across a variety of settings. Additionally, with the capability to generate unique and informative samples, we integrated our synthetic generation methodology into an active learning setting. While traditional active learning relies on a pool of unlabeled data to draw from with an acquisition function, our method is pool-less and does not require any acquisition function. Instead, we generate each batch of data uniquely based on the training losses from the previous batch. Pool-less Active Learning (PAL) via synthetic data generation is demonstrated to reduce the number of iterations required for model convergence during training on pathway figures.Includes bibliographical references

Understanding of Object Manipulation Actions Using Human Multi-Modal Sensory Data

Object manipulation actions represent an important share of the Activities of Daily Living (ADLs). In this work, we study how to enable service robots to use human multi-modal data to understand object manipulation actions, and how they can recognize such actions when humans perform them during human-robot collaboration tasks. The multi-modal data in this study consists of videos, hand motion data, applied forces as represented by the pressure patterns on the hand, and measurements of the bending of the fingers, collected as human subjects performed manipulation actions. We investigate two different approaches. In the first one, we show that multi-modal signal (motion, finger bending and hand pressure) generated by the action can be decomposed into a set of primitives that can be seen as its building blocks. These primitives are used to define 24 multi-modal primitive features. The primitive features can in turn be used as an abstract representation of the multi-modal signal and employed for action recognition. In the latter approach, the visual features are extracted from the data using a pre-trained image classification deep convolutional neural network. The visual features are subsequently used to train the classifier. We also investigate whether adding data from other modalities produces a statistically significant improvement in the classifier performance. We show that both approaches produce a comparable performance. This implies that image-based methods can successfully recognize human actions during human-robot collaboration. On the other hand, in order to provide training data for the robot so it can learn how to perform object manipulation actions, multi-modal data provides a better alternative