Multimodal Classification of Urban Micro-Events

In this paper we seek methods to effectively detect urban micro-events. Urban micro-events are events which occur in cities, have limited geographical coverage and typically affect only a small group of citizens. Because of their scale these are difficult to identify in most data sources. However, by using citizen sensing to gather data, detecting them becomes feasible. The data gathered by citizen sensing is often multimodal and, as a consequence, the information required to detect urban micro-events is distributed over multiple modalities. This makes it essential to have a classifier capable of combining them. In this paper we explore several methods of creating such a classifier, including early, late, hybrid fusion and representation learning using multimodal graphs. We evaluate performance on a real world dataset obtained from a live citizen reporting system. We show that a multimodal approach yields higher performance than unimodal alternatives. Furthermore, we demonstrate that our hybrid combination of early and late fusion with multimodal embeddings performs best in classification of urban micro-events

Multi-branch Convolutional Neural Network for Multiple Sclerosis Lesion Segmentation

In this paper, we present an automated approach for segmenting multiple sclerosis (MS) lesions from multi-modal brain magnetic resonance images. Our method is based on a deep end-to-end 2D convolutional neural network (CNN) for slice-based segmentation of 3D volumetric data. The proposed CNN includes a multi-branch downsampling path, which enables the network to encode information from multiple modalities separately. Multi-scale feature fusion blocks are proposed to combine feature maps from different modalities at different stages of the network. Then, multi-scale feature upsampling blocks are introduced to upsize combined feature maps to leverage information from lesion shape and location. We trained and tested the proposed model using orthogonal plane orientations of each 3D modality to exploit the contextual information in all directions. The proposed pipeline is evaluated on two different datasets: a private dataset including 37 MS patients and a publicly available dataset known as the ISBI 2015 longitudinal MS lesion segmentation challenge dataset, consisting of 14 MS patients. Considering the ISBI challenge, at the time of submission, our method was amongst the top performing solutions. On the private dataset, using the same array of performance metrics as in the ISBI challenge, the proposed approach shows high improvements in MS lesion segmentation compared with other publicly available tools.Comment: This paper has been accepted for publication in NeuroImag

HeMIS: Hetero-Modal Image Segmentation

We introduce a deep learning image segmentation framework that is extremely robust to missing imaging modalities. Instead of attempting to impute or synthesize missing data, the proposed approach learns, for each modality, an embedding of the input image into a single latent vector space for which arithmetic operations (such as taking the mean) are well defined. Points in that space, which are averaged over modalities available at inference time, can then be further processed to yield the desired segmentation. As such, any combinatorial subset of available modalities can be provided as input, without having to learn a combinatorial number of imputation models. Evaluated on two neurological MRI datasets (brain tumors and MS lesions), the approach yields state-of-the-art segmentation results when provided with all modalities; moreover, its performance degrades remarkably gracefully when modalities are removed, significantly more so than alternative mean-filling or other synthesis approaches.Comment: Accepted as an oral presentation at MICCAI 201

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table

Gaussian mixture model based probabilistic modeling of images for medical image segmentation

In this paper, we propose a novel image segmentation algorithm that is based on the probability distributions of the object and background. It uses the variational level sets formulation with a novel region based term in addition to the edge-based term giving a complementary functional, that can potentially result in a robust segmentation of the images. The main theme of the method is that in most of the medical imaging scenarios, the objects are characterized by some typical characteristics such a color, texture, etc. Consequently, an image can be modeled as a Gaussian mixture of distributions corresponding to the object and background. During the procedure of curve evolution, a novel term is incorporated in the segmentation framework which is based on the maximization of the distance between the GMM corresponding to the object and background. The maximization of this distance using differential calculus potentially leads to the desired segmentation results. The proposed method has been used for segmenting images from three distinct imaging modalities i.e. magnetic resonance imaging (MRI), dermoscopy and chromoendoscopy. Experiments show the effectiveness of the proposed method giving better qualitative and quantitative results when compared with the current state-of-the-art. INDEX TERMS Gaussian Mixture Model, Level Sets, Active Contours, Biomedical Engineerin

Predictive models for multibiometric systems

Recognizing a subject given a set of biometrics is a fundamental pattern recognition problem. This paper builds novel statistical models for multibiometric systems using geometric and multinomial distributions. These models are generic as they are only based on the similarity scores produced by a recognition system. They predict the bounds on the range of indices within which a test subject is likely to be present in a sorted set of similarity scores. These bounds are then used in the multibiometric recognition system to predict a smaller subset of subjects from the database as probable candidates for a given test subject. Experimental results show that the proposed models enhance the recognition rate beyond the underlying matching algorithms for multiple face views, fingerprints, palm prints, irises and their combinations