32,795 research outputs found
Transportation mode recognition fusing wearable motion, sound and vision sensors
We present the first work that investigates the potential of improving the performance of transportation mode recognition through fusing multimodal data from wearable sensors: motion, sound and vision. We first train three independent deep neural network (DNN) classifiers, which work with the three types of sensors, respectively. We then propose two schemes that fuse the classification results from the three mono-modal classifiers. The first scheme makes an ensemble decision with fixed rules including Sum, Product, Majority Voting, and Borda Count. The second scheme is an adaptive fuser built as another classifier (including Naive Bayes, Decision Tree, Random Forest and Neural Network) that learns enhanced predictions by combining the outputs from the three mono-modal classifiers. We verify the advantage of the proposed method with the state-of-the-art Sussex-Huawei Locomotion and Transportation (SHL) dataset recognizing the eight transportation activities: Still, Walk, Run, Bike, Bus, Car, Train and Subway. We achieve F1 scores of 79.4%, 82.1% and 72.8% with the mono-modal motion, sound and vision classifiers, respectively. The F1 score is remarkably improved to 94.5% and 95.5% by the two data fusion schemes, respectively. The recognition performance can be further improved with a post-processing scheme that exploits the temporal continuity of transportation. When assessing generalization of the model to unseen data, we show that while performance is reduced - as expected - for each individual classifier, the benefits of fusion are retained with performance improved by 15 percentage points. Besides the actual performance increase, this work, most importantly, opens up the possibility for dynamically fusing modalities to achieve distinct power-performance trade-off at run time
Generative Adversarial Text to Image Synthesis
Automatic synthesis of realistic images from text would be interesting and
useful, but current AI systems are still far from this goal. However, in recent
years generic and powerful recurrent neural network architectures have been
developed to learn discriminative text feature representations. Meanwhile, deep
convolutional generative adversarial networks (GANs) have begun to generate
highly compelling images of specific categories, such as faces, album covers,
and room interiors. In this work, we develop a novel deep architecture and GAN
formulation to effectively bridge these advances in text and image model- ing,
translating visual concepts from characters to pixels. We demonstrate the
capability of our model to generate plausible images of birds and flowers from
detailed text descriptions.Comment: ICML 201
Scalable multimodal convolutional networks for brain tumour segmentation
Brain tumour segmentation plays a key role in computer-assisted surgery. Deep
neural networks have increased the accuracy of automatic segmentation
significantly, however these models tend to generalise poorly to different
imaging modalities than those for which they have been designed, thereby
limiting their applications. For example, a network architecture initially
designed for brain parcellation of monomodal T1 MRI can not be easily
translated into an efficient tumour segmentation network that jointly utilises
T1, T1c, Flair and T2 MRI. To tackle this, we propose a novel scalable
multimodal deep learning architecture using new nested structures that
explicitly leverage deep features within or across modalities. This aims at
making the early layers of the architecture structured and sparse so that the
final architecture becomes scalable to the number of modalities. We evaluate
the scalable architecture for brain tumour segmentation and give evidence of
its regularisation effect compared to the conventional concatenation approach.Comment: Paper accepted at MICCAI 201
- …