285 research outputs found
Temporal Consistency Objectives Regularize the Learning of Disentangled Representations
There has been an increasing focus in learning interpretable feature
representations, particularly in applications such as medical image analysis
that require explainability, whilst relying less on annotated data (since
annotations can be tedious and costly). Here we build on recent innovations in
style-content representations to learn anatomy, imaging characteristics
(appearance) and temporal correlations. By introducing a self-supervised
objective of predicting future cardiac phases we improve disentanglement. We
propose a temporal transformer architecture that given an image conditioned on
phase difference, it predicts a future frame. This forces the anatomical
decomposition to be consistent with the temporal cardiac contraction in cine
MRI and to have semantic meaning with less need for annotations. We demonstrate
that using this regularization, we achieve competitive results and improve
semi-supervised segmentation, especially when very few labelled data are
available. Specifically, we show Dice increase of up to 19\% and 7\% compared
to supervised and semi-supervised approaches respectively on the ACDC dataset.
Code is available at: https://github.com/gvalvano/sdtnet .Comment: 9 pages, 4 figures (1 .gif), 1 tabl
Bio-Inspired Multi-Layer Spiking Neural Network Extracts Discriminative Features from Speech Signals
Spiking neural networks (SNNs) enable power-efficient implementations due to
their sparse, spike-based coding scheme. This paper develops a bio-inspired SNN
that uses unsupervised learning to extract discriminative features from speech
signals, which can subsequently be used in a classifier. The architecture
consists of a spiking convolutional/pooling layer followed by a fully connected
spiking layer for feature discovery. The convolutional layer of leaky,
integrate-and-fire (LIF) neurons represents primary acoustic features. The
fully connected layer is equipped with a probabilistic spike-timing-dependent
plasticity learning rule. This layer represents the discriminative features
through probabilistic, LIF neurons. To assess the discriminative power of the
learned features, they are used in a hidden Markov model (HMM) for spoken digit
recognition. The experimental results show performance above 96% that compares
favorably with popular statistical feature extraction methods. Our results
provide a novel demonstration of unsupervised feature acquisition in an SNN
Collaborative Layer-wise Discriminative Learning in Deep Neural Networks
Intermediate features at different layers of a deep neural network are known
to be discriminative for visual patterns of different complexities. However,
most existing works ignore such cross-layer heterogeneities when classifying
samples of different complexities. For example, if a training sample has
already been correctly classified at a specific layer with high confidence, we
argue that it is unnecessary to enforce rest layers to classify this sample
correctly and a better strategy is to encourage those layers to focus on other
samples.
In this paper, we propose a layer-wise discriminative learning method to
enhance the discriminative capability of a deep network by allowing its layers
to work collaboratively for classification. Towards this target, we introduce
multiple classifiers on top of multiple layers. Each classifier not only tries
to correctly classify the features from its input layer, but also coordinates
with other classifiers to jointly maximize the final classification
performance. Guided by the other companion classifiers, each classifier learns
to concentrate on certain training examples and boosts the overall performance.
Allowing for end-to-end training, our method can be conveniently embedded into
state-of-the-art deep networks. Experiments with multiple popular deep
networks, including Network in Network, GoogLeNet and VGGNet, on scale-various
object classification benchmarks, including CIFAR100, MNIST and ImageNet, and
scene classification benchmarks, including MIT67, SUN397 and Places205,
demonstrate the effectiveness of our method. In addition, we also analyze the
relationship between the proposed method and classical conditional random
fields models.Comment: To appear in ECCV 2016. Maybe subject to minor changes before
camera-ready versio
Convolutional LSTM Networks for Subcellular Localization of Proteins
Machine learning is widely used to analyze biological sequence data.
Non-sequential models such as SVMs or feed-forward neural networks are often
used although they have no natural way of handling sequences of varying length.
Recurrent neural networks such as the long short term memory (LSTM) model on
the other hand are designed to handle sequences. In this study we demonstrate
that LSTM networks predict the subcellular location of proteins given only the
protein sequence with high accuracy (0.902) outperforming current state of the
art algorithms. We further improve the performance by introducing convolutional
filters and experiment with an attention mechanism which lets the LSTM focus on
specific parts of the protein. Lastly we introduce new visualizations of both
the convolutional filters and the attention mechanisms and show how they can be
used to extract biological relevant knowledge from the LSTM networks
RoboCup 2D Soccer Simulation League: Evaluation Challenges
We summarise the results of RoboCup 2D Soccer Simulation League in 2016
(Leipzig), including the main competition and the evaluation round. The
evaluation round held in Leipzig confirmed the strength of RoboCup-2015
champion (WrightEagle, i.e. WE2015) in the League, with only eventual finalists
of 2016 competition capable of defeating WE2015. An extended, post-Leipzig,
round-robin tournament which included the top 8 teams of 2016, as well as
WE2015, with over 1000 games played for each pair, placed WE2015 third behind
the champion team (Gliders2016) and the runner-up (HELIOS2016). This
establishes WE2015 as a stable benchmark for the 2D Simulation League. We then
contrast two ranking methods and suggest two options for future evaluation
challenges. The first one, "The Champions Simulation League", is proposed to
include 6 previous champions, directly competing against each other in a
round-robin tournament, with the view to systematically trace the advancements
in the League. The second proposal, "The Global Challenge", is aimed to
increase the realism of the environmental conditions during the simulated
games, by simulating specific features of different participating countries.Comment: 12 pages, RoboCup-2017, Nagoya, Japan, July 201
LEED: Label-Free Expression Editing via Disentanglement
Recent studies on facial expression editing have obtained very promising
progress. On the other hand, existing methods face the constraint of requiring
a large amount of expression labels which are often expensive and
time-consuming to collect. This paper presents an innovative label-free
expression editing via disentanglement (LEED) framework that is capable of
editing the expression of both frontal and profile facial images without
requiring any expression label. The idea is to disentangle the identity and
expression of a facial image in the expression manifold, where the neutral face
captures the identity attribute and the displacement between the neutral image
and the expressive image captures the expression attribute. Two novel losses
are designed for optimal expression disentanglement and consistent synthesis,
including a mutual expression information loss that aims to extract pure
expression-related features and a siamese loss that aims to enhance the
expression similarity between the synthesized image and the reference image.
Extensive experiments over two public facial expression datasets show that LEED
achieves superior facial expression editing qualitatively and quantitatively.Comment: Accepted to ECCV 202
An ensemble of epoch-wise empirical Bayes for few-shot learning
Ministry of Education, Singapore under its Academic Research Funding Tier
A Theory of Cheap Control in Embodied Systems
We present a framework for designing cheap control architectures for embodied
agents. Our derivation is guided by the classical problem of universal
approximation, whereby we explore the possibility of exploiting the agent's
embodiment for a new and more efficient universal approximation of behaviors
generated by sensorimotor control. This embodied universal approximation is
compared with the classical non-embodied universal approximation. To exemplify
our approach, we present a detailed quantitative case study for policy models
defined in terms of conditional restricted Boltzmann machines. In contrast to
non-embodied universal approximation, which requires an exponential number of
parameters, in the embodied setting we are able to generate all possible
behaviors with a drastically smaller model, thus obtaining cheap universal
approximation. We test and corroborate the theory experimentally with a
six-legged walking machine. The experiments show that the sufficient controller
complexity predicted by our theory is tight, which means that the theory has
direct practical implications. Keywords: cheap design, embodiment, sensorimotor
loop, universal approximation, conditional restricted Boltzmann machineComment: 27 pages, 10 figure
Automatically Segmenting the Left Atrium from Cardiac Images Using Successive 3D U-Nets and a Contour Loss
International audienceRadiological imaging offers effective measurement of anatomy, which is useful in disease diagnosis and assessment. Previous study has shown that the left atrial wall remodeling can provide information to predict treatment outcome in atrial fibrillation. Nevertheless, the segmentation of the left atrial structures from medical images is still very time-consuming. Current advances in neural network may help creating automatic segmentation models that reduce the workload for clinicians. In this preliminary study, we propose automated, two-stage, three-dimensional U-Nets with convolutional neural network, for the challenging task of left atrial segmentation. Unlike previous two-dimensional image segmentation methods, we use 3D U-Nets to obtain the heart cavity directly in 3D. The dual 3D U-Net structure consists of, a first U-Net to coarsely segment and locate the left atrium, and a second U-Net to accurately segment the left atrium under higher resolution. In addition, we introduce a Contour loss based on additional distance information to adjust the final segmentation. We randomly split the data into training datasets (80 subjects) and validation datasets (20 subjects) to train multiple models, with different augmentation setting. Experiments show that the average Dice coefficients for validation datasets are around 0.91 - 0.92, the sensitivity around 0.90-0.94 and the specificity 0.99. Compared with traditional Dice loss, models trained with Contour loss in general offer smaller Hausdorff distance with similar Dice coefficient, and have less connected components in predictions. Finally, we integrate several trained models in an ensemble prediction to segment testing datasets
Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks
Biological plastic neural networks are systems of extraordinary computational
capabilities shaped by evolution, development, and lifetime learning. The
interplay of these elements leads to the emergence of adaptive behavior and
intelligence. Inspired by such intricate natural phenomena, Evolved Plastic
Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed
plastic neural networks with a large variety of dynamics, architectures, and
plasticity rules: these artificial systems are composed of inputs, outputs, and
plastic components that change in response to experiences in an environment.
These systems may autonomously discover novel adaptive algorithms, and lead to
hypotheses on the emergence of biological adaptation. EPANNs have seen
considerable progress over the last two decades. Current scientific and
technological advances in artificial neural networks are now setting the
conditions for radically new approaches and results. In particular, the
limitations of hand-designed networks could be overcome by more flexible and
innovative solutions. This paper brings together a variety of inspiring ideas
that define the field of EPANNs. The main methods and results are reviewed.
Finally, new opportunities and developments are presented
- …