435 research outputs found
From Maxout to Channel-Out: Encoding Information on Sparse Pathways
Motivated by an important insight from neural science, we propose a new
framework for understanding the success of the recently proposed "maxout"
networks. The framework is based on encoding information on sparse pathways and
recognizing the correct pathway at inference time. Elaborating further on this
insight, we propose a novel deep network architecture, called "channel-out"
network, which takes a much better advantage of sparse pathway encoding. In
channel-out networks, pathways are not only formed a posteriori, but they are
also actively selected according to the inference outputs from the lower
layers. From a mathematical perspective, channel-out networks can represent a
wider class of piece-wise continuous functions, thereby endowing the network
with more expressive power than that of maxout networks. We test our
channel-out networks on several well-known image classification benchmarks,
setting new state-of-the-art performance on CIFAR-100 and STL-10, which
represent some of the "harder" image classification benchmarks.Comment: 10 pages including the appendix, 9 figure
Deep learning for Gaussian process tomography model selection using the ASDEX Upgrade SXR system
Gaussian process tomography (GPT) is a method used for obtaining real-time
tomographic reconstructions of the plasma emissivity profile in a tokamak,
given some model for the underlying physical processes involved. GPT can also
be used, thanks to Bayesian formalism, to perform model selection -- i.e.,
comparing different models and choosing the one with maximum evidence. However,
the computations involved in this particular step may become slow for data with
high dimensionality, especially when comparing the evidence for many different
models. Using measurements collected by the ASDEX Upgrade Soft X-ray (SXR)
diagnostic, we train a convolutional neural network (CNN) to map SXR
tomographic projections to the corresponding GPT model whose evidence is
highest. We then compare the network's results, and the time required to
calculate them, with those obtained through analytical Bayesian formalism. In
addition, we use the network's classifications to produce tomographic
reconstructions of the plasma emissivity profile, whose quality we evaluate by
comparing their projection into measurement space with the existing
measurements themselves
Decoupled Actor-Critic
Actor-Critic methods are in a stalemate of two seemingly irreconcilable
problems. Firstly, critic proneness towards overestimation requires sampling
temporal-difference targets from a conservative policy optimized using
lower-bound Q-values. Secondly, well-known results show that policies that are
optimistic in the face of uncertainty yield lower regret levels. To remedy this
dichotomy, we propose Decoupled Actor-Critic (DAC). DAC is an off-policy
algorithm that learns two distinct actors by gradient backpropagation: a
conservative actor used for temporal-difference learning and an optimistic
actor used for exploration. We test DAC on DeepMind Control tasks in low and
high replay ratio regimes and ablate multiple design choices. Despite minimal
computational overhead, DAC achieves state-of-the-art performance and sample
efficiency on locomotion tasks.Comment: Preprin
Country-wide retrieval of forest structure from optical and SAR satellite imagery with deep ensembles
Monitoring and managing Earth’s forests in an informed manner is an important requirement for addressing challenges like biodiversity loss and climate change. While traditional in situ or aerial campaigns for forest assessments provide accurate data for analysis at regional level, scaling them to entire countries and beyond with high temporal resolution is hardly possible. In this work, we propose a method based on deep ensembles that densely estimates forest structure variables at country-scale with 10-m resolution, using freely available satellite imagery as input. Our method jointly transforms Sentinel-2 optical images and Sentinel-1 syntheticaperture radar images into maps of five different forest structure variables: 95th height percentile, mean height, density, Gini coefficient, and fractional cover. We train and test our model on reference data from 41 airborne laser scanning missions across Norway and demonstrate that it is able to generalize to unseen test regions, achieving normalized mean absolute errors between 11% and 15%, depending on the variable. Our work is also the first to propose a variant of so-called Bayesian deep learning to densely predict multiple forest structure variables with well-calibrated uncertainty estimates from satellite imagery. The uncertainty information increases the trustworthiness of the model and its suitability for downstream tasks that require reliable confidence estimates as a basis for decision making. We present an extensive set of experiments to validate the accuracy of the predicted maps as well as the quality of the predicted uncertainties. To demonstrate scalability, we provide Norway-wide maps for the five forest structure variables.publishedVersio
- …