82 research outputs found
Testing for Differences in Gaussian Graphical Models: Applications to Brain Connectivity
Functional brain networks are well described and estimated from data with
Gaussian Graphical Models (GGMs), e.g. using sparse inverse covariance
estimators. Comparing functional connectivity of subjects in two populations
calls for comparing these estimated GGMs. Our goal is to identify differences
in GGMs known to have similar structure. We characterize the uncertainty of
differences with confidence intervals obtained using a parametric distribution
on parameters of a sparse estimator. Sparse penalties enable statistical
guarantees and interpretable models even in high-dimensional and low-sample
settings. Characterizing the distributions of sparse models is inherently
challenging as the penalties produce a biased estimator. Recent work invokes
the sparsity assumptions to effectively remove the bias from a sparse estimator
such as the lasso. These distributions can be used to give confidence intervals
on edges in GGMs, and by extension their differences. However, in the case of
comparing GGMs, these estimators do not make use of any assumed joint structure
among the GGMs. Inspired by priors from brain functional connectivity we derive
the distribution of parameter differences under a joint penalty when parameters
are known to be sparse in the difference. This leads us to introduce the
debiased multi-task fused lasso, whose distribution can be characterized in an
efficient manner. We then show how the debiased lasso and multi-task fused
lasso can be used to obtain confidence intervals on edge differences in GGMs.
We validate the techniques proposed on a set of synthetic examples as well as
neuro-imaging dataset created for the study of autism
The Lov\'asz-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks
The Jaccard index, also referred to as the intersection-over-union score, is
commonly employed in the evaluation of image segmentation results given its
perceptual qualities, scale invariance - which lends appropriate relevance to
small objects, and appropriate counting of false negatives, in comparison to
per-pixel losses. We present a method for direct optimization of the mean
intersection-over-union loss in neural networks, in the context of semantic
image segmentation, based on the convex Lov\'asz extension of submodular
losses. The loss is shown to perform better with respect to the Jaccard index
measure than the traditionally used cross-entropy loss. We show quantitative
and qualitative differences between optimizing the Jaccard index per image
versus optimizing the Jaccard index taken over an entire dataset. We evaluate
the impact of our method in a semantic segmentation pipeline and show
substantially improved intersection-over-union segmentation scores on the
Pascal VOC and Cityscapes datasets using state-of-the-art deep learning
segmentation architectures.Comment: Accepted as a conference paper at CVPR 201
Greedy Bayesian Posterior Approximation with Deep Ensembles
Ensembles of independently trained neural networks are a state-of-the-art
approach to estimate predictive uncertainty in Deep Learning, and can be
interpreted as an approximation of the posterior distribution via a mixture of
delta functions. The training of ensembles relies on non-convexity of the loss
landscape and random initialization of their individual members, making the
resulting posterior approximation uncontrolled. This paper proposes a novel and
principled method to tackle this limitation, minimizing an -divergence
between the true posterior and a kernel density estimator in a function space.
We analyze this objective from a combinatorial point of view, and show that it
is submodular with respect to mixture components for any . Subsequently, we
consider the problem of ensemble construction, and from the marginal gain of
the total objective, we derive a novel diversity term for training ensembles
greedily. The performance of our approach is demonstrated on computer vision
out-of-distribution detection benchmarks in a range of architectures trained on
multiple datasets. The source code of our method is publicly available at
https://github.com/MIPT-Oulu/greedy_ensembles_training
Dense Transformer based Enhanced Coding Network for Unsupervised Metal Artifact Reduction
CT images corrupted by metal artifacts have serious negative effects on
clinical diagnosis. Considering the difficulty of collecting paired data with
ground truth in clinical settings, unsupervised methods for metal artifact
reduction are of high interest. However, it is difficult for previous
unsupervised methods to retain structural information from CT images while
handling the non-local characteristics of metal artifacts. To address these
challenges, we proposed a novel Dense Transformer based Enhanced Coding Network
(DTEC-Net) for unsupervised metal artifact reduction. Specifically, we
introduce a Hierarchical Disentangling Encoder, supported by the high-order
dense process, and transformer to obtain densely encoded sequences with
long-range correspondence. Then, we present a second-order disentanglement
method to improve the dense sequence's decoding process. Extensive experiments
and model discussions illustrate DTEC-Net's effectiveness, which outperforms
the previous state-of-the-art methods on a benchmark dataset, and greatly
reduces metal artifacts while restoring richer texture details
Jaccard Metric Losses: Optimizing the Jaccard Index with Soft Labels
IoU losses are surrogates that directly optimize the Jaccard index. In
semantic segmentation, leveraging IoU losses as part of the loss function is
shown to perform better with respect to the Jaccard index measure than
optimizing pixel-wise losses such as the cross-entropy loss alone. The most
notable IoU losses are the soft Jaccard loss and the Lovasz-Softmax loss.
However, these losses are incompatible with soft labels which are ubiquitous in
machine learning. In this paper, we propose Jaccard metric losses (JMLs), which
are identical to the soft Jaccard loss in a standard setting with hard labels,
but are compatible with soft labels. With JMLs, we study two of the most
popular use cases of soft labels: label smoothing and knowledge distillation.
With a variety of architectures, our experiments show significant improvements
over the cross-entropy loss on three semantic segmentation datasets
(Cityscapes, PASCAL VOC and DeepGlobe Land), and our simple approach
outperforms state-of-the-art knowledge distillation methods by a large margin.
Code is available at:
\href{https://github.com/zifuwanggg/JDTLosses}{https://github.com/zifuwanggg/JDTLosses}.Comment: Submitted to ICML2023. Code is available at
https://github.com/zifuwanggg/JDTLosse
- …