138 research outputs found
Parallel Selected Inversion for Space-Time Gaussian Markov Random Fields
Performing a Bayesian inference on large spatio-temporal models requires
extracting inverse elements of large sparse precision matrices for marginal
variances. Although direct matrix factorizations can be used for the inversion,
such methods fail to scale well for distributed problems when run on large
computing clusters. On the contrary, Krylov subspace methods for the selected
inversion have been gaining traction. We propose a parallel hybrid approach
based on domain decomposition, which extends the Rao-Blackwellized Monte Carlo
estimator for distributed precision matrices. Our approach exploits the
strength of Krylov subspace methods as global solvers and efficiency of direct
factorizations as base case solvers to compute the marginal variances using a
divide-and-conquer strategy. By introducing subdomain overlaps, one can achieve
a greater accuracy at an increased computational effort with little to no
additional communication. We demonstrate the speed improvements on both
simulated models and a massive US daily temperature data.Comment: 17 pages, 7 figure
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
Large language models (LLMs) have been recently leveraged as training data
generators for various natural language processing (NLP) tasks. While previous
research has explored different approaches to training models using generated
data, they generally rely on simple class-conditional prompts, which may limit
the diversity of the generated data and inherit systematic biases of LLM. Thus,
we investigate training data generation with diversely attributed prompts
(e.g., specifying attributes like length and style), which have the potential
to yield diverse and attributed generated data. Our investigation focuses on
datasets with high cardinality and diverse domains, wherein we demonstrate that
attributed prompts outperform simple class-conditional prompts in terms of the
resulting model's performance. Additionally, we present a comprehensive
empirical study on data generation encompassing vital aspects like bias,
diversity, and efficiency, and highlight three key observations: firstly,
synthetic datasets generated by simple prompts exhibit significant biases, such
as regional bias; secondly, attribute diversity plays a pivotal role in
enhancing model performance; lastly, attributed prompts achieve the performance
of simple class-conditional prompts while utilizing only 5\% of the querying
cost of ChatGPT associated with the latter. We release the generated dataset
and used prompts to facilitate future research. The data and code will be
available on \url{https://github.com/yueyu1030/AttrPrompt}.Comment: Work in progress. A shorter version is accepted to the ICML DMLR
worksho
New Frontiers in Bayesian Modeling Using the INLA Package in R
The INLA package provides a tool for computationally efficient Bayesian modeling and inference for various widely used models, more formally the class of latent Gaussian models. It is a non-sampling based framework which provides approximate results for Bayesian inference, using sparse matrices. The swift uptake of this framework for Bayesian modeling is rooted in the computational efficiency of the approach and catalyzed by the demand presented by the big data era. In this paper, we present new developments within the INLA package with the aim to provide a computationally efficient mechanism for the Bayesian inference of relevant challenging situations
Bayesian space-time gap filling for inference on extreme hot-spots: an application to Red Sea surface temperatures
We develop a method for probabilistic prediction of extreme value hot-spots
in a spatio-temporal framework, tailored to big datasets containing important
gaps. In this setting, direct calculation of summaries from data, such as the
minimum over a space-time domain, is not possible. To obtain predictive
distributions for such cluster summaries, we propose a two-step approach. We
first model marginal distributions with a focus on accurate modeling of the
right tail and then, after transforming the data to a standard Gaussian scale,
we estimate a Gaussian space-time dependence model defined locally in the time
domain for the space-time subregions where we want to predict. In the first
step, we detrend the mean and standard deviation of the data and fit a
spatially resolved generalized Pareto distribution to apply a correction of the
upper tail. To ensure spatial smoothness of the estimated trends, we either
pool data using nearest-neighbor techniques, or apply generalized additive
regression modeling. To cope with high space-time resolution of data, the local
Gaussian models use a Markov representation of the Mat\'ern correlation
function based on the stochastic partial differential equations (SPDE)
approach. In the second step, they are fitted in a Bayesian framework through
the integrated nested Laplace approximation implemented in R-INLA. Finally,
posterior samples are generated to provide statistical inferences through
Monte-Carlo estimation. Motivated by the 2019 Extreme Value Analysis data
challenge, we illustrate our approach to predict the distribution of local
space-time minima in anomalies of Red Sea surface temperatures, using a gridded
dataset (11315 days, 16703 pixels) with artificially generated gaps. In
particular, we show the improved performance of our two-step approach over a
purely Gaussian model without tail transformations
Optimization with Sparsity-Inducing Penalties
Sparse estimation methods are aimed at using or obtaining parsimonious
representations of data or models. They were first dedicated to linear variable
selection but numerous extensions have now emerged such as structured sparsity
or kernel selection. It turns out that many of the related estimation problems
can be cast as convex optimization problems by regularizing the empirical risk
with appropriate non-smooth norms. The goal of this paper is to present from a
general perspective optimization tools and techniques dedicated to such
sparsity-inducing penalties. We cover proximal methods, block-coordinate
descent, reweighted -penalized techniques, working-set and homotopy
methods, as well as non-convex formulations and extensions, and provide an
extensive set of experiments to compare various algorithms from a computational
point of view
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries
This two-volume set LNCS 12962 and 12963 constitutes the thoroughly refereed proceedings of the 7th International MICCAI Brainlesion Workshop, BrainLes 2021, as well as the RSNA-ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge, the Federated Tumor Segmentation (FeTS) Challenge, the Cross-Modality Domain Adaptation (CrossMoDA) Challenge, and the challenge on Quantification of Uncertainties in Biomedical Image Quantification (QUBIQ). These were held jointly at the 23rd Medical Image Computing for Computer Assisted Intervention Conference, MICCAI 2020, in September 2021. The 91 revised papers presented in these volumes were selected form 151 submissions. Due to COVID-19 pandemic the conference was held virtually. This is an open access book
- …