3,093 research outputs found
Bayesian Gene Set Analysis
Gene expression microarray technologies provide the simultaneous measurements
of a large number of genes. Typical analyses of such data focus on the
individual genes, but recent work has demonstrated that evaluating changes in
expression across predefined sets of genes often increases statistical power
and produces more robust results. We introduce a new methodology for
identifying gene sets that are differentially expressed under varying
experimental conditions. Our approach uses a hierarchical Bayesian framework
where a hyperparameter measures the significance of each gene set. Using
simulated data, we compare our proposed method to alternative approaches, such
as Gene Set Enrichment Analysis (GSEA) and Gene Set Analysis (GSA). Our
approach provides the best overall performance. We also discuss the application
of our method to experimental data based on p53 mutation status
Testing significance of features by lassoed principal components
We consider the problem of testing the significance of features in
high-dimensional settings. In particular, we test for differentially-expressed
genes in a microarray experiment. We wish to identify genes that are associated
with some type of outcome, such as survival time or cancer type. We propose a
new procedure, called Lassoed Principal Components (LPC), that builds upon
existing methods and can provide a sizable improvement. For instance, in the
case of two-class data, a standard (albeit simple) approach might be to compute
a two-sample -statistic for each gene. The LPC method involves projecting
these conventional gene scores onto the eigenvectors of the gene expression
data covariance matrix and then applying an penalty in order to de-noise
the resulting projections. We present a theoretical framework under which LPC
is the logical choice for identifying significant genes, and we show that LPC
can provide a marked reduction in false discovery rates over the conventional
methods on both real and simulated data. Moreover, this flexible procedure can
be applied to a variety of types of data and can be used to improve many
existing methods for the identification of significant features.Comment: Published in at http://dx.doi.org/10.1214/08-AOAS182 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Inverse Ising inference using all the data
We show that a method based on logistic regression, using all the data,
solves the inverse Ising problem far better than mean-field calculations
relying only on sample pairwise correlation functions, while still
computationally feasible for hundreds of nodes. The largest improvement in
reconstruction occurs for strong interactions. Using two examples, a diluted
Sherrington-Kirkpatrick model and a two-dimensional lattice, we also show that
interaction topologies can be recovered from few samples with good accuracy and
that the use of -regularization is beneficial in this process, pushing
inference abilities further into low-temperature regimes.Comment: 5 pages, 2 figures. Accepted versio
Analysis of Models for Decentralized and Collaborative AI on Blockchain
Machine learning has recently enabled large advances in artificial
intelligence, but these results can be highly centralized. The large datasets
required are generally proprietary; predictions are often sold on a per-query
basis; and published models can quickly become out of date without effort to
acquire more data and maintain them. Published proposals to provide models and
data for free for certain tasks include Microsoft Research's Decentralized and
Collaborative AI on Blockchain. The framework allows participants to
collaboratively build a dataset and use smart contracts to share a continuously
updated model on a public blockchain. The initial proposal gave an overview of
the framework omitting many details of the models used and the incentive
mechanisms in real world scenarios. In this work, we evaluate the use of
several models and configurations in order to propose best practices when using
the Self-Assessment incentive mechanism so that models can remain accurate and
well-intended participants that submit correct data have the chance to profit.
We have analyzed simulations for each of three models: Perceptron, Na\"ive
Bayes, and a Nearest Centroid Classifier, with three different datasets:
predicting a sport with user activity from Endomondo, sentiment analysis on
movie reviews from IMDB, and determining if a news article is fake. We compare
several factors for each dataset when models are hosted in smart contracts on a
public blockchain: their accuracy over time, balances of a good and bad user,
and transaction costs (or gas) for deploying, updating, collecting refunds, and
collecting rewards. A free and open source implementation for the Ethereum
blockchain and simulations written in Python is provided at
https://github.com/microsoft/0xDeCA10B. This version has updated gas costs
using newer optimizations written after the original publication.Comment: Accepted to ICBC 202
Foothill: A Quasiconvex Regularization for Edge Computing of Deep Neural Networks
Deep neural networks (DNNs) have demonstrated success for many supervised
learning tasks, ranging from voice recognition, object detection, to image
classification. However, their increasing complexity might yield poor
generalization error that make them hard to be deployed on edge devices.
Quantization is an effective approach to compress DNNs in order to meet these
constraints. Using a quasiconvex base function in order to construct a binary
quantizer helps training binary neural networks (BNNs) and adding noise to the
input data or using a concrete regularization function helps to improve
generalization error. Here we introduce foothill function, an infinitely
differentiable quasiconvex function. This regularizer is flexible enough to
deform towards and penalties. Foothill can be used as a binary
quantizer, as a regularizer, or as a loss. In particular, we show this
regularizer reduces the accuracy gap between BNNs and their full-precision
counterpart for image classification on ImageNet.Comment: Accepted in 16th International Conference of Image Analysis and
Recognition (ICIAR 2019
Differentially Private Model Selection with Penalized and Constrained Likelihood
In statistical disclosure control, the goal of data analysis is twofold: The
released information must provide accurate and useful statistics about the
underlying population of interest, while minimizing the potential for an
individual record to be identified. In recent years, the notion of differential
privacy has received much attention in theoretical computer science, machine
learning, and statistics. It provides a rigorous and strong notion of
protection for individuals' sensitive information. A fundamental question is
how to incorporate differential privacy into traditional statistical inference
procedures. In this paper we study model selection in multivariate linear
regression under the constraint of differential privacy. We show that model
selection procedures based on penalized least squares or likelihood can be made
differentially private by a combination of regularization and randomization,
and propose two algorithms to do so. We show that our private procedures are
consistent under essentially the same conditions as the corresponding
non-private procedures. We also find that under differential privacy, the
procedure becomes more sensitive to the tuning parameters. We illustrate and
evaluate our method using simulation studies and two real data examples
High-Dimensional Inference with the generalized Hopfield Model: Principal Component Analysis and Corrections
We consider the problem of inferring the interactions between a set of N
binary variables from the knowledge of their frequencies and pairwise
correlations. The inference framework is based on the Hopfield model, a special
case of the Ising model where the interaction matrix is defined through a set
of patterns in the variable space, and is of rank much smaller than N. We show
that Maximum Lik elihood inference is deeply related to Principal Component
Analysis when the amp litude of the pattern components, xi, is negligible
compared to N^1/2. Using techniques from statistical mechanics, we calculate
the corrections to the patterns to the first order in xi/N^1/2. We stress that
it is important to generalize the Hopfield model and include both attractive
and repulsive patterns, to correctly infer networks with sparse and strong
interactions. We present a simple geometrical criterion to decide how many
attractive and repulsive patterns should be considered as a function of the
sampling noise. We moreover discuss how many sampled configurations are
required for a good inference, as a function of the system size, N and of the
amplitude, xi. The inference approach is illustrated on synthetic and
biological data.Comment: Physical Review E: Statistical, Nonlinear, and Soft Matter Physics
(2011) to appea
Evolving Spatially Aggregated Features from Satellite Imagery for Regional Modeling
Satellite imagery and remote sensing provide explanatory variables at
relatively high resolutions for modeling geospatial phenomena, yet regional
summaries are often desirable for analysis and actionable insight. In this
paper, we propose a novel method of inducing spatial aggregations as a
component of the machine learning process, yielding regional model features
whose construction is driven by model prediction performance rather than prior
assumptions. Our results demonstrate that Genetic Programming is particularly
well suited to this type of feature construction because it can automatically
synthesize appropriate aggregations, as well as better incorporate them into
predictive models compared to other regression methods we tested. In our
experiments we consider a specific problem instance and real-world dataset
relevant to predicting snow properties in high-mountain Asia
Necessary and sufficient conditions of solution uniqueness in minimization
This paper shows that the solutions to various convex minimization
problems are \emph{unique} if and only if a common set of conditions are
satisfied. This result applies broadly to the basis pursuit model, basis
pursuit denoising model, Lasso model, as well as other models that
either minimize or impose the constraint , where
is a strictly convex function. For these models, this paper proves that,
given a solution and defining I=\supp(x^*) and s=\sign(x^*_I),
is the unique solution if and only if has full column rank and there
exists such that and for . This
condition is previously known to be sufficient for the basis pursuit model to
have a unique solution supported on . Indeed, it is also necessary, and
applies to a variety of other models. The paper also discusses ways to
recognize unique solutions and verify the uniqueness conditions numerically.Comment: 6 pages; revised version; submitte
- …