135,728 research outputs found
Trimmed Density Ratio Estimation
Density ratio estimation is a vital tool in both machine learning and
statistical community. However, due to the unbounded nature of density ratio,
the estimation procedure can be vulnerable to corrupted data points, which
often pushes the estimated ratio toward infinity. In this paper, we present a
robust estimator which automatically identifies and trims outliers. The
proposed estimator has a convex formulation, and the global optimum can be
obtained via subgradient descent. We analyze the parameter estimation error of
this estimator under high-dimensional settings. Experiments are conducted to
verify the effectiveness of the estimator.Comment: Made minor revisions. Restructured the introductory section
Robust Covariate Shift Adaptation for Density-Ratio Estimation
Consider a scenario where we have access to train data with both covariates
and outcomes while test data only contains covariates. In this scenario, our
primary aim is to predict the missing outcomes of the test data. With this
objective in mind, we train parametric regression models under a covariate
shift, where covariate distributions are different between the train and test
data. For this problem, existing studies have proposed covariate shift
adaptation via importance weighting using the density ratio. This approach
averages the train data losses, each weighted by an estimated ratio of the
covariate densities between the train and test data, to approximate the
test-data risk. Although it allows us to obtain a test-data risk minimizer, its
performance heavily relies on the accuracy of the density ratio estimation.
Moreover, even if the density ratio can be consistently estimated, the
estimation errors of the density ratio also yield bias in the estimators of the
regression model's parameters of interest. To mitigate these challenges, we
introduce a doubly robust estimator for covariate shift adaptation via
importance weighting, which incorporates an additional estimator for the
regression function. Leveraging double machine learning techniques, our
estimator reduces the bias arising from the density ratio estimation errors. We
demonstrate the asymptotic distribution of the regression parameter estimator.
Notably, our estimator remains consistent if either the density ratio estimator
or the regression function is consistent, showcasing its robustness against
potential errors in density ratio estimation. Finally, we confirm the soundness
of our proposed method via simulation studies
New Machine Learning Techniques for Simulation-Based Inference: InferoStatic Nets, Kernel Score Estimation, and Kernel Likelihood Ratio Estimation
We propose an intuitive, machine-learning approach to multiparameter
inference, dubbed the InferoStatic Networks (ISN) method, to model the score
and likelihood ratio estimators in cases when the probability density can be
sampled but not computed directly. The ISN uses a backend neural network that
models a scalar function called the inferostatic potential . In
addition, we introduce new strategies, respectively called Kernel Score
Estimation (KSE) and Kernel Likelihood Ratio Estimation (KLRE), to learn the
score and the likelihood ratio functions from simulated data. We illustrate the
new techniques with some toy examples and compare to existing approaches in the
literature. We mention en passant some new loss functions that optimally
incorporate latent information from simulations into the training procedure.Comment: 35 pages, 10 figures. Submission to SciPos
- …