A general procedure to combine estimators

A general method to combine several estimators of the same quantity is investigated. In the spirit of model and forecast averaging, the final estimator is computed as a weighted average of the initial ones, where the weights are constrained to sum to one. In this framework, the optimal weights, minimizing the quadratic loss, are entirely determined by the mean square error matrix of the vector of initial estimators. The averaging estimator is built using an estimation of this matrix, which can be computed from the same dataset. A non-asymptotic error bound on the averaging estimator is derived, leading to asymptotic optimality under mild conditions on the estimated mean square error matrix. This method is illustrated on standard statistical problems in parametric and semi-parametric models where the averaging estimator outperforms the initial estimators in most cases

Contributions To The Estimation Of The Logit, Log Odds And Common Odds Ratio

Asymptotic series expansions for the bias and mean square error of the logit estimator are developed. In addition, an estimator of the bias and asymptotic series for the expectation of this estimator and the estimator of the mean square error are derived. These formulations have been expressed in such a way that general coefficients can be calculated and then implemented for any choice of the parameters involved in the logit estimation problem. The formulas developed are applied, providing similar asymptotic series, for the estimation of the log odds ratio.;A general method for the linear combination of estimators is proposed. This procedure is applied to combine log odds ratios and new estimators are developed for the estimation of a common odds ratio. The small sample properties of these estimators and some widely used estimators are examined and compared in a Monte Carlo simulation. Since no one estimator is uniformly the best, a scheme for applying the most appropriate estimator in a given situation is proposed. Illustrations of settings in which this approach is feasible are examined

Model Reduction of Parametric Differential-Algebraic Systems by Balanced Truncation

In this article, we deduce a procedure to apply balanced truncation to parameter-dependent differential-algebraic systems. For that we solve multiple projected Lyapunov equations for different parameter values to compute the Gramians that are required for the truncation procedure. As this process would lead to high computational costs if we perform it for a large number of parameters. Hence, we combine this approach with the reduced basis method that determines a reduced representation of the Lyapunov equation solutions for the parameters of interest. Residual-based error estimators are then used to evaluate the quality of the approximations. After introducing the procedure for a general class of differential-algebraic systems we turn our focus to systems with a certain structure, for which the method can be applied particularly efficiently. We illustrate the effectiveness of our approach on several models from fluid dynamics and mechanics. We further consider an application of the method in the context of damping optimization

Semiparametric Shape-restricted Estimators for Nonparametric Regression

Estimating the conditional mean function that relates predictive covariates to a response variable of interest is a fundamental task in economics and statistics. In this manuscript, we propose some general nonparametric regression approaches that are widely applicable based on a simple yet significant decomposition of nonparametric functions into a semiparametric model with shape-restricted components. For instance, we observe that every Lipschitz function can be expressed as a sum of a monotone function and a linear function. We implement well-established shape-restricted estimation procedures, such as isotonic regression, to handle the ``nonparametric" components of the true regression function and combine them with a simple sample-splitting procedure to estimate the parametric components. The resulting estimators inherit several favorable properties from the shape-restricted regression estimators. Notably, it is practically tuning parameter free, converges at the minimax optimal rate, and exhibits an adaptive rate when the true regression function is ``simple". We also confirm these theoretical properties and compare the practice performance with existing methods via a series of numerical studies

Online Bootstrap Inference with Nonconvex Stochastic Gradient Descent Estimator

In this paper, we investigate the theoretical properties of stochastic gradient descent (SGD) for statistical inference in the context of nonconvex optimization problems, which have been relatively unexplored compared to convex settings. Our study is the first to establish provable inferential procedures using the SGD estimator for general nonconvex objective functions, which may contain multiple local minima. We propose two novel online inferential procedures that combine SGD and the multiplier bootstrap technique. The first procedure employs a consistent covariance matrix estimator, and we establish its error convergence rate. The second procedure approximates the limit distribution using bootstrap SGD estimators, yielding asymptotically valid bootstrap confidence intervals. We validate the effectiveness of both approaches through numerical experiments. Furthermore, our analysis yields an intermediate result: the in-expectation error convergence rate for the original SGD estimator in nonconvex settings, which is comparable to existing results for convex problems. We believe this novel finding holds independent interest and enriches the literature on optimization and statistical inference

Implementing Loss Distribution Approach for Operational Risk

To quantify the operational risk capital charge under the current regulatory framework for banking supervision, referred to as Basel II, many banks adopt the Loss Distribution Approach. There are many modeling issues that should be resolved to use the approach in practice. In this paper we review the quantitative methods suggested in literature for implementation of the approach. In particular, the use of the Bayesian inference method that allows to take expert judgement and parameter uncertainty into account, modeling dependence and inclusion of insurance are discussed