Jeffreys-prior penalty, finiteness and shrinkage in binomial-response generalized linear models

Penalization of the likelihood by Jeffreys' invariant prior, or by a positive power thereof, is shown to produce finite-valued maximum penalized likelihood estimates in a broad class of binomial generalized linear models. The class of models includes logistic regression, where the Jeffreys-prior penalty is known additionally to reduce the asymptotic bias of the maximum likelihood estimator; and also models with other commonly used link functions such as probit and log-log. Shrinkage towards equiprobability across observations, relative to the maximum likelihood estimator, is established theoretically and is studied through illustrative examples. Some implications of finiteness and shrinkage for inference are discussed, particularly when inference is based on Wald-type procedures. A widely applicable procedure is developed for computation of maximum penalized likelihood estimates, by using repeated maximum likelihood fits with iteratively adjusted binomial responses and totals. These theoretical results and methods underpin the increasingly widespread use of reduced-bias and similarly penalized binomial regression models in many applied fields

Constrained Statistical Inference: A Hybrid of Statistical Theory, Projective Geometry and Applied Optimization Techniques

In many data applications, in addition to determining whether a given risk factor affects an outcome, researchers are often interested in whether the factor has an increasing or decreasing effect. For instance, a clinical trial may test which dose provides the minimum effect; a toxicology study may wish to determine the effect of increasing exposure to a harmful contaminant on human health; and an economist may wish to determine an individual's optimal preferences subject to a budget constraint. In such situations, constrained statistical inference is typically used for analysis, as estimation and hypothesis testing incorporate the parameter orderings, or restrictions, in the methodology. Such methods unite statistical theory with elements of projective geometry and optimization algorithms. In many different models, authors have demonstrated constrained techniques lead to more efficient estimates and improved power over unconstrained methods, albeit at the expense of additional computation. In this paper, we review significant advancements made in the field of constrained inference, ranging from early work on isotonic regression for several normal means to recent advances of constraints in Bayesian techniques and mixed models. To illustrate the methods, a new analysis of an environmental study on the health effects in a population of newborns is provided

Markov switching models: an application to roadway safety

In this research, two-state Markov switching models are proposed to study accident frequencies and severities. These models assume that there are two unobserved states of roadway safety, and that roadway entities (e.g., roadway segments) can switch between these states over time. The states are distinct, in the sense that in the different states accident frequencies or severities are generated by separate processes (e.g., Poisson, negative binomial, multinomial logit). Bayesian inference methods and Markov Chain Monte Carlo (MCMC) simulations are used for estimation of Markov switching models. To demonstrate the applicability of the approach, we conduct the following three studies. In the first study, two-state Markov switching count data models are considered as an alternative to zero-inflated models for annual accident frequencies, in order to account for preponderance of zeros typically observed in accident frequency data. In the second study, two-state Markov switching Poisson model and two-state Markov switching negative binomial model are estimated using weekly accident frequencies on selected Indiana interstate highway segments over a five-year time period. In the third study, two-state Markov switching multinomial logit models are estimated for severity outcomes of accidents occurring on Indiana roads over a four-year time period. One of the most important results found in each of the three studies, is that in each case the estimated Markov switching models are strongly favored by roadway safety data and result in a superior statistical fit, as compared to the corresponding standard (non-switching) models.Comment: PhD dissertation (Purdue University), 122 pages, 7 figures, 19 table

Master of Science

thesisThis thesis is conducted to compare a crash-level severity model with an occupant-level severity model for single-vehicle crashes on rural, two-lane roads. A multinomial logit model is used to identify and quantify the main contributing factors to the severity of rural, two-lane highway, single-vehicle crashes including human, roadway, and environmental factors. A comprehensive analysis of 5 years of crashes on rural, two-lane highways in Illinois with roadway characteristics, vehicle information, and human factors will be provided. The modeling results show that lower crash severities are associated with wider lane widths, shoulder widths, and edge line widths, and larger traffic volumes, alcohol-impaired driving, no restraint use will increase crash severity significantly. It is also shown that the impacts of light condition and weather condition are counterintuitive but the results are consistent with some previous research. Goodness of fit test and IIA (independence of irrelevant alternatives) test are applied to examine the appropriateness of the multinomial logit model and to compare the fit of the crash-level model with the occupant-level model. It is found that there are consistent modeling results between the two models and the prediction of each severity level by crash-level model is more accurate than that of the occupant-level model

Equi-energy sampler with applications in statistical inference and statistical mechanics

We introduce a new sampling algorithm, the equi-energy sampler, for efficient statistical sampling and estimation. Complementary to the widely used temperature-domain methods, the equi-energy sampler, utilizing the temperature--energy duality, targets the energy directly. The focus on the energy function not only facilitates efficient sampling, but also provides a powerful means for statistical estimation, for example, the calculation of the density of states and microcanonical averages in statistical mechanics. The equi-energy sampler is applied to a variety of problems, including exponential regression in statistics, motif sampling in computational biology and protein folding in biophysics.Comment: This paper discussed in: [math.ST/0611217], [math.ST/0611219], [math.ST/0611221], [math.ST/0611222]. Rejoinder in [math.ST/0611224]. Published at http://dx.doi.org/10.1214/009053606000000515 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Approximate Profile Maximum Likelihood

We propose an efficient algorithm for approximate computation of the profile maximum likelihood (PML), a variant of maximum likelihood maximizing the probability of observing a sufficient statistic rather than the empirical sample. The PML has appealing theoretical properties, but is difficult to compute exactly. Inspired by observations gleaned from exactly solvable cases, we look for an approximate PML solution, which, intuitively, clumps comparably frequent symbols into one symbol. This amounts to lower-bounding a certain matrix permanent by summing over a subgroup of the symmetric group rather than the whole group during the computation. We extensively experiment with the approximate solution, and find the empirical performance of our approach is competitive and sometimes significantly better than state-of-the-art performance for various estimation problems