Dynamically optimal treatment allocation using Reinforcement Learning

Devising guidance on how to assign individuals to treatment is an important goal in empirical research. In practice, individuals often arrive sequentially, and the planner faces various constraints such as limited budget/capacity, or borrowing constraints, or the need to place people in a queue. For instance, a governmental body may receive a budget outlay at the beginning of a year, and it may need to decide how best to allocate resources within the year to individuals who arrive sequentially. In this and other examples involving inter-temporal trade-offs, previous work on devising optimal policy rules in a static context is either not applicable, or sub-optimal. Here we show how one can use offline observational data to estimate an optimal policy rule that maximizes expected welfare in this dynamic context. We allow the class of policy rules to be restricted for legal, ethical or incentive compatibility reasons. The problem is equivalent to one of optimal control under a constrained policy class, and we exploit recent developments in Reinforcement Learning (RL) to propose an algorithm to solve this. The algorithm is easily implementable with speedups achieved through multiple RL agents learning in parallel processes. We also characterize the statistical regret from using our estimated policy rule by casting the evolution of the value function under each policy in a Partial Differential Equation (PDE) form and using the theory of viscosity solutions to PDEs. We find that the policy regret decays at a $n^{-1/2}$ rate in most examples; this is the same rate as in the static case.Comment: 67 page

Potential of the system of rice intensification for systemic improvement in rice production and water use: the case of Andhra Pradesh, India

As opportunities to enhance the irrigation base for raising food production in the country are dwindling, India needs a more concerted effort to increase the efficiency and productivity of its irrigation systems. This study, based on an analysis of experience from the state of Andhra Pradesh, addresses the potential of the System of Rice Intensification (SRI) to contribute to systemic corrections in present paddy cultivation, both with regard to agronomic productivity and irrigation water use efficiency. This study points to the considerable increase in rice productivity and farmer incomes, which is being achieved in Andhra Pradesh with substantial reduction in irrigation water application, labor, and seed costs through utilization of SRI methods. Potential public savings on water and power costs could be drawn upon not only for promoting SRI but also to effect systemic corrections in the irrigation sector, to mutual advantage

Response of Pacific-sector Antarctic ice shelves to the El Niño/Southern Oscillation.

Satellite observations over the past two decades have revealed increasing loss of grounded ice in West Antarctica, associated with floating ice shelves that have been thinning. Thinning reduces an ice-shelf's ability to restrain grounded-ice discharge, yet our understanding of the climate processes that drive mass changes is limited. Here, we use ice-shelf height data from four satellite altimeter missions (1994-2017) to show a direct link between ice-shelf-height variability in the Antarctic Pacific sector and changes in regional atmospheric circulation driven by the El Niño-Southern Oscillation. This link is strongest from Dotson to Ross ice shelves and weaker elsewhere. During intense El Niño years, height increase by accumulation exceeds the height decrease by basal melting, but net ice-shelf mass declines as basal ice loss exceeds lower-density snow gain. Our results demonstrate a substantial response of Amundsen Sea ice shelves to global and regional climate variability, with rates of change in height and mass on interannual timescales that can be comparable to the longer-term trend, and with mass changes from surface accumulation offsetting a significant fraction of the changes in basal melting. This implies that ice-shelf height and mass variability will increase as interannual atmospheric variability increases in a warming climate

Nonparametric instrumental regression with errors in variables

This paper considers nonparametric instrumental variable regression when the endogenous variable is contaminated with classical measurement error. Existing methods are inconsistent in the presence of measurement error. We propose a wavelet deconvolution estimator for the structural function that modifies the generalized Fourier coefficients of the orthogonal series estimator to take into account the measurement error. We establish the convergence rates of our estimator for the cases of mildly/severely ill-posed models and ordinary/super smooth measurement errors. We characterize how the presence of measurement error slows down the convergence rates of the estimator. We also study the case where the measurement error density is unknown and needs to be estimated, and show that the estimation error of the measurement error density is negligible under mild conditions as far as the measurement error density is symmetric

Energy Cost Comparisons of Centre Pivot Systems in the Northern Texas High Plains

A study was done to compare energy costs of high, medium, and low pressure sprinkler systems in the North Texas High Plains and to estimate the amount of savings in dollar terms by adopting the change

The role of load in initiation and progression of cartilage pathology

The entire dissertation/thesis text is included in the research.pdf file; the official abstract appears in the short.pdf file (which also appears in the research.pdf); a non-technical general description, or public abstract, appears in the public.pdf file.Vita."December 2007"Thesis (Ph. D.) University of Missouri-Columbia 2007.Articular cartilage is primarily responsible for dissipation of load in diarthrodial joints. Load plays a critical role in maintaining cartilage health, but can also be a primary contributing factor in cartilage disease such as osteoarthritis. IL-1[Beta] is a cytokine involved in the initiation and progression of osteoarthritis through inciting cascades which cause inflammation and degradation. Coticosteroids are used extensively in the equine industry to treat the symptoms of osteoarthritis. They have beneficial effects in lameness and inflammation, but can exacerbate cartilage degradation and hinder tissue healing. It is important to understand the roles and interactions of load, IL-1[Beta], and corticosteroids in regards to cartilage health and disease. To understand the effects of corticosteroids and IL-1[Beta] on articular cartilage in vivo relevant gene expression, extracellular matrix composition, and biomarker production of cartilage were measured after subjecting to various combinations of load, corticosteroids, and IL-1[Beta]. The results from this study have given us insight into the effects of various loads on articular cartilage. Higher frequencies and durations of compressive loading seemed to have more pronounced deleterious effects on cartilage, even within physiological loading ranges. In combination with corticosteroids, compressive loads at 2 and 6 MPa delivered at high frequencies resulted in changes similar to those reported in corticosteroid-induced arthropathy. However similar compressive load delivered at lower frequencies at 2 and 6 MPa seemed to mitigate some of the deleterious effects of IL-1[Beta] as evidenced by decreased expression of matrix metalloproteinases when compared to unloaded samples and samples loaded at higher frequencies.Includes bibliographical reference

Economic Value of Groundwater Resources and Irrigated Agriculture in the Oklahoma Panhandle

An economic optimization model was developed using available groundwater resources in the Oklahoma Panhandle to estimate value of water for irrigated agriculture in the area. The model will serve as policy tool to analyze alternative water management strategies and conservation programs to assess the economic impact of depleting Ogallala Aquifer.Ogallala Aquifer, Irrigated Agriculture, Groundwater Conservation, Water Management Policy, Oklahoma Panhandle, Resource /Energy Economics and Policy,

Estimation of Irrigation Water Demand: A Case Study for the Texas High Plains

On the Texas High Plains, water from the Ogallala aquifer is used to supplement irrigation requirements, since annual rainfall is below 20.5 in/ year. This study addresses land and water demand by testing the effects of water costs, crop prices, and technology on water use and crop production acreages.Ogallala, Irrigation, Water Demand, Technology, Agricultural and Food Policy, Production Economics,

Multi-lectin Affinity Chromatography and Quantitative Proteomic Analysis Reveal Differential Glycoform Levels between Prostate Cancer and Benign Prostatic Hyperplasia Sera.

Currently prostate-specific antigen is used for prostate cancer (PCa) screening, however it lacks the necessary specificity for differentiating PCa from other diseases of the prostate such as benign prostatic hyperplasia (BPH), presenting a clinical need to distinguish these cases at the molecular level. Protein glycosylation plays an important role in a number of cellular processes involved in neoplastic progression and is aberrant in PCa. In this study, we systematically interrogate the alterations in the circulating levels of hundreds of serum proteins and their glycoforms in PCa and BPH samples using multi-lectin affinity chromatography and quantitative mass spectrometry-based proteomics. Specific lectins (AAL, PHA-L and PHA-E) were used to target and chromatographically separate core-fucosylated and highly-branched protein glycoforms for analysis, as differential expression of these glycan types have been previously associated with PCa. Global levels of CD5L, CFP, C8A, BST1, and C7 were significantly increased in the PCa samples. Notable glycoform-specific alterations between BPH and PCa were identified among proteins CD163, C4A, and ATRN in the PHA-L/E fraction and among C4BPB and AZGP1 glycoforms in the AAL fraction. Despite these modest differences, substantial similarities in glycoproteomic profiles were observed between PCa and BPH sera

Empirical likelihood for random sets

We extend the method of empirical likelihood to cover hypotheses involving the Aumann expectation of random sets. By exploiting the properties of random sets, we convert the testing problem into one involving a continuum of moment restrictions for which we propose two inferential procedures. The first, which we term marked empirical likelihood, corresponds to constructing a non-parametric likelihood for each moment restriction and assessing the resulting process. The second, termed sieve empirical likelihood, corresponds to constructing a likelihood for a vector of moments with growing dimension. We derive the asymptotic distributions under the null and sequence of local alternatives for both types of tests and prove their consistency. The applicability of these inferential procedures is demonstrated in the context of two examples on the mean of interval observations and best linear predictors for interval outcomes