Deep Reinforcement Learning for Approximate Policy Iteration: Convergence Analysis and a Post-Earthquake Disaster Response Case Study

Approximate Policy Iteration (API) is a Class of Reinforcement Learning (RL) Algorithms that Seek to Solve the Long-Run Discounted Reward Markov Decision Process (MDP), Via the Policy Iteration Paradigm, Without Learning the Transition Model in the Underlying Bellman Equation. Unfortunately, These Algorithms Suffer from a Defect Known as Chattering in Which the Solution (Policy) Delivered in Each Iteration of the Algorithm Oscillates between Improved and Worsened Policies, Leading to Sub-Optimal Behavior. Two Causes for This that Have Been Traced to the Crucial Policy Improvement Step Are: (I) the Inaccuracies in the Policy Improvement Function and (Ii) the Exploration/exploitation Tradeoff Integral to This Step, Which Generates Variability in Performance. Both of These Defects Are Amplified by Simulation Noise. Deep RL Belongs to a Newer Class of Algorithms in Which the Resolution of the Learning Process is Refined Via Mechanisms Such as Experience Replay And/or Deep Neural Networks for Improved Performance. in This Paper, a New Deep Learning Approach is Developed for API Which Employs a More Accurate Policy Improvement Function, Via an Enhanced Resolution Bellman Equation, Thereby Reducing Chattering and Eliminating the Need for Exploration in the Policy Improvement Step. Versions of the New Algorithm for Both the Long-Run Discounted MDP and Semi-MDP Are Presented. Convergence Properties of the New Algorithm Are Studied Mathematically, and a Post-Earthquake Disaster Response Case Study is Employed to Demonstrate Numerically the Algorithm\u27s Efficacy

A Spike-shaped Anchorage For Steel Reinforced Polymer (SRP)-strengthened Concrete Structures

Steel reinforced polymer (SRP) composite has recently emerged as an effective and economical solution for strengthening of reinforced concrete (RC) structures. Premature debonding failure of unanchored SRP at low load levels generally governs the performance of RC structures strengthened with externally bonded SRP. Therefore, a novel yet simple spike-shaped anchorage system was proposed in this study to prevent the debonding failure of SRP and to improve the interfacial shear capacity. Experimental investigation through single-lap shear tests of SRP-concrete joints showed that the anchorage system changed the failure mode from composite debonding to fiber rupture. In addition, the anchorage system substantially increased the peak load and reduced the interfacial slippage of the SRP-concrete joint compared to the unanchored condition. A numerical procedure based on the finite difference method was developed to predict the full-range load response, and results matched well with the full-range experimental responses of anchored and unanchored specimens. Parametric study of the test results and numerical simulation based on finite difference method both showed that the fiber rupture failure mode could be achieved for anchors in various positions along the bonded length. The closer the anchor is to the loaded end, the less global slip was obtained when the load reached the peak value

The rural almshouse population in Missouri

Cooperative rural research."June, 1938."This is a study of the rural almshouse in Missouri. The various types of almshouse administration and methods of inmate care are defined and described, together with the social characteristics of the inmate population. As a matter of considerable contemporary interest an analysis was made of the eligibility of inmates for old age assistance in order to determine the extent to which counties might be relieved of their inmate population through this type of public assistance. Examination of data on nearly 3,000 inmates indicated that more than one-half (52.4 per cent) were under 70 years of age which is the present minimum age limit for old age assistance. It was further determined that at least one-half of the ones eligible due to age were not likely to be approved for old age assistance since they were either physically or mentally disabled. It was concluded that less than one-fourth of the total inmate population might qualify for old age assistance and that additional provisions would be necessary if the rural counties are to be relieved of the care of their almshouse population.By C.T. Pihlblad, Arthur W. Nebel, and Joseph H. Stokes, in collaboration with Melvin W. Sneed and Cecil L. Gregory.Cooperative Rural Research ... The Agricultural Experiment Station, University of Missouri; The Rural Section, Division of Social Research, Federal Works Progress Administration; and the State Social Security Commission of Missouri Cooperating.Introduction -- Administration and control -- Personnel and management -- Almshouses and almshouse inmates -- Eligibility of inmates for old age assistance -- Consolidation of almshouses -- Suggested recommendations -- Appendix

Stereotactic radiosurgery for brain metastases: analysis of outcome and risk of brain radionecrosis

<p>Abstract</p> <p>Purpose</p> <p>to investigate the factors affecting survival and toxicity in patients treated with stereotactic radiosurgery (SRS), with special attention to volumes of brain receiving a specific dose (V10 - V16 Gy) as predictors for brain radionecrosis.</p> <p>Patients and Methods</p> <p>Two hundred six consecutive patients with 310 cerebral metastases less than 3.5 cm were treated with SRS as primary treatment and followed prospectively at University of Rome La Sapienza Sant'Andrea Hospital. Overall survival, brain control, and local control were estimated using the Kaplan-Meier method calculated from the time of SRS. Univariate and multivariate analysis using a Cox proportional hazards regression model were performed to determine the predictive value of prognostic factors for treatment outcome and SRS-related complications.</p> <p>Results</p> <p>Median overall survival and brain control were 14.1 months and 10 months, respectively. The 1-year and 2-year survival rates were 58% and 24%, and respective brain control were 43% and 22%. Sixteen patients recurred locally after SRS, with 1-year and 2-year local control rates of 92% and 84%, respectively. On multivariate analysis, stable extracranial disease and KPS >70 were associated with the most significant survival benefit. Neurological complications were recorded in 27 (13%) patients. Severe neurological complications (RTOG Grade 3 and 4) occurred in 5.8% of patients. Brain radionecrosis occurred in 24% of treated lesions, being symptomatic in 10% and asymptomatic in 14%. On multivariate analysis, V10 through V16 Gy were independent risk factors for radionecrosis, with V10 Gy and V12 Gy being the most predictive (p = 0.0001). For V10 Gy >12.6 cm³and V12 Gy >10.9 cm³the risk of radionecrosis was 47%.</p> <p>Conclusions</p> <p>SRS alone represents a feasible option as initial treatment for patients with brain metastases, however a significant subset of patients may develop neurological complications. Lesions with V12 Gy >8.5 cm³carries a risk of radionecrosis >10% and should be considered for hypofractionated stereotactic radiotherapy especially when located in/near eloquent areas.</p

The role of whole brain radiation therapy in the management of newly diagnosed brain metastases: a systematic review and evidence-based clinical practice guideline

QUESTION: Should patients with newly-diagnosed metastatic brain tumors undergo open surgical resection versus whole brain radiation therapy (WBRT) and/or other treatment modalities such as radiosurgery, and in what clinical settings? TARGET POPULATION: These recommendations apply to adults with a newly diagnosed single brain metastasis amenable to surgical resection. RECOMMENDATIONS: Surgical resection plus WBRT versus surgical resection alone Level 1 Surgical resection followed by WBRT represents a superior treatment modality, in terms of improving tumor control at the original site of the metastasis and in the brain overall, when compared to surgical resection alone. Surgical resection plus WBRT versus SRS + or - WBRT Level 2 Surgical resection plus WBRT, versus stereotactic radiosurgery (SRS) plus WBRT, both represent effective treatment strategies, resulting in relatively equal survival rates. SRS has not been assessed from an evidence-based standpoint for larger lesions (\u3e3 cm) or for those causing significant mass effect (\u3e1 cm midline shift). Level 3 Underpowered class I evidence along with the preponderance of conflicting class II evidence suggests that SRS alone may provide equivalent functional and survival outcomes compared with resection + WBRT for patients with single brain metastases, so long as ready detection of distant site failure and salvage SRS are possible. Note The following question is fully addressed in the WBRT guideline paper within this series by Gaspar et al. Given that the recommendation resulting from the systematic review of the literature on this topic is also highly relevant to the discussion of the role of surgical resection in the management of brain metastases, this recommendation has been included below

The Ross Sea Dipole-temperature, snow accumulation and sea ice variability in the Ross Sea region, Antarctica, over the past 2700 years

High-resolution, well-dated climate archives provide an opportunity to investigate the dynamic interactions of climate patterns relevant for future projections. Here, we present data from a new, annually dated ice core record from the eastern Ross Sea, named the Roosevelt Island Climate Evolution (RICE) ice core. Comparison of this record with climate reanalysis data for the 1979-2012 interval shows that RICE reliably captures temperature and snow precipitation variability in the region. Trends over the past 2700 years in RICE are shown to be distinct from those in West Antarctica and the western Ross Sea captured by other ice cores. For most of this interval, the eastern Ross Sea was warming (or showing isotopic enrichment for other reasons), with increased snow accumulation and perhaps decreased sea ice concentration. However, West Antarctica cooled and the western Ross Sea showed no significant isotope temperature trend. This pattern here is referred to as the Ross Sea Dipole. Notably, during the Little Ice Age, West Antarctica and the western Ross Sea experienced colder than average temperatures, while the eastern Ross Sea underwent a period of warming or increased isotopic enrichment. From the 17th century onwards, this dipole relationship changed. All three regions show current warming, with snow accumulation declining in West Antarctica and the eastern Ross Sea but increasing in the western Ross Sea. We interpret this pattern as reflecting an increase in sea ice in the eastern Ross Sea with perhaps the establishment of a modern Roosevelt Island polynya as a local moisture source for RICE

Iodine-125 brachytherapy for brain tumours - a review

Iodine-125 brachytherapy has been applied to brain tumours since 1979. Even though the physical and biological characteristics make these implants particularly attractive for minimal invasive treatment, the place for stereotactic brachytherapy is still poorly defined

Snow Chemistry Across Antarctica

An updated compilation of published and new data of major-ion (Ca, Cl, K, Mg, Na, NO3, SO4) and methylsulfonate (MS) concentrations in snow from 520 Antarctic sites is provided by the national ITASE (International Trans-Antarctic Scientific Expedition) programmes of Australia, Brazil, China, Germany, Italy, Japan, Korea, New Zealand, Norway, the United Kingdom, the United States and the national Antarctic programme of Finland. The comparison shows that snow chemistry concentrations vary by up to four orders of magnitude across Antarctica and exhibit distinct geographical patterns. The Antarctic-wide comparison of glaciochemical records provides a unique opportunity to improve our understanding of the fundamental factors that ultimately control the chemistry of snow or ice samples. This paper aims to initiate data compilation and administration in order to provide a framework for facilitation of Antarctic-wide snow chemistry discussions across all ITASE nations and other contributing groups. The data are made available through the ITASE web page (http:// www2.umaine.edu/itase/content/syngroups/snowchem.html) and will be updated with new data as they are provided. In addition, recommendations for future research efforts are summarized