Zap Q-Learning for Optimal Stopping Time Problems

The objective in this paper is to obtain fast converging reinforcement learning algorithms to approximate solutions to the problem of discounted cost optimal stopping in an irreducible, uniformly ergodic Markov chain, evolving on a compact subset of $\mathbb{R}^n$. We build on the dynamic programming approach taken by Tsitsikilis and Van Roy, wherein they propose a Q-learning algorithm to estimate the optimal state-action value function, which then defines an optimal stopping rule. We provide insights as to why the convergence rate of this algorithm can be slow, and propose a fast-converging alternative, the "Zap-Q-learning" algorithm, designed to achieve optimal rate of convergence. For the first time, we prove the convergence of the Zap-Q-learning algorithm under the assumption of linear function approximation setting. We use ODE analysis for the proof, and the optimal asymptotic variance property of the algorithm is reflected via fast convergence in a finance example

A study of serum lipid profile in normal pregnancy and pregnancy induced hypertensive disorders: a case-control study

Background: Pregnancy induced hypertensive disorders are one of the commonest complication of pregnancy which accounts for 12% of the maternal and perinatal mortality and morbidity. Dyslipidemias are associated with endothelial dysfunction that may result in proteinuria and hypertension which is a clinical hallmark of PIH. It affects both maternal health as well as fetal growth. Hence, this study was done to assess the role of altered lipid profile in the development of PIH.Methods: A Case Control study was conducted at the Department of Biochemistry, Kurnool Medical College and Govt General Hospital, Kurnool in collaboration with its Obstetrics Dept during the period of November 2015-2017. A total of 300 pregnant women, primigravida /multigravida with singleton pregnancy, in the age group of 18‐ 35 years with >20 weeks of gestation were included in the study. Subjects were divided into gestational hypertensives, n=39 (BP ≥140/80) and preeclamptic women, n=111 (≥140/80 and proteinuria) as cases. Age matched normotensive pregnant women, n=150 (BP 120/80) were recruited as Controls. Subjects with history of multiple pregnancies, pregnancy with congenital anomalies, chronic hypertension, diabetes mellitus, cardiac/thyroid/hepatic/renal disease, dyslipidemia were excluded. Total cholesterol, TG, HDL, LDL, VLDL were performed.Results: A comparison of these values between hypertensive and normotensive women showed a significant rise in TC, TG, LDL and VLDL. HDL-C showed a significant decrease in hypertensive women compared to normal pregnant women. LDL: HDL and TG:HDL ratios were higher in PIH group.Conclusions: The results of this study suggests an abnormal lipid metabolism, predominantly high TG concentrations and low HDL-C, which may add to the promotion of vascular dysfunction and oxidative stress seen in PIH. This association is significant in understanding the development of hypertension during pregnancy and is useful in early diagnosis and prevention of PIH

Patterns in the spectral composition of sunlight and biologically meaningful spectral photon ratios as affected by atmospheric factors

Plants rely on spectral cues present in their surroundings, generated by the constantly changing light environment, to guide their growth and reproduction. Photoreceptors mediate the capture of information by plants from the light environment over a wide range of wavelengths, but despite extensive evidence that plants respond to various light cues, only fragmentary data have been published showing patterns of diurnal, seasonal and geographical variation in the spectral composition of daylight. To illustrate patterns in spectral photon ratios, we measured time series of irradiance spectra at two distinct geographical and climatological locations, Helsinki, Finland and Gual Pahari, India. We investigated the drivers behind variation of the spectral photon ratios measured at these two locations, based on the analysis of over 400 000 recorded spectra. Differences in spectral irradiance were explained by different atmospheric factors identified through multiple regression model analysis and comparison to spectral irradiance at ground level simulated with a radiative transfer model. Local seasonal and diurnal changes in spectral photon ratios were related to solar elevation angle, atmospheric water-vapour content and total ozone column thickness and deviated from their long-term averages to an extent likely to affect plant photobiology. We suggest that future studies should investigate possible effects of varying photon ratios on terrestrial plants. Solar elevation angle especially affects the patterns of B:G and B:R ratios. Water vapour has a large effect on the R:FR photon ratio and modelled climate scenarios predict that increasing global temperatures will result in increased atmospheric water vapour. The development of proxy models, utilising available data from weather and climate models, for relevant photon ratios as a function of solar elevation angle and atmospheric factors would facilitate the interpretation of results from past, present and future field studies of plants and vegetation.Peer reviewe

対流圏に見られる鉛直微細構造

We report the design, synthesis, detailed characterization, and analysis of a new multifunctional pi-conjugated bola-amphiphilic chromophore: oligo-(p-phenyleneethynylene)-dicarboxylic acid with dialkoxyoctadecyl side chains (OPE-C-18-1). OPE-C-18-1 shows two polymorphs at 123 K (OPE-C-18-1') and 373 K (OPE-C-18-1 `'), whose crystal structures were characterized via single crystal X-ray diffraction. OPE-C-18-1 also exhibits thermotropic liquid crystalline property revealing a columnar phase. The inherent pi-conjugation of OPE-C-18-1 imparts luminescence to the system. Photoluminescence measurements on the mesophase also reveal similar luminescence as in the crystalline state. Additionally, OPE-C-18-1 shows mechano-hypsochromic luminescence behavior. Density functional theory (DFT)-based calculations unravel the origins behind the simultaneous existence of all these properties. Nanoindentation experiments on the single crystal reveal its mechanical strength and accurately correlate the molecular arrangement with the liquid crystalline and mechanochromic luminescence behavior

Surgical Explantation of Transcatheter Aortic Valve Bioprostheses: A Statewide Experience

BACKGROUND: Despite the rapid adoption of transcatheter aortic valve replacement (TAVR) since its initial approval in 2011, the frequency and outcomes of surgical explantation of TAVR devices (TAVR-explant) is poorly understood. METHODS: Patients undergoing TAVR-explant between January 2012 and June 2020 at 33 hospitals in Michigan were identified in the Society of Thoracic Surgeons Database and linked to index TAVR data from the Transcatheter Valve Therapy Registry through a statewide quality collaborative. The primary outcome was operative mortality. Indications for TAVR-explant, contraindications to redo TAVR, operative data, and outcomes were collected from Society of Thoracic Surgeons and Transcatheter Valve Therapy databases. Baseline Society of Thoracic Surgeons Predicted Risk of Mortality was compared between index TAVR and TAVR-explant. RESULTS: Twenty-four surgeons at 12 hospitals performed TAVR-explants in 46 patients (median age, 73). The frequency of TAVR-explant was 0.4%, and the number of explants increased annually. Median time to TAVR-explant was 139 days and among known device types explanted, most were self-expanding valves (29/41, 71%). Common indications for TAVR-explant were procedure-related failure (35%), paravalvular leak (28%), and need for other cardiac surgery (26%). Contraindications to redo TAVR included need for other cardiac surgery (28%), unsuitable noncoronary anatomy (13%), coronary obstruction (11%), and endocarditis (11%). Overall, 65% (30/46) of patients underwent concomitant procedures, including aortic repair/replacement in 33% (n=15), mitral surgery in 22% (n=10), and coronary artery bypass grafting in 16% (n=7). The median Society of Thoracic Surgeons Predicted Risk of Mortality was 4.2% at index TAVR and 9.3% at TAVR-explant (P=0.001). Operative mortality was 20% (9/46) and 76% (35/46) of patients had in-hospital complications. Of patients alive at discharge, 37% (17/37) were discharged home and overall 3-month survival was 73±14%. CONCLUSIONS: TAVR-explant is rare but increasing, and its clinical impact is substantial. As the utilization of TAVR expands into younger and lower-risk patients, providers should consider the potential for future TAVR-explant during selection of an initial valve strategy

Prescription opioid use before and after kidney transplant: Implications for posttransplant outcomes

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/146648/1/AJT14714-sup-0001-AppendixS1.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/146648/2/ajt14714_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/146648/3/ajt14714.pd