Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Conventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach

Reinforced Axial Refinement Network for Monocular 3D Object Detection

Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image. This is an ill-posed problem with a major difficulty lying in the information loss by depth-agnostic cameras. Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space. To improve the efficiency of sampling, we propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step. This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it. The proposed framework, Reinforced Axial Refinement Network (RAR-Net), serves as a post-processing stage which can be freely integrated into existing monocular 3D detection methods, and improve the performance on the KITTI dataset with small extra computational costs.Comment: Accepted by ECCV 202

Direct Formation of Supermassive Black Holes via Multi-Scale Gas Inflows in Galaxy Mergers

Observations of distant bright quasars suggest that billion solar mass supermassive black holes (SMBHs) were already in place less than a billion years after the Big Bang. Models in which light black hole seeds form by the collapse of primordial metal-free stars cannot explain their rapid appearance due to inefficient gas accretion. Alternatively, these black holes may form by direct collapse of gas at the center of protogalaxies. However, this requires metal-free gas that does not cool efficiently and thus is not turned into stars, in contrast with the rapid metal enrichment of protogalaxies. Here we use a numerical simulation to show that mergers between massive protogalaxies naturally produce the required central gas accumulation with no need to suppress star formation. Merger-driven gas inflows produce an unstable, massive nuclear gas disk. Within the disk a second gas inflow accumulates more than 100 million solar masses of gas in a sub-parsec scale cloud in one hundred thousand years. The cloud undergoes gravitational collapse, which eventually leads to the formation of a massive black hole. The black hole can grow to a billion solar masses in less than a billion years by accreting gas from the surrounding disk.Comment: 26 pages, 4 Figures, submitted to Nature (includes Supplementary Information

Meiotic Transmission of Drosophila pseudoobscura Chromosomal Arrangements

Drosophila pseudoobscura harbors a rich gene arrangement polymorphism on the third chromosome generated by a series of overlapping paracentric inversions. The arrangements suppress recombination in heterokaryotypic individuals, which allows for the selective maintenance of coadapted gene complexes. Previous mapping experiments used to determine the degree to which recombination is suppressed in gene arrangement heterozygotes produced non-recombinant progeny in non-Mendelian ratios. The deviations from Mendelian expectations could be the result of viability differences between wild and mutant chromosomes, meiotic drive because of achiasmate pairing of homologues in heterokaryotypic females during meiosis, or a combination of both mechanisms. The possibility that the frequencies of the chromosomal arrangements in natural populations are affected by mechanisms other than adaptive selection led us to consider these hypotheses. We performed reciprocal crosses involving both heterozygous males and females to determine if the frequency of the non-recombinant progeny deviates significantly from Mendelian expectations and if the frequencies deviate between reciprocal crosses. We failed to observe non-Mendelian ratios in multiple crosses, and the frequency of the non-recombinant classes differed in only one of five pairs of reciprocal crosses despite sufficient power to detect these differences in all crosses. Our results indicate that deviations from Mendelian expectations in recombination experiments involving the D. pseudoobscura inversion system are most likely due to fitness differences of gene arrangement karyotypes in different environments

Sequential occurrence of thrombotic thrombocytopenic purpura, essential thrombocythemia, and idiopathic thrombocytopenic purpura in a 42-year-old African-American woman: a case report and review of the literature

<p>Abstract</p> <p>Introduction</p> <p>Thrombotic thrombocytopenic purpura and idiopathic thrombocytopenic purpura are two well recognized syndromes that are characterized by low platelet counts. In contrast, essential thrombocythemia is a myeloproliferative disease characterized by abnormally high platelet numbers.</p> <p>The coexistence of thrombotic thrombocytopenic purpura and idiopathic thrombocytopenic purpura in a single patient has been reported in the literature on a few occasions. However, having essential thrombocythemia complicating the picture has never been reported before.</p> <p>Case presentation</p> <p>We present a case where thrombotic thrombocytopenic purpura, essential thrombocythemia, and idiopathic thrombocytopenic purpura were diagnosed in a 42-year-old African-American woman in the space of a few years; we are reporting this case with the aim of drawing attention to this undocumented occurrence, which remains under investigation.</p> <p>Conclusions</p> <p>As the three conditions have different natural histories and require different treatment modalities, it is important to recognize that these diseases may be seen sequentially. This case emphasizes the importance of reviewing peripheral blood smears for evaluation of thrombocytopenia and bone marrow aspirations for diagnosis of thrombocythemia in order to reach an accurate diagnosis and tailor therapy accordingly. Moreover, this case demonstrates the variability and complexity of platelet disorders. This occurrence of three different types of platelet disorders in one patient remains a pure observation on our part; regardless, this does raise the possibility of a common underlying, as yet undiscovered, pathophysiology that could explain the phenomenon.</p

Management of orthodontic emergencies in primary care – self-reported confidence of general dental practitioners

Objective: To determine general dental practitioners’ (GDPs) confidence in managing orthodontic emergencies. Design: Cross-sectional study. Setting: Primary dental care. Subjects and methods: An online survey was distributed to dentists practicing in Wales. The survey collected basic demographic information and included descriptions of ten common orthodontic emergency scenarios. Main outcome measure Respondents’ self-reported confidence in managing the orthodontic emergency scenarios on a 5‑point Likert scale. Differences between the Likert responses and the demographic variables were investigated using chi-squared tests. Results: The median number of orthodontic emergencies encountered by respondents over the previous six months was 1. Overall, the self-reported confidence of respondents was high with 7 of the 10 scenarios presented scoring a median of 4 indicating that GDPs were ‘confident’ in their management. Statistical analysis revealed that GDPs who saw more orthodontic emergencies in the previous six months were more confident when managing the presented scenarios. Other variables such as age, gender, geographic location of practice and number of years practising dentistry were not associated with self reported confidence. Conclusions: Despite GDPs encountering very few orthodontic emergencies in primary care, they appear to be confident in dealing with commonly arising orthodontic emergency situations

Impact of seasonal variation, age and smoking status on human semen parameters: The Massachusetts General Hospital experience

BACKGROUND: To investigate the relationship of human semen parameters with season, age and smoking status. METHODS: The present study used data from subjects recruited into an ongoing cross-sectional study on the relationship between environmental agents and semen characteristics. Our population consisted of 306 patients who presented to the Vincent Memorial Andrology Laboratory of Massachusetts General Hospital for semen evaluation. Sperm concentration and motility were measured with computer aided sperm analysis (CASA). Sperm morphology was scored using Tygerberg Kruger strict criteria. Regression analyses were used to investigate the relationships between semen parameters and season, age and smoking status, adjusting for abstinence interval. RESULTS: Sperm concentration in the spring was significantly higher than in winter, fall and summer (p < 0.05). There was suggestive evidence of higher sperm motility and percent of sperm with normal morphology in the spring than in the other seasons. There were no statistically significant relationships between semen parameters and smoking status, though current smokers tended to have lower sperm concentration. We also did not find a statistically significant relationship between age and semen parameters. CONCLUSIONS: We found seasonal variations in sperm concentration and suggestive evidence of seasonal variation in sperm motility and percent sperm with normal morphology. Although smoking status was not a significant predictor of semen parameters, this may have been due to the small number of current smokers in the study

Ecologic Niche Modeling of Blastomyces dermatitidis in Wisconsin

Background: Blastomycosis is a potentially fatal mycosis that is acquired by inhaling infectious spores of Blastomyces dermatitidis present in the environment. The ecology of this pathogen is poorly understood, in part because it has been extremely difficult to identify the niche(s) it occupies based on culture isolation of the organism from environmental samples. Methodology/Principal Findings: We investigated the ecology of blastomycosis by performing maximum entropy modeling of exposure sites from 156 cases of human and canine blastomycosis to provide a regional-scale perspective of the geographic and ecologic distribution of B. dermatitidis in Wisconsin. Based on analysis with climatic, topographic, surface reflectance and other environmental variables, we predicted that ecologic conditions favorable for maintaining the fungus in nature occur predominantly within northern counties and counties along the western shoreline of Lake Michigan. Areas of highest predicted occurrence were often in proximity to waterways, especially in northcentral Wisconsin, where incidence of infection is highest. Ecologic conditions suitable for B. dermatitidis are present in urban and rural environments, and may differ at the extremes of distribution of the species in the state. Conclusions/Significance: Our results provide a framework for a more informed search for specific environmental factors modulating B. dermatitidis occurrence and transmission and will be useful for improving public health awareness of relativ