Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Conventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach

Elevated antibody to D-alanyl lipoteichoic acid indicates caries experience associated with fluoride and gingival health

BACKGROUND: Acidogenic, acid-tolerant bacteria induce dental caries and require D-alanyl glycerol lipoteichoic acid (D-alanyl LTA) on their cell surface. Because fluoride inhibits acid-mediated enamel demineralization, an elevated antibody response to D-alanyl LTA may indicate subjects with more acidogenic bacteria and, therefore, an association of DMFT with fluoride exposure and gingival health not apparent in low responders. METHODS: Cluster analysis was used to identify low antibody content. Within low and high responders (control and test subjects), the number of teeth that were decayed missing and filled (DMFT), or decayed only (DT) were regressed against fluoride exposure in the water supply and from dentrifice use. The latter was determined from gingival health: prevalences of plaque (PL) and bleeding on probing (BOP), and mean pocket depth (PD). Age was measured as a possible confounding cofactor. RESULTS: In 35 high responders, DMFT associated with length of exposure to fluoridated water (F score), PL and BOP (R(2) = 0.51, p < 0.001), whereas in 67 low D-ala-IgG responders, DMFT associated with PL, age, and PD (R(2) = 0.26, p < 0.001). BOP correlated strongly with number of 7 7 decayed teeth (DT) in 54 high responders (R(2) = 0.57, p < 0.001), but poorly in 97 low responders (R(2) = 0.12, p < 0.001). The strength of the PD association with DMFT, or of BOP with DT, in high responders significantly differed from that in low responders (p < 0.05). CONCLUSION: Caries associates with gingival health and fluoridated water exposure in high D-alanyl LTA antibody responders

Loss of strength in Ni3Al at elevated temperatures

Stress decrease above the stress peak temperature (750 K) is studied in h123i single crystals of Ni3(Al, 3 at.% Hf ). Two thermally activated deformation mechanisms are evidenced on the basis of stress relaxation and strain rate change experiments. From 500 to 1070 K, the continuity of the activation volume/temperature curves reveals a single mechanism of activation enthalpy 3.8 eV/atom and volume 90 b3 at 810K with an athermal stress of 330 MPa. Over the very same temperature interval, impurity or solute diffusion towards dislocation cores is evidenced through serrated yielding, peculiar shapes of stress–strain curves while changing the rate of straining and stress relaxation experiments. This complicates the identification of the deformation mechanism, which is likely connected with cube glide. From 1070 to 1270 K, the high-temperature mechanism has an activation enthalpy and volume of 4.8 eV/atom and 20 b3, respectively, at 1250 K

Primary mediastinal large B-cell lymphoma in HIV: report of two cases

Primary mediastinal large B cell lymphoma (PMLBCL) is a subtype of diffuse large B cell lymphoma arising in the mediastinum with distinctive clinical and morphological features. Though diffuse large B cell lymphoma is one of the most common non-Hodgkin lymphoma associated with AIDS, there are no data available regarding the association of HIV and PMLBCL. We report here two cases of PMLBCL arising in AIDS patients. In both cases, PMLBCL presented in a setting of low CD4 T-cell count as rapidly enlarging mediastinal mass. The morphologic and immunophenotypic findings are characteristic of PMLBCL. One of the two patients, a 25-year-old woman who had localized disease and evidence of Epstein–Barr virus in lymphoma cells, did not respond to chemotherapy and died of disease progression 5 months after diagnosis. The second patient, a 38-year-old male with disseminated disease, responded to therapy and is disease-free after 9 months of follow-up

Soluble tumor necrosis factor receptor 1 and 2 predict outcomes in advanced chronic kidney disease : a prospective cohort study

Background : Soluble tumor necrosis factor receptors 1 (sTNFR1) and 2 (sTNFR2) have been associated to progression of renal failure, end stage renal disease and mortality in early stages of chronic kidney disease (CKD), mostly in the context of diabetic nephropathy. The predictive value of these markers in advanced stages of CKD irrespective of the specific causes of kidney disease has not yet been defined. In this study, the relationship between sTNFR1 and sTNFR2 and the risk for adverse cardiovascular events (CVE) and all-cause mortality was investigated in a population with CKD stage 4-5, not yet on dialysis, to minimize the confounding by renal function. Patients and methods : In 131 patients, CKD stage 4-5, sTNFR1, sTNFR2 were analysed for their association to a composite endpoint of all-cause mortality or first non-fatal CVE by univariate and multivariate Cox proportional hazards models. In the multivariate models, age, gender, CRP, eGFR and significant comorbidities were included as covariates. Results : During a median follow-up of 33 months, 40 events (30.5%) occurred of which 29 deaths (22.1%) and 11 (8.4%) first non-fatal CVE. In univariate analysis, the hazard ratios (HR) of sTNFR1 and sTNFR2 for negative outcome were 1.49 (95% confidence interval (CI): 1.28-1.75) and 1.13 (95% CI: 1.06-1.20) respectively. After adjustment for clinical covariables (age, CRP, diabetes and a history of cardiovascular disease) both sTNFRs remained independently associated to outcomes (HR: sTNFR1: 1.51, 95% CI: 1.30-1.77; sTNFR2: 1.13, 95% CI: 1.06-1.20). A subanalysis of the non-diabetic patients in the study population confirmed these findings, especially for sTNFR1. Conclusion : sTNFR1 and sTNFR2 are independently associated to all-cause mortality or an increased risk for cardiovascular events in advanced CKD irrespective of the cause of kidney disease

Predicting the Distribution of Spiral Waves from Cell Properties in a Developmental-Path Model of Dictyostelium Pattern Formation

The slime mold Dictyostelium discoideum is one of the model systems of biological pattern formation. One of the most successful answers to the challenge of establishing a spiral wave pattern in a colony of homogeneously distributed D. discoideum cells has been the suggestion of a developmental path the cells follow (Lauzeral and coworkers). This is a well-defined change in properties each cell undergoes on a longer time scale than the typical dynamics of the cell. Here we show that this concept leads to an inhomogeneous and systematic spatial distribution of spiral waves, which can be predicted from the distribution of cells on the developmental path. We propose specific experiments for checking whether such systematics are also found in data and thus, indirectly, provide evidence of a developmental path

Iron deposition and inflammation in multiple sclerosis. Which one comes first?

Whether iron deposition is an epiphenomenon of the multiple sclerosis (MS) disease process or may play a primary role in triggering inflammation and disease development remains unclear at this time, and should be studied at the early stages of disease pathogenesis. However, it is difficult to study the relationship between iron deposition and inflammation in early MS due to the delay between the onset of symptoms and diagnosis, and the poor availability of tissue specimens. In a recent article published in BMC Neuroscience, Williams et al. investigated the relationship between inflammation and iron deposition using an original animal model labeled as "cerebral experimental autoimmune encephalomyelitis", which develops CNS perivascular iron deposits. However, the relative contribution of iron deposition vs. inflammation in the pathogenesis and progression of MS remains unknown. Further studies should establish the association between inflammation, reduced blood flow, iron deposition, microglia activation and neurodegeneration. Creating a representative animal model that can study independently such relationship will be the key factor in this endeavor

Cost-effectiveness of replacing skeletal traction by interlocked intramedullary nailing for femoral shaft fractures in a provincial trauma hospital in Cambodia

In this article the costs and effectiveness of introducing the SIGN nailing system for femoral shaft fractures in a provincial trauma hospital in Cambodia are compared to those of Perkin’s traction treatment. At an average cost per patient of $1,107 in the traction group and$888 in the nail group (p < 0.01), and with better clinical outcomes in the nail group, internal fixation is more cost-effective than conservative treatment

Relevance of JAK2V617F positivity to hematological diseases - survey of samples from a clinical genetics laboratory

<p>Abstract</p> <p>Background</p> <p>JAK2V617F is found in the majority of patients with Ph- myeloproliferative neoplasms (MPNs) and has become a valuable marker for diagnosis of MPNs. However, it has also been found in many other hematological diseases, and some studies even detected the presence of JAK2V617F in normal blood samples. This casts doubt on the primary role of JAK2V617F in the pathogenesis of MPNs and its diagnostic value.</p> <p>Methods</p> <p>In the present study, we analyzed JAK2V617F positivity with 232 normal blood samples and 2663 patient blood, bone marrow, and amniotic fluid specimens obtained from a clinical genetics laboratory by using a simple DNA extraction method and a sensitive nested allele-specific PCR strategy.</p> <p>Results</p> <p>We found JAK2V617F present in the majority (78%) of MPN patients and in a small fraction (1.8-8.7%) of patients with other specific hematological diseases but not at all in normal healthy donors or patients with non-hematological diseases. We also revealed associations of JAK2V617F with novel as well as known chromosomal abnormalities.</p> <p>Conclusions</p> <p>Our study suggests that JAK2V617F positivity is associated with specific hematological malignancies and is an excellent diagnostic marker for MPNs. The data also indicate that the nested allele-specific PCR method provides clinically relevant information and should be conducted for all cases suspected of having MPNs as well as for other related diseases.</p