Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Conventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach

Magnetic signatures of plasma-depleted flux tubes in the Saturnian inner magnetosphere

Initial Cassini observations have revealed evidence for interchanging magnetic flux tubes in the inner Saturnian magnetosphere. Some of the reported flux tubes differ remarkably by their magnetic signatures, having a depressed or enhanced magnetic pressure relative to their surroundings. The ones with stronger fields have been interpreted previously as either outward moving mass-loaded or inward moving plasma-depleted flux tubes based on magnetometer observations only. We use detailed multi-instrumental observations of small and large density depletions in the inner Saturnian magnetosphere from Cassini Rev. A orbit that enable us to discriminate amongst the two previous and opposite interpretations. Our analysis undoubtedly confirms the similar nature of both types of reported interchanging magnetic flux tubes, which are plasma-depleted, whatever their magnetic signatures are. Their different magnetic signature is clearly an effect associated with latitude. These Saturnian plasma-depleted flux tubes ultimately may play a similar role as the Jovian ones

Bacterial persisters are a stochastically formed subpopulation of low-energy cells.

Persisters represent a small subpopulation of non- or slow-growing bacterial cells that are tolerant to killing by antibiotics. Despite their prominent role in the recalcitrance of chronic infections to antibiotic therapy, the mechanism of their formation has remained elusive. We show that sorted cells of Escherichia coli with low levels of energy-generating enzymes are better able to survive antibiotic killing. Using microfluidics time-lapse microscopy and a fluorescent reporter for in vivo ATP measurements, we find that a subpopulation of cells with a low level of ATP survives killing by ampicillin. We propose that these low ATP cells are formed stochastically as a result of fluctuations in the abundance of energy-generating components. These findings point to a general "low energy" mechanism of persister formation

A Compromise between Neutrino Masses and Collider Signatures in the Type-II Seesaw Model

A natural extension of the standard $SU(2)_{\rm L} \times U(1)_{\rm Y}$ gauge model to accommodate massive neutrinos is to introduce one Higgs triplet and three right-handed Majorana neutrinos, leading to a $6\times 6$ neutrino mass matrix which contains three $3\times 3$ sub-matrices $M_{\rm L}$, $M_{\rm D}$ and $M_{\rm R}$. We show that three light Majorana neutrinos (i.e., the mass eigenstates of $\nu_e$, $\nu_\mu$ and $\nu_\tau$) are exactly massless in this model, if and only if $M_{\rm L} = M_{\rm D} M_{\rm R}^{-1} M_{\rm D}^T$ exactly holds. This no-go theorem implies that small but non-vanishing neutrino masses may result from a significant but incomplete cancellation between $M_{\rm L}$ and $M_{\rm D} M_{\rm R}^{-1} M_{\rm D}^T$ terms in the Type-II seesaw formula, provided three right-handed Majorana neutrinos are of ${\cal O}(1)$ TeV and experimentally detectable at the LHC. We propose three simple Type-II seesaw scenarios with the $A_4 \times U(1)_{\rm X}$ flavor symmetry to interpret the observed neutrino mass spectrum and neutrino mixing pattern. Such a TeV-scale neutrino model can be tested in two complementary ways: (1) searching for possible collider signatures of lepton number violation induced by the right-handed Majorana neutrinos and doubly-charged Higgs particles; and (2) searching for possible consequences of unitarity violation of the $3\times 3$ neutrino mixing matrix in the future long-baseline neutrino oscillation experiments.Comment: RevTeX 19 pages, no figure

Bilateral Assessment of Functional Tasks for Robot-assisted Therapy Applications

This article presents a novel evaluation system along with methods to evaluate bilateral coordination of arm function on activities of daily living tasks before and after robot-assisted therapy. An affordable bilateral assessment system (BiAS) consisting of two mini-passive measuring units modeled as three degree of freedom robots is described. The process for evaluating functional tasks using the BiAS is presented and we demonstrate its ability to measure wrist kinematic trajectories. Three metrics, phase difference, movement overlap, and task completion time, are used to evaluate the BiAS system on a bilateral symmetric (bi-drink) and a bilateral asymmetric (bi-pour) functional task. Wrist position and velocity trajectories are evaluated using these metrics to provide insight into temporal and spatial bilateral deficits after stroke. The BiAS system quantified movements of the wrists during functional tasks and detected differences in impaired and unimpaired arm movements. Case studies showed that stroke patients compared to healthy subjects move slower and are less likely to use their arm simultaneously even when the functional task requires simultaneous movement. After robot-assisted therapy, interlimb coordination spatial deficits moved toward normal coordination on functional tasks

Natural images from the birthplace of the human eye

Here we introduce a database of calibrated natural images publicly available through an easy-to-use web interface. Using a Nikon D70 digital SLR camera, we acquired about 5000 six-megapixel images of Okavango Delta of Botswana, a tropical savanna habitat similar to where the human eye is thought to have evolved. Some sequences of images were captured unsystematically while following a baboon troop, while others were designed to vary a single parameter such as aperture, object distance, time of day or position on the horizon. Images are available in the raw RGB format and in grayscale. Images are also available in units relevant to the physiology of human cone photoreceptors, where pixel values represent the expected number of photoisomerizations per second for cones sensitive to long (L), medium (M) and short (S) wavelengths. This database is distributed under a Creative Commons Attribution-Noncommercial Unported license to facilitate research in computer vision, psychophysics of perception, and visual neuroscience.Comment: Submitted to PLoS ON

Incidence of community-acquired lower respiratory tract infections and pneumonia among older adults in the United Kingdom: a population-based study.

Community-acquired lower respiratory tract infections (LRTI) and pneumonia (CAP) are common causes of morbidity and mortality among those aged ≥65 years; a growing population in many countries. Detailed incidence estimates for these infections among older adults in the United Kingdom (UK) are lacking. We used electronic general practice records from the Clinical Practice Research Data link, linked to Hospital Episode Statistics inpatient data, to estimate incidence of community-acquired LRTI and CAP among UK older adults between April 1997-March 2011, by age, sex, region and deprivation quintile. Levels of antibiotic prescribing were also assessed. LRTI incidence increased with fluctuations over time, was higher in men than women aged ≥70 and increased with age from 92.21 episodes/1000 person-years (65-69 years) to 187.91/1000 (85-89 years). CAP incidence increased more markedly with age, from 2.81 to 21.81 episodes/1000 person-years respectively, and was higher among men. For both infection groups, increases over time were attenuated after age-standardisation, indicating that these rises were largely due to population aging. Rates among those in the most deprived quintile were around 70% higher than the least deprived and were generally higher in the North of England. GP antibiotic prescribing rates were high for LRTI but lower for CAP (mostly due to immediate hospitalisation). This is the first study to provide long-term detailed incidence estimates of community-acquired LRTI and CAP in UK older individuals, taking person-time at risk into account. The summary incidence commonly presented for the ≥65 age group considerably underestimates LRTI/CAP rates, particularly among older individuals within this group. Our methodology and findings are likely to be highly relevant to health planners and researchers in other countries with aging populations

Propylene glycol inactivates respiratory viruses and prevents airborne transmission

Viruses are vulnerable as they transmit between hosts, and we aimed to exploit this critical window. We found that the ubiquitous, safe, inexpensive and biodegradable small molecule propylene glycol (PG) has robust virucidal activity. Propylene glycol rapidly inactivates a broad range of viruses including influenza A, SARS-CoV-2 and rotavirus and reduces disease burden in mice when administered intranasally at concentrations commonly found in nasal sprays. Most critically, vaporised PG efficiently abolishes influenza A virus and SARS-CoV-2 infectivity within airborne droplets, potently preventing infection at levels well below those tolerated by mammals. We present PG vapour as a first-in-class non-toxic airborne virucide that can prevent transmission of existing and emergent viral pathogens, with clear and immediate implications for public health

Search for Exotic Strange Quark Matter in High Energy Nuclear Reactions

We report on a search for metastable positively and negatively charged states of strange quark matter in Au+Pb reactions at 11.6 A GeV/c in experiment E864. We have sampled approximately six billion 10% most central Au+Pb interactions and have observed no strangelet states (baryon number A < 100 droplets of strange quark matter). We thus set upper limits on the production of these exotic states at the level of 1-6 x 10^{-8} per central collision. These limits are the best and most model independent for this colliding system. We discuss the implications of our results on strangelet production mechanisms, and also on the stability question of strange quark matter.Comment: 21 pages, 9 figures, to be published in Nuclear Physics A (Carl Dover memorial edition

Rationale, design and methods of the Study of Work and Pain (SWAP): a cluster randomised controlled trial testing the addition of a vocational advice service to best current primary care for patients with musculoskeletal pain (ISRCTN 52269669)

Background Musculoskeletal pain is a major contributor to short and long term work absence. Patients seek care from their general practitioner (GP) and yet GPs often feel ill-equipped to deal with work issues. Providing a vocational case management service in primary care, to support patients with musculoskeletal problems to remain at or return to work, is one potential solution but requires robust evaluation to test clinical and cost-effectiveness. Methods/Design This protocol describes a cluster randomised controlled trial, with linked qualitative interviews, to investigate the effect of introducing a vocational advice service into general practice, to provide a structured approach to managing work related issues in primary care patients with musculoskeletal pain who are absent from work or struggling to remain in work. General practices (n = 6) will be randomised to offer best current care or best current care plus a vocational advice service. Adults of working age who are absent from or struggling to remain in work due to a musculoskeletal pain problem will be invited to participate and 330 participants will be recruited. Data collection will be through patient completed questionnaires at baseline, 4 and 12 months. The primary outcome is self-reported work absence at 4 months. Incremental cost-utility analysis will be undertaken to calculate the cost per additional QALY gained and incremental net benefits. A linked interview study will explore the experiences of the vocational advice service from the perspectives of GPs, nurse practitioners (NPs), patients and vocational advisors. Discussion This paper presents the rationale, design, and methods of the Study of Work And Pain (SWAP) trial. The results of this trial will provide evidence to inform primary care practice and guide the development of services to provide support for musculoskeletal pain patients with work-related issues. Trial registration Current Controlled Trials ISRCTN52269669