Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Conventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach

Life-course origins of social inequalities in adult immune cell markers of inflammation in a developing southern Chinese population: the Guangzhou Biobank Cohort Study

BACKGROUND: Socioeconomic position (SEP) throughout life is associated with cardiovascular disease, though the mechanisms linking these two are unclear. It is also unclear whether there are critical periods in the life course when exposure to better socioeconomic conditions confers advantages or whether SEP exposures accumulate across the whole life course. Inflammation may be a mechanism linking socioeconomic position (SEP) with cardiovascular disease. In a large sample of older residents of Guangzhou, in southern China, we examined the association of life course SEP with inflammation. METHODS: In baseline data on 9,981 adults (>/= 50 years old) from the Guangzhou Biobank Cohort Study (2006-08), we used multivariable linear regression and model fit to assess the associations of life course SEP at four stages (childhood, early adult, late adult and current) with white blood, granulocyte and lymphocyte cell counts. RESULTS: A model including SEP at all four life stages best explained the association of life course SEP with white blood and granulocyte cell count for men and women, with early adult SEP (education) making the largest contribution. A critical period model best explained the association of life course SEP with lymphocyte count, with sex-specific associations. Early adult SEP was negatively associated with lymphocytes for women. CONCLUSIONS: Low SEP throughout life may negatively impact late adult immune-inflammatory status. However, some aspects of immune-inflammatory status may be sensitive to earlier exposures, with sex-specific associations. The findings were compatible with the hypothesis that in a developing population, upregulation of the gonadotropic axis with economic development may obscure the normally protective effects of social advantage for men.published_or_final_versio

Behavioral determinants as predictors of return to work after long-term sickness absence: an application of the theory of planned behavior

Background The aim of this prospective, longitudinal cohort study was to analyze the association between the three behavioral determinants of the theory of planned behavior (TPB) model-attitude, subjective norm and self-efficacy-and the time to return-to-work (RTW) in employees on long-term sick leave. Methods The study was based on a sample of 926 employees on sickness absence (maximum duration of 12 weeks). The employees filled out a baseline questionnaire and were subsequently followed until the tenth month after listing sick. The TPB-determinants were measured at baseline. Work attitude was measured with a Dutch language version of the Work Involvement Scale. Subjective norm was measured with a self-structured scale reflecting a person's perception of social support and social pressure. Self-efficacy was measured with the three subscales of a standardised Dutch version of the general self-efficacy scale (ALCOS): willingness to expend effort in completing the behavior, persistence in the face of adversity, and willingness to initiate behavior. Cox proportional hazards regression analyses were used to identify behavioral determinants of the time to RTW. Results Median time to RTW was 160 days. In the univariate analysis, all potential prognostic factors were significantly associated (P < 0.15) with time to RTW: work attitude, social support, and the three subscales of self-efficacy. The final multivariate model with time to RTW as the predicted outcome included work attitude, social support and willingness to expend effort in completing the behavior as significant predictive factors. Conclusions This prospective, longitudinal cohort-study showed that work attitude, social support and willingness to expend effort in completing the behavior are significantly associated with a shorter time to RTW in employees on long-term sickness absence. This provides suggestive evidence for the relevance of behavioral characteristics in the prediction of duration of sickness absence. It may be a promising approach to address the behavioral determinants in the development of interventions focusing on RTW in employees on long-term sick leave

The clinical features of the piriformis syndrome: a systematic review

Piriformis syndrome, sciatica caused by compression of the sciatic nerve by the piriformis muscle, has been described for over 70 years; yet, it remains controversial. The literature consists mainly of case series and narrative reviews. The objectives of the study were: first, to make the best use of existing evidence to estimate the frequencies of clinical features in patients reported to have PS; second, to identify future research questions. A systematic review was conducted of any study type that reported extractable data relevant to diagnosis. The search included all studies up to 1 March 2008 in four databases: AMED, CINAHL, Embase and Medline. Screening, data extraction and analysis were all performed independently by two reviewers. A total of 55 studies were included: 51 individual and 3 aggregated data studies, and 1 combined study. The most common features found were: buttock pain, external tenderness over the greater sciatic notch, aggravation of the pain through sitting and augmentation of the pain with manoeuvres that increase piriformis muscle tension. Future research could start with comparing the frequencies of these features in sciatica patients with and without disc herniation or spinal stenosis

Single Gene Deletions of Orexin, Leptin, Neuropeptide Y, and Ghrelin Do Not Appreciably Alter Food Anticipatory Activity in Mice

Timing activity to match resource availability is a widely conserved ability in nature. Scheduled feeding of a limited amount of food induces increased activity prior to feeding time in animals as diverse as fish and rodents. Typically, food anticipatory activity (FAA) involves temporally restricting unlimited food access (RF) to several hours in the middle of the light cycle, which is a time of day when rodents are not normally active. We compared this model to calorie restriction (CR), giving the mice 60% of their normal daily calorie intake at the same time each day. Measurement of body temperature and home cage behaviors suggests that the RF and CR models are very similar but CR has the advantage of a clearly defined food intake and more stable mean body temperature. Using the CR model, we then attempted to verify the published result that orexin deletion diminishes food anticipatory activity (FAA) but observed little to no diminution in the response to CR and, surprisingly, that orexin KO mice are refractory to body weight loss on a CR diet. Next we tested the orexigenic neuropeptide Y (NPY) and ghrelin and the anorexigenic hormone, leptin, using mouse mutants. NPY deletion did not alter the behavior or physiological response to CR. Leptin deletion impaired FAA in terms of some activity measures, such as walking and rearing, but did not substantially diminish hanging behavior preceding feeding time, suggesting that leptin knockout mice do anticipate daily meal time but do not manifest the full spectrum of activities that typify FAA. Ghrelin knockout mice do not have impaired FAA on a CR diet. Collectively, these results suggest that the individual hormones and neuropepetides tested do not regulate FAA by acting individually but this does not rule out the possibility of their concerted action in mediating FAA

Attitudes to colorectal cancer screening among ethnic minority groups in the UK

Background: Colorectal screening by Flexible Sigmoidoscopy (FS) is under evaluation in the UK. Evidence from existing cancer screening programmes indicates lower participation among minority ethnic groups than the white-British population. To ensure equality of access, it is important to understand attitudes towards screening in all ethnic groups so that barriers to screening acceptance can be addressed.Methods: Open- and closed-ended questions on knowledge about colorectal cancer and attitudes to FS screening were added to Ethnibus (TM) - a monthly, nationwide survey of the main ethnic minority communities living in the UK (Indian, Pakistani, Bangladeshi, Caribbean, African, and Chinese). Interviews (n = 875) were conducted, face-to-face, by multilingual field-workers, including 125 interviews with white-British adults.Results: All respondents showed a notable lack of knowledge about causes of colorectal cancer, which was more pronounced in ethnic minority than white-British adults. Interest in FS screening was uniformly high (> 60%), with more than 90% of those interested saying it would provide 'peace of mind'. The most frequently cited barrier to screening 'in your community' was embarrassment, particularly among ethnic minority groups.Conclusion: Educational materials should recognise that non-white groups may be less knowledgeable about colorectal cancer. The findings of the current study suggest that embarrassment may be a greater deterrent to participation to FS screening among ethnic minority groups, but this result requires exploration in further research

Reliability and validity of a short form household food security scale in a Caribbean community

BACKGROUND: We evaluated the reliability and validity of the short form household food security scale in a different setting from the one in which it was developed. METHODS: The scale was interview administered to 531 subjects from 286 households in north central Trinidad in Trinidad and Tobago, West Indies. We evaluated the six items by fitting item response theory models to estimate item thresholds, estimating agreement among respondents in the same households and estimating the slope index of income-related inequality (SII) after adjusting for age, sex and ethnicity. RESULTS: Item-score correlations ranged from 0.52 to 0.79 and Cronbach's alpha was 0.87. Item responses gave within-household correlation coefficients ranging from 0.70 to 0.78. Estimated item thresholds (standard errors) from the Rasch model ranged from -2.027 (0.063) for the 'balanced meal' item to 2.251 (0.116) for the 'hungry' item. The 'balanced meal' item had the lowest threshold in each ethnic group even though there was evidence of differential functioning for this item by ethnicity. Relative thresholds of other items were generally consistent with US data. Estimation of the SII, comparing those at the bottom with those at the top of the income scale, gave relative odds for an affirmative response of 3.77 (95% confidence interval 1.40 to 10.2) for the lowest severity item, and 20.8 (2.67 to 162.5) for highest severity item. Food insecurity was associated with reduced consumption of green vegetables after additionally adjusting for income and education (0.52, 0.28 to 0.96). CONCLUSIONS: The household food security scale gives reliable and valid responses in this setting. Differing relative item thresholds compared with US data do not require alteration to the cut-points for classification of 'food insecurity without hunger' or 'food insecurity with hunger'. The data provide further evidence that re-evaluation of the 'balanced meal' item is required

The role of power in financial statement fraud schemes

In this paper, we investigate a large-scale financial statement fraud to better understand the process by which individuals are recruited to participate in financial statement fraud schemes. The case reveals that perpetrators often use power to recruit others to participate in fraudulent acts. To illustrate how power is used, we propose a model, based upon the classical French and Raven taxonomy of power, that explains how one individual influences another individual to participate in financial statement fraud. We also provide propositions for future research

Angular and Current-Target Correlations in Deep Inelastic Scattering at HERA

Correlations between charged particles in deep inelastic ep scattering have been studied in the Breit frame with the ZEUS detector at HERA using an integrated luminosity of 6.4 pb-1. Short-range correlations are analysed in terms of the angular separation between current-region particles within a cone centred around the virtual photon axis. Long-range correlations between the current and target regions have also been measured. The data support predictions for the scaling behaviour of the angular correlations at high Q2 and for anti-correlations between the current and target regions over a large range in Q2 and in the Bjorken scaling variable x. Analytic QCD calculations and Monte Carlo models correctly describe the trends of the data at high Q2, but show quantitative discrepancies. The data show differences between the correlations in deep inelastic scattering and e+e- annihilation.Comment: 26 pages including 10 figures (submitted to Eur. J. Phys. C