Application of reinforcement learning methods to computer game dynamics

The dynamics of the game world present both challenges and opportunities for AI to make a useful difference. Learning smart behaviours for game assets is a first step towards realistic conflict or cooperation. The scope this thesis is the application of Reinforcement Learning to moving assets in the game world. Game sessions a generate stream data on asset's performance which must be processed on the fly. The lead objective is to produce fast, lightweight and flexible learning algorithms for run-time embedding. The motivation from current work is to shorten the time to achieve a workable policy solution by investigating the exploration / exploitation balance, overcome the curse of dimensionality of complex systems, and avoid the use of extra endogenous parameters which require multiple data passes and use a simple state aggregation rather than functional approximation. How action selection (AS) contributes to efficient learning is a key issue in RL since is determines the balance between exploiting and confirming the current policy or exploring an early less likely policy which may prove better in the long run. The methodology deploys the simulation of several AS using 10-armed bandit problem averaged over 10000 epochs. The results show a considerable variation in performance in terms of latency and asymptotic direction. The Upper Confidence Bound comes out leader over most of the episode range, especially at about 100. Using insight from action selection order statistics are applied to determine a criterion for the convergence of policy evaluation. The probability that the action of maximum sample mean is indeed the action of maximum population mean (PMSMMPM) is calculated using the 3 armed bandit problem. PMSMMPM reaches 0.988 by play 26 which provides evidence for it as a convergence criterion. An iteration stopping rule is defined using PMSMMPM and it shows plausible properties as the population parameters are varied. A mathematical analysis of the approximation (P21) of just taking the top two actions yields a minimum sampling size for any level of P21. Using the gradient of P21 a selection rule is derived and when combined with UCB a new complete exploratory policy is demonstrated for 3-arm bandit that requires just over half the sample size when compared with pure UCB. The results provide evidence that the augmented UCB selection rule will contribute to faster learning. TD sarsa(0) learning algorithm has been applied to learn a steering policy for the untried caravan reversing problem and for the kerb avoiding steering problem of racing car both using negative rewards on failure and a simple aggregation. The output policy for the caravan is validated as non jack-knifing for a high proportion of start states. The racing car policy has a similar validation outcome for two exploratory polies which are compared and contrasted

Larval settlement and juvenile group dynamics in the domino damselfish (Dascyllus albisella)

Patterns of settlement of larvae and population dynamics of juveniles are poorly known for coral reef fishes. During 1987 to 1989, I studied these phenomena in the domino damselfish (Dascyllus albisella), a species endemic to the Hawaiian Islands. Larvae settle onto branching coral heads as new recruits (10-15 mm in length), usually with conspecific groups, and remain on the coral heads through juvenile life until maturity (70 mm total length). By conducting experiments on natural patch reefs and on an artificially distributed grid of coral heads, I found that most larvae settle at night, and that they settle preferentially on corals supporting large conspecific groups compared to small groups or empty corals. Within a group, juveniles form a linear dominance hierarchy based on fish size; aggressive interactions are mainly directed by larger fish towards smaller fish. Tagging studies demonstrated that growth was retarded in larger groups and for fish of low social status, but that survival, especially of new recruits, was enhanced in larger groups. Therefore, I identified both a growth cost and a survival benefit to group living. I derived a measure of net benefit of group living by combining size-specific growth and survival data into an estimate of the probability of reaching mature size. This estimate increased with group size in 1988 but not in 1987. I developed a simulation model which used my field data on settlement rate, settlement preferences, and juvenile growth and survival to predict demography of juvenile groups. The model successfully predicted seasonal fluctuations in mean group size, and estimated the number of fish maturing in 1987 and 1988, as a function of settlement rate and preferences and of juvenile growth and survival. Numbers maturing were directly related to settlement rate in both years, except at high rates in 1987, suggesting that primary recruitment limitation of adult numbers could be occurring. Settlement preferences also influenced numbers maturing. At all settlement rates, numbers maturing differed between years, suggesting that secondary recruitment limitation of adult numbers may also occur

Reviews

The following publications have been reviewed by the mentioned authors;Design Assignments for GCSE and Standard Grade by Jeff Nocholls, reviewed by Bob BoothDesign and Communication for Foundation Courses by Peter Gowers, reviewed by Peter ToftThe Celtic Art Source Book by Courtney Davis, reviewed by Rosemary BoothThe World of Plastics by the British Plastics Federation, reviewed by George RaperIllumination: A Source Book for Modern Calligraphers by Christopher Jarman, reviewed by John LancasterCraft, Design and Technology. Materials: Investigation and Choice. A database for pupil use by David Barles, Simon Powell, Sheila Nelmes and Adrienne Jones, reviewed by David BarlexDesign in Context by Penny Sparke, reviewed by John H. CarswellDesignwise 2 by N. McLean, reviewed by H. G. DentonAirbrushing the Human Form by Andy Charlesworth, reviewed by S. W. GarnerDouglas Scott by Johnathan Glancey, reviewed by K. J. McAule

Circadian preference and physical and cognitive performance in adolescence:A scoping review

Adolescence is a crucial period of development which coincides with changes in circadian rhythmicity. This may augment the impact of circadian preference on performance in this group. We aimed to scope the literature available on chronotypes and their effect on physical and mental aspects of performance in adolescents. Methods: Studies were identified by systematically searching bibliographical databases and grey literature. Results: The Morningness-Eveningness Questionnaire was the most frequently reported tool for circadian preference assessment. Academic achievement was the most prevailing outcome, with evidence suggesting that morning type adolescents tend to outperform evening types, yet the results vary depending on multiple factors. Performance in tests of intelligence and executive functions was generally better at optimal times of the day (synchrony effect). Physical performance was examined in 8 studies, with very heterogeneous outcomes. Conclusions: Although the associations between circadian preference and performance in adolescents are evident in some areas, there are many factors that may be involved in the relationship and require further investigation. This review highlights the assessment of physical performance in relation to chronotypes, the multidimensional assessment of circadian preference, and the need for longitudinal studies as priorities for further research

Single-ended differential protection in MTDC networks using optical sensors

This paper presents a method for rapid detection of faults on VSC multi-terminal HVDC transmission networks using multi-point optical current sensing. The proposed method uses differential protection as a guiding principle, and is implemented using current measurements obtained from optical current sensors distributed along the transmission line. Performance is assessed through detailed transient simulation using Matlab/Simulink® models, integrating inductive DC-line terminations, detailed DC circuit breaker models and a network of fiber-optic current sensors. Moreover, the feasibility and required performance of optical-based measurements is validated through laboratory testing. Simulation results demonstrate that the proposed protection algorithm can effectively, and within very short period of time, discriminate between faults on the protected line (internal faults), and those occurring on adjacent lines or busbars (external faults). Hardware tests prove that the scheme can be achieved with the existing, available sensing technology

Ensuring an Essential Supply of Allied Health Professions (AHP) Placements: Using Crowdsourcing to Develop a National Call to Action

Sustainable growth in the Allied Health Professions (AHP) workforce is an ambition of the United Kingdom’s NHS Long Term Plan. However historically, access to good quality placements has been a barrier to increasing pre-registration training numbers. This article focuses on work carried out by Health Education England (HEE) to gain insights on the impact of the COVID-19 pandemic on capacity. Using a pragmatic, embedded mixed-methods approach, insights were gathered using an online workshop, crowdsourcing, open for two weeks in the summer of 2020. AHP placement stakeholders could vote, share ideas or comment. Descriptive data were extracted, and comments made were analysed using inductive thematic analysis. Participants (N = 1,800) made over 8,500 comments. The themes identified included: diversity of placement opportunity, improved placement coordination, a more joined-up system, supervision models and educator capacity. Alongside considering the challenges to placement capacity, several areas of innovative practice owing to the pandemic were highlighted. Generated insights have shaped the aims and objectives of the Health Education (HEE) pre-registration AHP student practice learning programme for 2020/2021 and beyond. The COVID-19 pandemic has disrupted the delivery of AHP placements. In the absence of face-to-face activities, crowdsourcing provided an online data collection tool offering stakeholders an opportunity to engage with the placement capacity agenda and share learning. Findings have shaped the HEE approach to short-term placement recovery and long-term growth

Physical activity, diet and other behavioural interventions for improving cognition and school achievement in children and adolescents with obesity or overweight

Background: The global prevalence of childhood and adolescent obesity is high. Lifestyle changes towards a healthy diet, increased physical activity and reduced sedentary activities are recommended to prevent and treat obesity. Evidence suggests that changing these health behaviours can benefit cognitive function and school achievement in children and adolescents in general. There are various theoretical mechanisms that suggest that children and adolescents with excessive body fat may benefit particularly from these interventions. Objectives: To assess whether lifestyle interventions (in the areas of diet, physical activity, sedentary behaviour and behavioural therapy) improve school achievement, cognitive function (e.g. executive functions) and/or future success in children and adolescents with obesity or overweight, compared with standard care, waiting-list control, no treatment, or an attention placebo control group. Search methods: In February 2017, we searched CENTRAL, MEDLINE and 15 other databases. We also searched two trials registries, reference lists, and handsearched one journal from inception. We also contacted researchers in the field to obtain unpublished data. Selection criteria: We included randomised and quasi-randomised controlled trials (RCTs) of behavioural interventions for weight management in children and adolescents with obesity or overweight. We excluded studies in children and adolescents with medical conditions known to affect weight status, school achievement and cognitive function. We also excluded self- and parent-reported outcomes. Data collection and analysis: Four review authors independently selected studies for inclusion. Two review authors extracted data, assessed quality and risks of bias, and evaluated the quality of the evidence using the GRADE approach. We contacted study authors to obtain additional information. We used standard methodological procedures expected by Cochrane. Where the same outcome was assessed across different intervention types, we reported standardised effect sizes for findings from single-study and multiple-study analyses to allow comparison of intervention effects across intervention types. To ease interpretation of the effect size, we also reported the mean difference of effect sizes for single-study outcomes. Main results: We included 18 studies (59 records) of 2384 children and adolescents with obesity or overweight. Eight studies delivered physical activity interventions, seven studies combined physical activity programmes with healthy lifestyle education, and three studies delivered dietary interventions. We included five RCTs and 13 cluster-RCTs. The studies took place in 10 different countries. Two were carried out in children attending preschool, 11 were conducted in primary/elementary school-aged children, four studies were aimed at adolescents attending secondary/high school and one study included primary/elementary and secondary/high school-aged children. The number of studies included for each outcome was low, with up to only three studies per outcome. The quality of evidence ranged from high to very low and 17 studies had a high risk of bias for at least one item. None of the studies reported data on additional educational support needs and adverse events. Compared to standard practice, analyses of physical activity-only interventions suggested high-quality evidence for improved mean cognitive executive function scores. The mean difference (MD) was 5.00 scale points higher in an after-school exercise group compared to standard practice (95% confidence interval (CI) 0.68 to 9.32; scale mean 100, standard deviation 15; 116 children, 1 study). There was no statistically significant beneficial effect in favour of the intervention for mathematics, reading, or inhibition control. The standardised mean difference (SMD) for mathematics was 0.49 (95% CI -0.04 to 1.01; 2 studies, 255 children, moderate-quality evidence) and for reading was 0.10 (95% CI -0.30 to 0.49; 2 studies, 308 children, moderate-quality evidence). The MD for inhibition control was -1.55 scale points (95% CI -5.85 to 2.75; scale range 0 to 100; SMD -0.15, 95% CI -0.58 to 0.28; 1 study, 84 children, very low-quality evidence). No data were available for average achievement across subjects taught at school. There was no evidence of a beneficial effect of physical activity interventions combined with healthy lifestyle education on average achievement across subjects taught at school, mathematics achievement, reading achievement or inhibition control. The MD for average achievement across subjects taught at school was 6.37 points lower in the intervention group compared to standard practice (95% CI -36.83 to 24.09; scale mean 500, scale SD 70; SMD -0.18, 95% CI -0.93 to 0.58; 1 study, 31 children, low-quality evidence). The effect estimate for mathematics achievement was SMD 0.02 (95% CI -0.19 to 0.22; 3 studies, 384 children, very low-quality evidence), for reading achievement SMD 0.00 (95% CI -0.24 to 0.24; 2 studies, 284 children, low-quality evidence), and for inhibition control SMD -0.67 (95% CI -1.50 to 0.16; 2 studies, 110 children, very low-quality evidence). No data were available for the effect of combined physical activity and healthy lifestyle education on cognitive executive functions. There was a moderate difference in the average achievement across subjects taught at school favouring interventions targeting the improvement of the school food environment compared to standard practice in adolescents with obesity (SMD 0.46, 95% CI 0.25 to 0.66; 2 studies, 382 adolescents, low-quality evidence), but not with overweight. Replacing packed school lunch with a nutrient-rich diet in addition to nutrition education did not improve mathematics (MD -2.18, 95% CI -5.83 to 1.47; scale range 0 to 69; SMD -0.26, 95% CI -0.72 to 0.20; 1 study, 76 children, low-quality evidence) and reading achievement (MD 1.17, 95% CI -4.40 to 6.73; scale range 0 to 108; SMD 0.13, 95% CI -0.35 to 0.61; 1 study, 67 children, low-quality evidence). Authors' conclusions: Despite the large number of childhood and adolescent obesity treatment trials, we were only able to partially assess the impact of obesity treatment interventions on school achievement and cognitive abilities. School and community-based physical activity interventions as part of an obesity prevention or treatment programme can benefit executive functions of children with obesity or overweight specifically. Similarly, school-based dietary interventions may benefit general school achievement in children with obesity. These findings might assist health and education practitioners to make decisions related to promoting physical activity and healthy eating in schools. Future obesity treatment and prevention studies in clinical, school and community settings should consider assessing academic and cognitive as well as physical outcomes

Longitudinal associations between childhood obesity and academic achievement:Systematic review with focus group data

Purpose The purposes of this study were to review the evidence on longitudinal associations between child and adolescent obesity and academic achievement and to provide perceptions of adolescents with obesity and their parents on this topic. Recent Findings Synthesis of 31 studies (from 17 cohorts) suggested that relationships between obesity and academic achievement are not well established, except for adolescent girls’ maths attainment, potentially mediated by both weight-related bullying and executive cognitive functions. Focus groups with adolescent girls with obesity confirmed experiences of psychosocial distress at school particularly during Physical Education. Adolescents perceived that obesity was not related to academic achievement directly, but by their attitude to school. Summary Interventions are warranted to promote psychosocial wellbeing and cognitive abilities linked to academic achievement in adolescent girls with obesity. Physical Education should be a positive experience for children and adolescents with obesity