Static and bootstrapped neuro-simulation for complex robots in evolutionary robotics

Evolutionary Robotics (ER) is a field of study focused on the automatic development of controllers and robot morphologies. Evolving controllers on real-world hardware is time-consuming and can damage hardware through wear. Robotic simulators can be used as an alternative to a real-world robot in order to speed up the ER process. Most simulation techniques in practice use physics-based models that rely on an understanding of the robotic system in question. Developing effective physics-based simulators is time consuming and requires a significant level of specialised knowledge. A lengthy simulator development and tuning process is typically required before the ER process can begin. Artificial Neural Networks simulators (SNNs) can be used as an alternative to a physics based simulation approach. SNNs are simple to construct, do not require significant levels of prior knowledge of the robotic system, are computationally efficient and can be highly accurate. Two types of ER approaches utilising SNNs exist. The Static Neuro-Simulation (SNS) approach involves developing SNNs before the ER process where these SNNs are used instead of a physics-based simulator. Alternatively, SNNs can be developed during the ER process, called the Bootstrapped Neuro-Simulation (BNS) approach. Prior work investigating SNNs has largely been limited to simple robots. A complex robot has many degrees of freedom and ifa low-level controller design is used, the solution search space is high-dimensional and difficult to traverse. Prior work investigating the SNS and BNS approaches have mostly relied on simplified controller designs which rely on built-in prior knowledge of intended robot behaviours. This research uses low-level controller designs which in turn rely on low level simulators. Most ER studies are conducted on a single type of robot morphology. This research investigates the SNS and BNS approaches on two significantly different classes of robots. A Hexapod and Snake robot are used to study the SNS and BNS approaches. The Hexapod robot exhibits limbed, walking behaviours. The Snake robot is limbless and generates crawling behaviours. Demonstrating the viability of the SNS and BNS approaches for two different classes of robots provides strong evidence that the tested approaches are likely viable on other classes of robots. Various proposed improvements to the SNS and BNS approaches are investigated. The Results demonstrate that the SNS and BNS approaches are viable when applied to Hexapod and Snake robots without restricting controller designs to those with significant levels of built-in prior knowledge of robot behaviours. SNNs configured in ensembles improve the likely performance outcomes of solutions. The expected benefit of adding simulator noise during the evolutionary process were not as pronounced for problems investigated in this work

Damage recovery for robot controllers and simulators evolved using bootstrapped neuro-simulation

Robots are becoming increasingly complex. This has made manually designing the software responsible for controlling these robots (controllers) challenging, leading to the creation of the field of evolutionary robotics (ER). The ER approach aims to automatically evolve robot controllers and morphologies by utilising concepts from biological evolution. ER techniques use evolutionary algorithms (EA) to evolve populations of controllers - a process that requires the evaluation of a large number of controllers. Performing these evaluations on a real-world robot is both infeasibly time-consuming and poses the risk of damage to the robot. Simulators present a solution to the issue by allowing the evaluation of controllers to take place on a virtual robot. Traditional methods of controller evolution in simulation encounter two major issues. Firstly, physics simulators are complex to create and are often very computationally expensive. Secondly, the reality gap is encountered when controllers are evolved in simulators that are unable to simulate the real world well enough due to implications or small inaccuracies in the simulation, which together cause controllers in the simulation to be unable to transfer effectively to reality. Bootstrapped Neuro-Simulation (BNS) is an ER algorithm that aims to address the issues inherent with the use of simulators. The algorithm concurrently creates a simulator and evolves a population of controllers. The process starts with an initially random population of controllers and an untrained simulator neural network (SNN), a type of robot simulator which utilises artificial neural networks (ANNs) to simulate a robot's behaviour. Controllers are then continually selected for evaluation in the real world, and the data from these real-world evaluations is used to train the controller-evaluation SNN. BNS is a relatively new algorithm that has not yet been explored in depth. An investigation was, therefore, conducted into BNS's ability to evolve closed-loop controllers. BNS was successful in evolving such controllers, and various adaptations to the algorithm were investigated for their ability to improve the evolution of closed-loop controllers. In addition, the factors which had the greatest impact on BNS's effectiveness were reported upon. Damage recovery is an area that has been the focus of a great deal of research. This is because the progression of the field of robotics means that robots no longer operate only in the safe environments that they once did. Robots are now put to use in areas as inaccessible as the surface of Mars, where repairs by a human are impossible. Various methods of damage recovery have previously been proposed and evaluated, but none focused on BNS as a method of damage recovery. In this research, it was hypothesised that BNS's constantly learning nature would allow it to recover from damage, as it would continue to use new information about the state of the real robot to evolve new controllers capable of functioning in the damaged robot. BNS was found to possess the hypothesised damage recovery ability. The algorithm's evaluation was carried out through the evolution of controllers for simple navigation and light-following tasks for a wheeled robot, as well as a locomotion task for a complex legged robot. Various adaptations to the algorithm were then evaluated through extensive parameter investigations in simulation, showing varying levels of effectiveness. These results were further confirmed through evaluation of the adaptations and effective parameter values in real-world evaluations on a real robot. Both a simple and more complex robot morphology were investigated

Radiation induced brain necrosis after proton therapy for head and neck cancer

Protonterapi gir reduksjon i normalvevsdoser sammenlignet med strålebehandling med fotoner, og det er antatt at dette vil resultere i en lavere forekomst av stråleinduserte seinskader. For å kompensere for en økt biologisk effektivitet av protoner benyttes en konstant verdi for den relative biologiske effektiviteten (RBE) som er satt til 1.1. Det er imidlertid kjent at RBE varierer blant annet med lineær energioverføring (LET), dose og strålesensitivitetsparameteren α/β. Et stort antall RBE-modeller basert på disse variablene har blitt utviklet for bedre å beskrive proton RBE. Klinisk er det manglende kunnskap om effekten av en variabel proton RBE, men det er sterke in vitro-bevis for en økning i RBE som en funksjon av LET. For kreft lokalisert i skallebasisregion kan strålebehandling medføre at deler av hjernen får høye stråledoser. Disse pasientene har derfor en livslang risiko for å utvikle stråleindusert hjernenekrose. Denne diagnosen stilles som oftest på grunnlag av MR-radiologi, typisk ser en kontrastoppladede lesjoner på T1-vektede sekvenser og økt intensitet på T2 sekvenser. I litteraturen er insidensen av disse bildeendringene (RAIC) godt beskrevet etter intensitetsmodulert strålebehandling (IMRT) for HNC, men det er begrenset antall studier på pasienter behandlet med protonterapi. Nylig har det kommet forskning som indikerer at RAIC forekommer hyppigere etter protonterapi. Det har blitt stilt en hypotese om at dette er relatert til økning i RBE på grunn av forhøyet LET i distale delen av protonstrålen. Målet med dette doktorgradsarbeidet var derfor å utforske risikofaktorer for utvikling av RAIC hos pasienter behandlet med protonterapi for hode-halskreft lokalisert i skallebaseregionen. Pasientene i studien hadde blitt behandlet med intensitetsmodulert protonterapi (IMPT) og/eller passiv teknikk (PSPT) ved MD Anderson Cancer Center mellom 2010 og 2018. I første artikkel ble insidens av RAIC undersøkt og dose grenser for å redusere risiko ble identifisert. Insidens av RAIC var 17% noe som samsvarte godt med det som har blitt funnet i andre studier etter protonterapi. Flertallet av disse ble behandlet for nasopharynx eller sinonasal kreft (77%). Det ble funnet lesjoner i temporallappene, frontallappen og i cerebellum, de fleste i kant eller så vidt overlappende med CTV. Ingen av pasientene hadde symptomer assosiert med hjernenekrose. Lesjoner var i progresjon hos 18% av pasientene. V67 Gy(RBE) < 0.2cc til hjerne be identifisert som den viktigste dose-variabelen for å begrense risikoen for å utvikle RAIC. I artikkel II var målet å undersøke hvordan RBE-variasjoner kan påvirke estimert risiko for å utvikle stråleindusert temporallappsnekrose. Monte Carlo-simuleringer ble brukt til å beregne variabel RBE-vektede doser (RWDVar) ved hjelp av to publiserte RBE-modeller. Vi fant at RWDVar var signifikant høyere enn doser kalkulert med RBE = 1.1 (RWDFix). Vi fant videre indikasjoner for at risikoen for å utvikle temporallappsnekrose kan bli undervurdert hvis dosegrenser vurderes basert på RWDFix. Maksimal dose til temporallappen var svært influert av variabel RBE, noe som resulterte i store usikkerheter og økning i estimert risiko, mens de andre undersøkte dosevariablene var mindre påvirket av variabel RBE. Resultatet fra denne studien viser at å inkludere RWDVar som en del av IMPT behandlingsplanevaluering kan gi verdifull klinisk informasjon når det gjelder beskyttelse av temporallappen. I artikkel III så vi etter korrelasjoner mellom områder med strålingsnekrose, dose og dose-gjennomsnittlig LET (LETd). Femten pasienter diagnostisert med RAIC som hadde blitt behandlet med IMPT ble inkludert i analysen. Nøyaktige dose- og LETd-fordelinger ble beregnet ved hjelp av Monte Carlo-simuleringer og ekstrahert på voxelnivå fra pasientenes behandlingsplaner. En logistisk regresjonsmodell som estimerer tilfeldige og faste effekter ble brukt i analysen. Analysen avdekket betydelige interpasient variasjoner, men allikevel en signifikant korrelasjon mellom økende LETd og regioner med RAIC. Resultatene våre antydet at LETd-effekten kan være av klinisk betydning for noen pasienter, og at LETd-vurdering i kliniske behandlingsplaner derfor bør tas i betraktning. Samlet sett har dette arbeidet gitt økt kunnskap om risiko for utvikling av stråleeffekter i hjernen etter protonterapi. Forekomsten av RAIC for hode-hals kreft i skallebasisregionen er sammenlignbar med det en har sett i andre protonserier. Våre funn tyder på at variabel RBE-relaterte usikkerheter og potensielle LETd-effekter kan være avgjørende og bør inngå som del av klinisk behandlingsplanevaluering. Selv om dette ofte vurderes implisitt ved protonterapi, bør av beregnings- og planleggingsverktøy basert på spesifikke LETd-data sammen med fysisk dose implementeres i klinisk praksis.With proton therapy, reduction in normal tissue doses is achievable with equal or better target dose conformity due to the Bragg Peak effect. The main rationale for proton therapy today is based on an assumption that this will translate into a more favorable treatment outcome, specifically in terms of lower normal tissue complication rates. However, proton therapy is accompanied with an inherent uncertainty in the actual biological dose delivered. It is well recognized that the constant Relative Biological Effectiveness (RBE) currently applied in clinical proton beams is a simplification of the reality; rather than being a fixed factor, RBE varies depending on several physical, biological and treatment related factors. A wide range of models derived from in-vitro data have been proposed to describe the variable RBE based on the Linear Energy Transfer (LET), dose and α/β. Clinically, the relationship between biological effect and variable RBE is not well understood, however there are strong in-vitro evidence for an increase in RBE as a function of LET. During radiotherapy for head and neck cancer (HNC) at the skull base region, patients may receive high radiation doses to parts of the brain and will therefore have a lifelong risk of developing radiation-induced brain necrosis. Patients are most commonly diagnosed on the basis of characteristic changes on Magnetic Resonance Images (MRI), including contrast enhanced lesions on T1-weighted sequences or hyperintensities on T2-weighted sequences. There are numerous publications addressing these radiation associate image changes (RAIC) after intensity modulated radiotherapy (IMRT) for HNC, however, there are limited number of studies in patients treated with proton therapy. Previous research in pediatric and adult patient cohorts treated for both intracranial and extracranial skull base tumors suggest that RAIC occur more frequently after proton therapy. It has been hypothesized that this may be explained by an increase in the LET at the distal part of the proton beam. The aim of this PhD work was to explore RAIC in patients treated with proton therapy for HNC at the skull base region. The patient material included a wide range of HNCs treated with intensity modulated proton therapy (IMPT) and/ or passive scattering proton therapy (PSPT) at MD Anderson Cancer Center between 2010 and 2018. In paper I, the incidence and patterns of RAIC were investigated, and practical brain dose constraints (RBE = 1.1) associated with RAIC were derived. The incidence of RAIC corresponded reasonably well with observed rates previously reported after proton therapy. During a median latency time of 24 months, RAIC were found on follow-up MRIs in 22 out of 127 patients (17%). The majority of the patients with RAIC were treated for nasopharyngeal or sinonasal cancers (77%). Lesions were found in the temporal lobes, frontal lobes and the cerebellum, typically outside or slightly overlapping with the CTV. All lesions were asymptomatic. On the last available follow-up MRI, 18% of the lesions were in progression, whereas 27% had resolved. RAIC was significantly associated with dosimetric variables only. Brain V67 Gy (RBE) < 0.2cc was identified as the most important dose volume threshold in order to limit risk of developing RAIC. In paper II, the aim was to investigate the influence of RBE variations on the assessment of risk of developing temporal lobe necrosis. The patient material included 45 patients treated with IMPT and who had a follow-up time of 24 months or longer. Image changes diagnosed radiation necrosis was observed in sixteen temporal lobes. Monte Carlo simulations were used to calculate RWDVar based on two previously published RBE models. The RWDVar was significantly increased compared to RWDFix. We further found indications that the risk of developing temporal lobe necrosis could be underestimated when evaluating dose constraints according to RWDFix. Dose-volume predictors with near-maximum doses were less influenced by RBE variations than the maximum dose. The result from this study suggests that including RWDVar as part of IMPT treatment plan evaluation may provide valuable clinical information in terms of temporal lobe protection. In paper III we looked for correlations between regions of radiation necrosis, dose and dose-averaged LET (LETd). Fifteen patients with RAIC who had been treated with IMPT were included in the analysis. Accurate dose- and LETd distributions were calculated using Monte Carlo simulations and extracted voxel-by-voxel from the patients’ treatment plans using an in-house developed MATLAB-script. Mixed effect logistic regression methodology were used for analysis. The analysis revealed substantial interpatient variations, however an overall significant correlation between increasing LETd and regions with RAIC. Our results suggested that the LETd effect could be of clinical significance for some patients and that LETd assessment in clinical treatment plans should therefore be taken into consideration. Overall, this work has provided increased knowledge on risk factors for development of radiation effects in the brain after proton therapy. Incidence rates of RAIC in HNC at the skull base are comparable to other proton series. Our findings suggest that variable RBE related uncertainties and potential LETd effects are essential to consider in clinical treatment plan evaluation. Although often considered implicitly in the mind of the clinician for proton therapy, continued evidence such as the current work may lead to changes in clinical practice, namely the implementation of computerized calculation and planning tools based on specifically LETd data along with physical dose.Doktorgradsavhandlin

Effects of Diversity and Neuropsychological Performance in an NFL Cohort

Objective: The aim of this study was to examine the effect of ethnicity on neuropsychological test performance by comparing scores of white and black former NFL athletes on each subtest of the WMS. Participants and Methods: Data was derived from a de-identified database in South Florida consisting of 63 former NFL white (n=28, 44.4%) and black (n=35, 55.6%) athletes (Mage= 50.38; SD= 11.57). Participants completed the following subtests of the WMS: Logical Memory I and II, Verbal Paired Associates I and II, and Visual Reproduction I and II. Results: A One-Way ANOVA yielded significant effect between ethnicity and performance on several subtests from the WMS-IV. Black athletes had significantly lower scores compared to white athletes on Logical Memory II: F(1,61) = 4.667, p= .035, Verbal Paired Associates I: F(1,61) = 4.536, p = .037, Verbal Paired Associates: II F(1,61) = 4.677, p = .034, and Visual Reproduction I: F(1,61) = 6.562, p = .013. Conclusions: Results suggest significant differences exist between white and black athletes on neuropsychological test performance, necessitating the need for proper normative samples for each ethnic group. It is possible the differences found can be explained by the psychometric properties of the assessment and possibility of a non-representative sample for minorities, or simply individual differences. Previous literature has found white individuals to outperform African-Americans on verbal and non-verbal cognitive tasks after controlling for socioeconomic and other demographic variables (Manly & Jacobs, 2002). This highlights the need for future investigators to identify cultural factors and evaluate how ethnicity specifically plays a role on neuropsychological test performance. Notably, differences between ethnic groups can have significant implications when evaluating a sample of former athletes for cognitive impairment, as these results suggest retired NFL minorities may be more impaired compared to retired NFL white athletes

Distinguishing Performance on Tests of Executive Functions Between Those with Depression and Anxiety

Objective: To see if there are differences in executive functions between those diagnosed with Major Depressive Disorder (MDD) and those with Generalized Anxiety Disorder (GAD).Participants and Methods: The data were chosen from a de-identified database at a neuropsychological clinic in South Florida. The sample used was adults diagnosed with MDD (n=75) and GAD (n=71) and who had taken the Halstead Category Test, Trail Making Test, Stroop Test, and the Wisconsin Card Sorting Test. Age (M=32.97, SD=11.75), gender (56.7% female), and race (52.7% White) did not differ between groups. IQ did not differ but education did (MDD=13.41 years, SD=2.45; GAD=15.11 years, SD=2.40), so it was ran as a covariate in the analyses. Six ANCOVAs were run separately with diagnosis being held as the fixed factor and executive function test scores held as dependent variables. Results: The MDD group only performed worse on the Category Test than the GAD group ([1,132]=4.022, p\u3c .05). Even though both WCST scores used were significantly different between the two groups, both analyses failed Levene’s test of Equality of Error Variances, so the data were not interpreted. Conclusions: Due to previous findings that those diagnosed with MDD perform worse on tests of executive function than normal controls (Veiel, 1997), this study wanted to compare executive function performance between those diagnosed with MDD and those with another common psychological disorder. The fact that these two groups only differed on the Category Test shows that there may not be much of a difference in executive function deficits between those with MDD and GAD. That being said, not being able to interpret the scores on the WCST test due to a lack of homogeneity of variance indicates that a larger sample size is needed to compare these two types of patients, as significant differences may be found. The results of this specific study, however, could mean that the Category Test could be used in assisting the diagnosis of a MDD patient

The Effect of Ethnicity on Neuropsychological Test Performance of Former NFL Athletes

Objective: To investigate the effect of ethnicity on neuropsychological test performance by specifically exploring differences between white and black former NFL athletes on subtests of the WAIS-IV. Participants and Methods: Data was derived from a de-identified database in Florida consisting of 63 former NFL athletes (Mage=50.38; SD=11.57); 28 white and 35 black. Participants completed the following subtests of the WAIS-IV: Block Design, Similarities, Digit Span, Matrix Reasoning, Arithmetic, Symbol Search, Visual Puzzles, Coding, and Cancellation. Results: One-Way ANOVA yielded a significant effect between ethnicity and performance on several subtests. Black athletes had significantly lower scaled scores than white athletes on Block Design F(1,61)=14.266, p\u3c.001, Similarities F(1,61)=5.904, p=.018, Digit Span F(1,61)=8.985, p=.004, Arithmetic F(1,61)=16.07, p\u3c.001 and Visual Puzzles F(1,61)=16.682, p\u3c .001. No effect of ethnicity was seen on performance of Matrix Reasoning F(1,61)=2.937, p=.092, Symbol Search F(1,61)=3.619, p=.062, Coding F(1,61)=3.032, p=.087 or Cancellation F(1,61)=2.289, p=.136. Conclusions: Results reveal significant differences between white and black athletes on all subtests of the WAIS-IV but those from the Processing Speed Scale and Matrix Reasoning. These findings align with previous literature that found white individuals to outperform African-Americans on verbal and non-verbal tasks after controlling for socioeconomic and demographic variables (Manly & Jacobs, 2002). These differences may also be a reflection of the WAIS-IV’s psychometric properties and it is significant to consider the normative sample used may not be appropriate for African-Americans. This study highlights the need for future research to identify how ethnicity specifically influences performance, sheds light on the importance of considering cultural factors when interpreting test results, and serves as a call to action to further understand how and why minorities may not be accurately represented in neuropsychological testing

Regional Cerebral Blood Flow Patterns in Children vs. Adults with ADHD Combined and Inattentive Types: A SPECT Study

Objective: The current study sought to determine whether ADHD Combined Type (ADHD-C) and ADHD Primarily Inattentive Type (ADHD-PI) showed differential regional cerebral blood flow (rCBF) patterns in children vs. adults. Participants and Methods: The overall sample (N=1484) was effectively split into four groups: adults with ADHD-PI (n=519), adults with ADHD-C (n=405), children with ADHD-PI (n=192), children with ADHD-C (n=368). All participants were void of bipolar, schizophrenia, autism, neurocognitive disorders, and TBI. The data were collected from a de-identified archival database of individuals who underwent SPECT scans at rest. Results: Using αConclusions: Overall, the current study suggested that children may show rCBF differences between different ADHD subtypes, but adults may not. The current study did not find significance in any of the 17 brain regions examined when comparing adults with ADHD-C to adults with ADHD-PI. All significant findings were attributed to the children with ADHD-C group showing aberrant blood flow rate than at least one other group. Previous research has supported that the differentiation of these subtypes as distinctive disorders is difficult to make in adults (Sobanski et al., 2006). Other research has indicated the potential of imaging techniques to differentiate the two in children (Al-Amin, Zinchenko, & Geyer, 2018). The current findings support nuanced ways in which rCBF patterns of ADHD-C and ADHD-PI differ between children and adults

Pseudoneglect and visual attention networks

Pseudoneglect represents the tendency for healthy individuals to show a slight but consistent bias in favour of stimuli appearing in the left visual field. The bias is often measured using variants of the line bisection task. An accurate model of the functional architecture of the visuospatial attention system must account for this widely observed phenomenon, as well as for modulation of the direction and magnitude of the bias within individuals by a variety of factors relating to the state of the participant and/or stimulus characteristics. To date, the neural correlates of pseudoneglect remain relatively unmapped. In the current thesis, I employed a combination of psychophysical measurements, electroencephalography (EEG) recording and transcranial direct current stimulation (tDCS) in an attempt to probe the neural generator(s) of pseudoneglect. In particular, I wished to utilise and investigate some of the factors known to modulate the bias (including age, time-on-task and the length of the to-be-bisected line) in order to identify neural processes and activity that are necessary and sufficient for the lateralized bias to arise. Across four experiments utilising a computerized version of a perceptual line bisection task, pseudoneglect was consistently observed at baseline in healthy young participants. However, decreased line length (experiments 1, 2 and 3), time-on-task (experiment 1) and healthy aging (experiment 3) were all found to modulate the bias. Specifically, all three modulations induced a rightward shift in subjective midpoint estimation. Additionally, the line length and time-on-task effects (experiment 1) and the line length and aging effects (experiment 3) were found to have additive relationships. In experiment 2, EEG measurements revealed the line length effect to be reflected in neural activity 100 – 200ms post-stimulus onset over source estimated posterior regions of the right hemisphere (RH: temporo-parietal junction (TPJ)). Long lines induced a hemispheric asymmetry in processing (in favour of the RH) during this period that was absent in short lines. In experiment 4, bi-parietal tDCS (Left Anodal/Right Cathodal) induced a polarity-specific rightward shift in bias, highlighting the crucial role played by parietal cortex in the genesis of pseudoneglect. The opposite polarity (Left Cathodal/Right Anodal) did not induce a change in bias. The combined results from the four experiments of the current thesis provide converging evidence as to the crucial role played by the RH in the genesis of pseudoneglect and in the processing of visual input more generally. The reduction in pseudoneglect with decreased line length, increased time-on-task and healthy aging may be explained by a reduction in RH function, and hence contribution to task processing, induced by each of these modulations. I discuss how behavioural and neuroimaging studies of pseudoneglect (and its various modulators) can provide empirical data upon which accurate formal models of visuospatial attention networks may be based and further tested