Why it is important to build robots capable of doing science

Science, like any other cognitive activity, is grounded in the sensorimotor interaction of our bodies with the environment. Human embodiment thus constrains the class of scientific concepts and theories which are accessible to us. The paper explores the possibility of doing science with artificial cognitive agents, in the framework of an interactivist-constructivist cognitive model of science. Intelligent robots, by virtue of having different sensorimotor capabilities, may overcome the fundamental limitations of human science and provide important technological innovations. Mathematics and nanophysics are prime candidates for being studied by artificial scientists

Neural Signatures of Motor Skill in the Resting Brain

Stroke-induced disturbances of large-scale cortical networks are known to be associated with the extent of motor deficits. We argue that identifying brain networks representative of motor behavior in the resting brain would provide significant insights for current neurorehabilitation approaches. Particularly, we aim to investigate the global configuration of brain rhythms and their relation to motor skill, instead of learning performance as broadly studied. We empirically approach this problem by conducting a three-dimensional physical space visuomotor learning experiment during electroencephalographic (EEG) data recordings with thirty-seven healthy participants. We demonstrate that across-subjects variations in average movement smoothness as the quantified measure of subjects' motor skills can be predicted from the global configuration of resting-state EEG alpha-rhythms (8-14 Hz) recorded prior to the experiment. Importantly, this neural signature of motor skill was found to be orthogonal to (independent of) task -- as well as to learning-related changes in alpha-rhythms, which we interpret as an organizing principle of the brain. We argue that disturbances of such configurations in the brain may contribute to motor deficits in stroke, and that reconfiguring stroke patients' brain rhythms by neurofeedback may enhance post-stroke neurorehabilitation.Comment: 2019 IEEE International Conference on Systems, Man, and Cybernetics (IEEE SMC 2019

Sensory Manipulation as a Countermeasure to Robot Teleoperation Delays: System and Evidence

In the field of robotics, robot teleoperation for remote or hazardous environments has become increasingly vital. A major challenge is the lag between command and action, negatively affecting operator awareness, performance, and mental strain. Even with advanced technology, mitigating these delays, especially in long-distance operations, remains challenging. Current solutions largely focus on machine-based adjustments. Yet, there's a gap in using human perceptions to improve the teleoperation experience. This paper presents a unique method of sensory manipulation to help humans adapt to such delays. Drawing from motor learning principles, it suggests that modifying sensory stimuli can lessen the perception of these delays. Instead of introducing new skills, the approach uses existing motor coordination knowledge. The aim is to minimize the need for extensive training or complex automation. A study with 41 participants explored the effects of altered haptic cues in delayed teleoperations. These cues were sourced from advanced physics engines and robot sensors. Results highlighted benefits like reduced task time and improved perceptions of visual delays. Real-time haptic feedback significantly contributed to reduced mental strain and increased confidence. This research emphasizes human adaptation as a key element in robot teleoperation, advocating for improved teleoperation efficiency via swift human adaptation, rather than solely optimizing robots for delay adjustment.Comment: Submitted to Scientific Report

Are There Brain-Based Predictors of the Ability to Learn a New Skill in Healthy Ageing and Can They Help in the Design of Effective Therapy after Stroke?

This thesis aimed at looking for neural correlates of motor adaptation as a model of rehabilitation after brain injury. Healthy adults across the lifespan and stroke patients were tested in a force-field learning paradigm. This thesis focuses on EEG analysis and the complex relationship of brain-derived measures with observed behaviour. To describe each domain in detail, the focus was first on finding group differences between older and younger healthy adults in a similar manner as it was later between stroke patients versus healthy controls. The analyses were finalised by looking for relationships between the EEG and motor performance data in a multiple linear regression approach. As candidate EEG biomarkers of motor adaptation, error related event related potential around movement onset in the frontocentral electrodes was chosen in time domain. In the time-frequency domain, the focus was on movement related beta band spectral perturbation, looking at the electrodes over the primary motor cortex and the frontocentral ROI found significant in the time domain. Finally, functional connectivity was analysed focusing first on electrode over the primary motor cortex contralateral to the movement as a seed region, to narrow down the analysis to bilateral motor cortex connectivity and connectivity between primary motor cortex contralateral to the movement and the frontocentral region identified as important in the time domain analysis. The crucial part of the project was analysing the relationship between the neural and kinematic measures. The most important predictor of summed error in motor adaptation was the connectivity between C3 and C4 electrode at the baseline prestimulus period in motor adaptation condition and pinch asymmetry. Higher prestimulus interhemispheric connectivity was associated with bigger deviation from the optimal trajectory. When looking at summed error dynamic derivative as a dependent variable - performance index - it was the ERP at the central error-related ROI that explained the most variance. It can be concluded that higher baseline interhemispheric connectivity can be a reflection of a maladaptive process, perhaps related to increased interhemispheric inhibition. It is important to also note that the same connectivity at different timepoints in the movement can be of different significance - differences between stroke patients and controls were present in the postmovement period. In conclusion, brain information could be helpful for e.g. stratifying patients into different intensity programs based on their predicted potential to recover. Moreover, brain information could be utilised to apply closed-loop systems modulating the intensity of tasks to reach the optimal brain state that facilitates learning. I believe this work will help incorporating brain-derived measures in informing neurorehabilitation programmes in the future

Hand eye coordination in surgery

The coordination of the hand in response to visual target selection has always been regarded as an essential quality in a range of professional activities. This quality has thus far been elusive to objective scientific measurements, and is usually engulfed in the overall performance of the individuals. Parallels can be drawn to surgery, especially Minimally Invasive Surgery (MIS), where the physical constraints imposed by the arrangements of the instruments and visualisation methods require certain coordination skills that are unprecedented. With the current paradigm shift towards early specialisation in surgical training and shortened focused training time, selection process should identify trainees with the highest potentials in certain specific skills. Although significant effort has been made in objective assessment of surgical skills, it is only currently possible to measure surgeons’ abilities at the time of assessment. It has been particularly difficult to quantify specific details of hand-eye coordination and assess innate ability of future skills development. The purpose of this thesis is to examine hand-eye coordination in laboratory-based simulations, with a particular emphasis on details that are important to MIS. In order to understand the challenges of visuomotor coordination, movement trajectory errors have been used to provide an insight into the innate coordinate mapping of the brain. In MIS, novel spatial transformations, due to a combination of distorted endoscopic image projections and the “fulcrum” effect of the instruments, accentuate movement generation errors. Obvious differences in the quality of movement trajectories have been observed between novices and experts in MIS, however, this is difficult to measure quantitatively. A Hidden Markov Model (HMM) is used in this thesis to reveal the underlying characteristic movement details of a particular MIS manoeuvre and how such features are exaggerated by the introduction of rotation in the endoscopic camera. The proposed method has demonstrated the feasibility of measuring movement trajectory quality by machine learning techniques without prior arbitrary classification of expertise. Experimental results have highlighted these changes in novice laparoscopic surgeons, even after a short period of training. The intricate relationship between the hands and the eyes changes when learning a skilled visuomotor task has been previously studied. Reactive eye movement, when visual input is used primarily as a feedback mechanism for error correction, implies difficulties in hand-eye coordination. As the brain learns to adapt to this new coordinate map, eye movements then become predictive of the action generated. The concept of measuring this spatiotemporal relationship is introduced as a measure of hand-eye coordination in MIS, by comparing the Target Distance Function (TDF) between the eye fixation and the instrument tip position on the laparoscopic screen. Further validation of this concept using high fidelity experimental tasks is presented, where higher cognitive influence and multiple target selection increase the complexity of the data analysis. To this end, Granger-causality is presented as a measure of the predictability of the instrument movement with the eye fixation pattern. Partial Directed Coherence (PDC), a frequency-domain variation of Granger-causality, is used for the first time to measure hand-eye coordination. Experimental results are used to establish the strengths and potential pitfalls of the technique. To further enhance the accuracy of this measurement, a modified Jensen-Shannon Divergence (JSD) measure has been developed for enhancing the signal matching algorithm and trajectory segmentations. The proposed framework incorporates high frequency noise filtering, which represents non-purposeful hand and eye movements. The accuracy of the technique has been demonstrated by quantitative measurement of multiple laparoscopic tasks by expert and novice surgeons. Experimental results supporting visual search behavioural theory are presented, as this underpins the target selection process immediately prior to visual motor action generation. The effects of specialisation and experience on visual search patterns are also examined. Finally, pilot results from functional brain imaging are presented, where the Posterior Parietal Cortical (PPC) activation is measured using optical spectroscopy techniques. PPC has been demonstrated to involve in the calculation of the coordinate transformations between the visual and motor systems, which establishes the possibilities of exciting future studies in hand-eye coordination

EEG Characterization of Sensorimotor Networks: Implications in Stroke

The purpose of this dissertation was to use electroencephalography (EEG) to characterize sensorimotor networks and examine the effects of stroke on sensorimotor networks. Sensorimotor networks play an essential role in completion of everyday tasks, and when damaged, as in stroke survivors, the successful completion of seemingly simple motor tasks becomes fantasy. When sensorimotor networks are impaired as a result of stroke, varying degrees of sensorimotor deficits emerge, most often including loss of sensation and difficulty generating upper extremity movements. Although sensory therapies, such as the application of tendon vibration, have been shown to reduce the sensorimotor deficits after stroke, the underlying sensorimotor mechanisms associated with such improvements are unknown. While sensorimotor networks have been studied extensively, unanswered questions still surround their role in basic control paradigms and how their role changes after stroke. EEG provides a way to probe the high-speed temporal dynamics of sensorimotor networks that other more common imaging modalities lack. Sensorimotor network function was examined in controls during a task designed to differentiate potential mechanisms of arm stabilization and determine to what degree the sensorimotor network is involved. After sensorimotor network function was characterized in controls, we examined the effect of stroke on the sensorimotor network during rest and described the reorganization that occurs. Lastly, we explored tendon vibration as a sensory therapy for stroke survivors and determined if sensorimotor network mechanisms underlie improvements in arm tracking performance due to wrist tendon vibration. We observed cortical activity and connectivity that suggests sensorimotor networks are involved in the control of arm stability, cortical networks reorganize to more asymmetric, local networks after stroke, and tendon vibration normalizes sensorimotor network activity and connectivity during motor control after stroke. This dissertation was among the first studies using EEG to characterize the high-speed temporal dynamics of sensorimotor networks following stroke. This new knowledge has led to a better understanding of how sensorimotor networks function under ordinary circumstances as well as extreme situations such as stroke and revealed previously unknown mechanisms by which tendon vibration improves motor control in stroke survivors, which will lead to better therapeutic approaches