Adaptive physical human-robot interaction (PHRI) with a robotic nursing assistant.

Recently, more and more robots are being investigated for future applications in health-care. For instance, in nursing assistance, seamless Human-Robot Interaction (HRI) is very important for sharing workspaces and workloads between medical staff, patients, and robots. In this thesis we introduce a novel robot - the Adaptive Robot Nursing Assistant (ARNA) and its underlying components. ARNA has been designed specifically to assist nurses with day-to-day tasks such as walking patients, pick-and-place item retrieval, and routine patient health monitoring. An adaptive HRI in nursing applications creates a positive user experience, increase nurse productivity and task completion rates, as reported by experimentation with human subjects. ARNA has been designed to include interface devices such as tablets, force sensors, pressure-sensitive robot skins, LIDAR and RGBD camera. These interfaces are combined with adaptive controllers and estimators within a proposed framework that contains multiple innovations. A research study was conducted on methods of deploying an ideal HumanMachine Interface (HMI), in this case a tablet-based interface. Initial study points to the fact that a traded control level of autonomy is ideal for tele-operating ARNA by a patient. The proposed method of using the HMI devices makes the performance of a robot similar for both skilled and un-skilled workers. A neuro-adaptive controller (NAC), which contains several neural-networks to estimate and compensate for system non-linearities, was implemented on the ARNA robot. By linearizing the system, a cross-over usability condition is met through which humans find it more intuitive to learn to use the robot in any location of its workspace, A novel Base-Sensor Assisted Physical Interaction (BAPI) controller is introduced in this thesis, which utilizes a force-torque sensor at the base of the ARNA robot manipulator to detect full body collisions, and make interaction safer. Finally, a human-intent estimator (HIE) is proposed to estimate human intent while the robot and user are physically collaborating during certain tasks such as adaptive walking. A NAC with HIE module was validated on a PR2 robot through user studies. Its implementation on the ARNA robot platform can be easily accomplished as the controller is model-free and can learn robot dynamics online. A new framework, Directive Observer and Lead Assistant (DOLA), is proposed for ARNA which enables the user to interact with the robot in two modes: physically, by direct push-guiding, and remotely, through a tablet interface. In both cases, the human is being “observed” by the robot, then guided and/or advised during interaction. If the user has trouble completing the given tasks, the robot adapts their repertoire to lead users toward completing goals. The proposed framework incorporates interface devices as well as adaptive control systems in order to facilitate a higher performance interaction between the user and the robot than was previously possible. The ARNA robot was deployed and tested in a hospital environment at the School of Nursing of the University of Louisville. The user-experience tests were conducted with the help of healthcare professionals where several metrics including completion time, rate and level of user satisfaction were collected to shed light on the performance of various components of the proposed framework. The results indicate an overall positive response towards the use of such assistive robot in the healthcare environment. The analysis of these gathered data is included in this document. To summarize, this research study makes the following contributions: Conducting user experience studies with the ARNA robot in patient sitter and walker scenarios to evaluate both physical and non-physical human-machine interfaces. Evaluation and Validation of Human Intent Estimator (HIE) and Neuro-Adaptive Controller (NAC). Proposing the novel Base-Sensor Assisted Physical Interaction (BAPI) controller. Building simulation models for packaged tactile sensors and validating the models with experimental data. Description of Directive Observer and Lead Assistance (DOLA) framework for ARNA using adaptive interfaces

TIP trajectory tracking of flexible-joint manipulators

In most robot applications, the control of the manipulator’s end-effector along a specified desired trajectory is the main concern. In these applications, the end-effector (tip) of the manipulator is required to follow a given trajectory. Several methods have been so far proposed for the motion control of robot manipulators. However, most of these control methods ignore either joint friction or joint elasticity which can be caused by the transmission systems (e.g. belts and gearboxes). This study aims at development of a comprehensive control strategy for the tip-trajectory tracking of flexible-joint robot manipulators. While the proposed control strategy takes into account the effect of the friction and the elasticity in the joints, it also provides a highly accurate motion for the manipulator’s end-effector. During this study several approaches have been developed, implemented and verified experimentally/numerically for the tip trajectory tracking of robot manipulators. To compensate for the elasticity of the joints two methods have been proposed; they are a composite controller whose design is based on the singular perturbation theory and integral manifold concept, and a swarm controller which is a novel biologically-inspired controller and its concept is inspired by the movement of real biological systems such as flocks of birds and schools of fishes. To compensate for the friction in the joints two new approaches have been also introduced. They are a composite compensation strategy which consists of the non-linear dynamic LuGre model and a Proportional-Derivative (PD) compensator, and a novel friction compensation method whose design is based on the Work-Energy principle. Each of these proposed controllers has some advantages and drawbacks, and hence, depending on the application of the robot manipulator, they can be employed. For instance, the Work-Energy method has a simpler form than the LuGre-PD compensator and can be easily implemented in industrial applications, yet it provides less accuracy in friction compensation. In addition to design and develop new controllers for flexible-joint manipulators, another contribution of this work lays in the experimental verification of the proposed control strategies. For this purpose, experimental setups of a two-rigid-link flexible-joint and a single-rigid-link flexible-joint manipulators have been employed. The proposed controllers have been experimentally tested for different trajectories, velocities and several flexibilities of the joints. This ensures that the controllers are able to perform effectively at different trajectories and speeds. Besides developing control strategies for the flexible-joint manipulators, dynamic modeling and vibration suppression of flexible-link manipulators are other parts of this study. To derive dynamic equations for the flexible-link flexible-joint manipulators, the Lagrange method is used. The simulation results from Lagrange method are then confirmed by the finite element analysis (FEA) for different trajectories. To suppress the vibration of flexible manipulators during the manoeuvre, a collocated sensor-actuator is utilized, and a proportional control method is employed to adjust the voltage applied to the piezoelectric actuator. Based on the controllability of the states and using FEA, the optimum location of the piezoelectric along the manipulator is found. The effect of the controller’s gain and the delay between the input and output of the controller are also analyzed through a stability analysis

Automatic testing of organic strain gauge tactile sensors.

Human-Robot Interaction is a developing field of science, that is posed to augment everything we do in life. Skin sensors that can detect touch, temperature, distance, and other physical interaction parameters at the human-robot interface are very important to enhancing the collaboration between humans and machines. As such, these sensors must be efficiently tested and characterized to give accurate feedback from the sensor to the robot. The objective of this work is to create a diversified software testing suite that removes as much human intervention as possible. The tests and methodology discussed here provide multiple realistic scenarios that the sensors undergo during repeated experiments. This capability allows for easy repeatable tests without interference from the test engineer, increasing productivity and efficiency. The foundation of this work has two main pieces: force feedback control to drive the test actuator, and computer vision functionality to guide alignment of the test actuator and sensors arranged in a 2D array. The software running automated tests was also made compatible with the testbench hardware via LabVIEW programs. The program uses set coordinates to complete a raster scan of the SkinCell that locates individual sensors. Tests are then applied at each sensor using a force controller. The force feedback control system uses a Proportional Integral Derivative (PID) controller that reads in force readings from a load cell to correct itself or follow a desired trajectory. The motion of the force actuator was compared to that of the projected trajectory to test for accuracy and time delay. The proposed motor control allows for dynamic force to stimulate the sensors giving a more realistic test then a stable force. A top facing camera was introduced to take in the starting position of a SkinCell before testing. Then, computer vision algorithms were proposed to extract the location of the cell and individual sensors before generating a coordinate plane. This allows for the engineer to skip over manual alignment of the sensors, saving more time and providing more accurate destinations. Finally, the testbench was applied to numerous sensors developed by the research team at the Louisville Automation and Robotics Research Institute (LARRI) for testing and data analysis. Force loads are applied to the individual sensors while recording response. Afterwards, postprocessing of the data was conducted to compare responses within the SkinCell as well as to other sensors manufactured using different methods

Soft pneumatic devices for blood circulation improvement

The research activity I am presenting in this thesis lies within the framework of a cooperation between the University of Cagliari (Applied Mechanics and Robotics lab, headed by professor Andrea Manuello Bertetto, and the research group of physicians referencing to professor Alberto Concu at the Laboratory of Sports Physiology, Department of Medical Sciences), and the Polytechnic of Turin (professor Carlo Ferraresi and his equipe at the Group of Automation and Robotics, Department of Mechanical and Aerospace Engineering) This research was also funded by the Italian Ministry of Research (MIUR – PRIN 2009). My activity has been mainly carried on at the Department of Mechanics, Robotics lab under the supervision of prof. Manuello; I have also spent one year at the Control Lab of the School of Electrical Engineering at Aalto University (Helsinki, Finland). The tests on the patients were taken at the Laboratory of Sports Physiology, Cagliari. I will be describing the design, development and testing of some soft pneumatic flexible devices meant to apply an intermittent massage and to restore blood circulation in lower limbs in order to improve cardiac output and wellness in general. The choice of the actuators, as well as the pneumatic circuits and air distribution system and PLC control patterns will be outlined. The trial run of the devices have been field--‐tested as soon a prototype was ready, so as to tune its features step--‐by--‐ step. I am also giving a characterization of a commercial thin force sensor after briefly reviewing some other type of thin pressure transducer. It has been used to gauge the contact pressure between the actuator and the subject’s skin in order to correlate the level of discomfort to the supply pressure, and to feed this value back to regulate the supply air flow. In order for the massage to be still effective without causing pain or distress or any cutoff to the blood flow, some control objective have been set, consisting in the regulation of the contact force so that it comes to the constant set point smoothly and its value holds constant until unloading occurs. The targets of such mechatronic devices range from paraplegic patients lacking of muscle tone because of their spinal cord damage, to elite endurance athletes needing a circulation booster when resting from practicing after serious injuries leading to bed rest. Encouraging results have been attained for both these two categories, based on the monitored hemodynamic variables

Enhancing brain/neural-machine interfaces for upper limb motor restoration in chronic stroke and cervical spinal cord injury

Operation of assistive exoskeletons based on voluntary control of sensorimotor rhythms (SMR, 8-12 Hz) enables intuitive control of finger or arm movements in severe paralysis after chronic stroke or cervical spinal cord injury (SCI). To improve reliability of such systems outside the laboratory, in particular when brain activity is recorded non-invasively with scalp electroencephalography (EEG), a hybrid EEG/electrooculography (EOG) brain/neural-machine interface (B/NMI) was recently introduced. Besides providing assistance, recent studies indicate that repeated use of such systems can trigger neural recovery. However, important prerequisites have to achieved before broader use in clinical settings or everyday life environments is feasible. Current B/NMI systems predominantly restore hand function, but do not allow simultaneous control of more proximal joints for whole-arm motor coordination as required for most stroke survivors suffering from paralysis in the entire upper limb. Besides paralysis, cognitive impairments including post-stroke fatigue due to the brain lesion reduce the capacity to maintain effortful B/NMI control over a longer period of time. This impedes the applicability in daily life assistance and might even limits the efficacy of neurorehabilitation training. In contrast to stroke survivors, tetraplegics due to cervical SCI lack motor function in both hands. Given that most activities of daily living (ADL) involve bimanual manipulation, e.g., to open the lid of a bottle, bilateral exoskeleton control is required but was not shown yet in tetraplegics. To further enhance B/NMI systems, we first investigated whether B/NMI whole-arm exoskeleton control in hemiplegia after chronic stroke is feasible and safe. In contrast to simple grasping, control of more complex tasks involving the entire upper limb was not feasible with established B/NMIs because high- dimensionality of such multiple joint systems exceeds the bandwidth of these interfaces. Thus, we blended B/NMI control with vision-guidance to receive a semiautonomous whole-arm exoskeleton control. Such setup allowed to divide ADL tasks into a sequence of EEG/EOG-triggered sub-tasks reducing complexity for the user. While, for instance, a drinking task was resolved into EOG-induced reaching, lifting and placing back the cup, grasping and releasing movements were based on intuitive SMR control. Feasibility of such shared vision-guided B/NMI control was assumed when executions were initialized within 3 s (fluent control) and a minimum of 75 % of subtasks were executed within that time (reliable control). We showed feasibility in healthy subjects as well as stroke survivors without report of any side effects documenting safe use. Similarly, feasibility and safety of bilateral B/NMI control after cervical SCI was evaluated. To enable bilateral B/NMI control, established EEG-based grasping and EOG-based releasing or stop commands were complemented with a novel EOG command allowing to switch laterality by performing prolonged horizontal eye movements (>1 s) to the left or to the right. Study results with healthy subjects and tetraplegics document fluent initialization of grasping motions below 3 s as well as safe use as unintended grasping could be stopped before a full motion was conducted. Superiority of novel bilateral control was documented by a higher accuracy of up to 22 % in tetraplegics compared to a bilateral control without prolonged EOG command. Lastly, as reliable B/NMI control is cognitively demanding, e.g., by imagining or attempting the desired movements, we investigated whether heart rate variability (HRV) can be used as biomarker to predict declining control performance, which is often reported in stroke survivors due to their cognitive impairments. Referring to the close brain-heart connection, we showed in healthy subjects that a decline in HRV is specific as well as predictive to a decline in B/NMI control performance within a single training session. The predictive link was revealed by a Granger-causality analysis. In conclusion, we could demonstrate important enhancements in B/NMI control paradigms including complex whole-arm exoskeleton control as well as individual performance monitoring within a training session based on HRV. Both achievements contribute to broaden the use as a standard therapy in stroke neurorehabilitation. Especially the predictive characteristic of HRV paves the way for adaptive B/NMI control paradigms to account for individual differences among impaired stroke survivors. Moreover, we also showed feasibility and safety of a novel implementation for bilateral B/NMI control, which is necessary for reliable operation of two hand-exoskeletons for bimanual ADLs after SCI

Applications of Mathematical Models in Engineering

The most influential research topic in the twenty-first century seems to be mathematics, as it generates innovation in a wide range of research fields. It supports all engineering fields, but also areas such as medicine, healthcare, business, etc. Therefore, the intention of this Special Issue is to deal with mathematical works related to engineering and multidisciplinary problems. Modern developments in theoretical and applied science have widely depended our knowledge of the derivatives and integrals of the fractional order appearing in engineering practices. Therefore, one goal of this Special Issue is to focus on recent achievements and future challenges in the theory and applications of fractional calculus in engineering sciences. The special issue included some original research articles that address significant issues and contribute towards the development of new concepts, methodologies, applications, trends and knowledge in mathematics. Potential topics include, but are not limited to, the following: Fractional mathematical models; Computational methods for the fractional PDEs in engineering; New mathematical approaches, innovations and challenges in biotechnologies and biomedicine; Applied mathematics; Engineering research based on advanced mathematical tools

EVA London 2022: Electronic Visualisation and the Arts

The Electronic Visualisation and the Arts London 2022 Conference (EVA London 2022) is co-sponsored by the Computer Arts Society (CAS) and BCS, the Chartered Institute for IT, of which the CAS is a Specialist Group. Of course, this has been a difficult time for all conferences, with the Covid-19 pandemic. For the first time since 2019, the EVA London 2022 Conference is a physical conference. It is also an online conference, as it was in the previous two years. We continue with publishing the proceedings, both online, with open access via ScienceOpen, and also in our traditional printed form, for the second year in full colour. Over recent decades, the EVA London Conference on Electronic Visualisation and the Arts has established itself as one of the United Kingdom’s most innovative and interdisciplinary conferences. It brings together a wide range of research domains to celebrate a diverse set of interests, with a specialised focus on visualisation. The long and short papers in this volume cover varied topics concerning the arts, visualisations, and IT, including 3D graphics, animation, artificial intelligence, creativity, culture, design, digital art, ethics, heritage, literature, museums, music, philosophy, politics, publishing, social media, and virtual reality, as well as other related interdisciplinary areas. The EVA London 2022 proceedings presents a wide spectrum of papers, demonstrations, Research Workshop contributions, other workshops, and for the seventh year, the EVA London Symposium, in the form of an opening morning session, with three invited contributors. The conference includes a number of other associated evening events including ones organised by the Computer Arts Society, Art in Flux, and EVA International. As in previous years, there are Research Workshop contributions in this volume, aimed at encouraging participation by postgraduate students and early-career artists, accepted either through the peer-review process or directly by the Research Workshop chair. The Research Workshop contributors are offered bursaries to aid participation. In particular, EVA London liaises with Art in Flux, a London-based group of digital artists. The EVA London 2022 proceedings includes long papers and short “poster” papers from international researchers inside and outside academia, from graduate artists, PhD students, industry professionals, established scholars, and senior researchers, who value EVA London for its interdisciplinary community. The conference also features keynote talks. A special feature this year is support for Ukrainian culture after its invasion earlier in the year. This publication has resulted from a selective peer review process, fitting as many excellent submissions as possible into the proceedings. This year, submission numbers were lower than previous years, mostly likely due to the pandemic and a new requirement to submit drafts of long papers for review as well as abstracts. It is still pleasing to have so many good proposals from which to select the papers that have been included. EVA London is part of a larger network of EVA international conferences. EVA events have been held in Athens, Beijing, Berlin, Brussels, California, Cambridge (both UK and USA), Canberra, Copenhagen, Dallas, Delhi, Edinburgh, Florence, Gifu (Japan), Glasgow, Harvard, Jerusalem, Kiev, Laval, London, Madrid, Montreal, Moscow, New York, Paris, Prague, St Petersburg, Thessaloniki, and Warsaw. Further venues for EVA conferences are very much encouraged by the EVA community. As noted earlier, this volume is a record of accepted submissions to EVA London 2022. Associated online presentations are in general recorded and made available online after the conference

EU privacy and data protection law applied to AI: unveiling the legal problems for individuals

AI-powered emotion recognition, typing with thoughts or eavesdropping virtual assistants: three non-fictional examples illustrate how AI may impact society. AI-related products and services increasingly find their way into daily life. Are the EU's fundamental rights to privacy and data protection equipped to protect individuals effectively? In addressing this question, the dissertation concludes that no new legal framework is needed. Instead, adjustments are required. First, the extent of adjustments depends on the AI discipline. There is nothing like 'the AI'. AI covers various concepts, including the disciplines machine learning, natural language processing, computer vision, affective computing and automated reasoning. Second, the extent of adjustments depends on the type of legal problem: legal provisions are violated (type 1), cannot be enforced (type 2) or are not fit for purpose (type 3). Type 2 and 3 problems require either adjustments of current provisions or new judicial interpretations. Two instruments might be helpful for more effective legislation: rebuttable presumptions and reversal of proof. In some cases, the solution is technical, not legal. Research in AI should solve reasoning deficiencies in AI systems and their lack of common sense.Effective Protection of Fundamental Rights in a pluralist worl