A multilevel framework to measure, model, promote, and enhance the symbiotic cooperation between humans and robotic devices

In the latest decades, the common perception about the role of robotic devices in the modern society dramatically changed. In the early stages of robotics, temporally located in the years of the economic boom, the development of new devices was driven by the industrial need of producing more while reducing production time and costs. The demand was, therefore, for robotic devices capable of substituting the humans in performing simple and repetitive activities. The execution of predefined basic activities in the shortest amount of time, inside carefully engineered and confined environments, was the mission of robotic devices. Beside the results obtained in the industrial sector, a progressive widening of the fields interested in robotics – such as rehabilitation, elderly care, and medicine – led to the current vision of the device role. Indeed, these challenging fields require the robot to be a partner, which works side-by-side with the human. Therefore, the device needs to be capable of actively and efficiently interacting with humans, to provide support and overcome their limits in the execution of shared activities, even in highly unpredictable everyday environments. Highly complex and advanced robots, such as surgical robots, rehabilitation devices, flexible manipulators, and service and companion robots, have been recently introduced into the market; despite their complexity, however, they are still tools to be used to perform, better or faster, very specific tasks. The current open challenge is, therefore, to develop a new generation of symbiotically cooperative robotic partners, adding to the devices the capability to detect, understand, and adapt to the real intentions, capabilities, and needs of the humans. To achieve this goal, a bidirectional information channel shall be built to connect the human and the device. In one direction, the device requires to be informed about the state of its user; in the other direction, the human needs to be informed about the state of the whole interacting system. This work reports the research activities that I conducted during my PhD studies in this research direction. Those activities led to the design, development, and assessment on a real application of an innovative multilevel framework to close the cooperation loop between a human and a robotic device, thus promoting and enhancing their symbiotic interaction. Three main levels have been identified as core elements to close this loop: the measure level, the model level, and the extract/synthesize level. The former aims at collecting experimental measures from the whole interacting system; the second aims at estimating and predicting its dynamic behavior; the last aims at providing quantitative information to both the human and the device about their performances and about how to modify their behavior to improve their interaction symbiosis. Within the measure level, the focus has been concentrated on investigating, critically comparing, and selecting the most suitable and advanced technologies to measure kinematics and dynamics quantities in a portable and minimally intrusive way. Particular attention has been paid to new emerging technologies; moreover, useful protocols and pipelines already recognized as de-facto in other fields have been successfully adapted to fit the needs of the man-machine interaction context. Finally, the design of a new sensor has been started to overcome the lack of tools capable of effectively measuring human-device interaction forces. To implement the model level, a common platform to perform integrated multilevel simulations – i.e. simulations where the device and the human are considered together as interacting entities – has been selected and extensively validated. Furthermore, critical aspects characterizing the modeling of the device, the human, and their interactions have been studied and possible solutions have been proposed. For example, modeling the mechanics and the control within the selected software platform allowed accurate estimations of their behavior. To estimate human behavior, new methodologies and approaches based on anatomical neuromusculoskeletal models have been developed, validated, and released as open-source tools for the community, to allow accurate estimates of both kinematics and dynamics at run-time – i.e. at the same time that the movements are performed. An inverse kinematics approach has been developed and validated to estimate human joint angles from the orientation measurements provided by wearable inertial systems. Additionally, a state of the art neuromusculoskeletal modeling toolbox has been improved and interfaced with the other tools of the multilevel framework, to accurately predict human muscle forces, joint moments, and muscle and joint stiffness from electromyographic and kinematic measures. To estimate and predict the interactions, contact models, parameters optimization procedures, and high-level cooperation strategies have been investigated, developed, and applied. Within the extract/synthesize level, the information provided by the other levels has been combined together to develop informative feedbacks for both the device and the human. In one direction, the device has been provided with control signals defining how to adjust the provided support to comply with the task goals and with the human current capabilities and needs. In the other direction, quantitative feedbacks have been developed to inform the human about task execution performances, task targets, and support provided by the device. This information has been provided to the user as visual feedbacks designed to be both exhaustively informative and minimally distractive, to prevent possible loss of focus. Moreover, additional feedbacks have been devised to help external observers – therapists in the rehabilitation contexts or task planners and ergonomists in the industrial field – in the design and refinement of effective personalized tasks and long-term goals. The integration of all the hardware and software tools of each level in a modular, flexible, and reliable software framework, based on a well known robotic middleware, has been fundamental to handle the communication and information exchange processes. The developed general framework has been finally specialized to face the specific needs of robotic-aided gait rehabilitation. In this context, indeed, the final aim of promoting the symbiotic cooperation is translatable in maximizing treatment effectiveness for the patients by actively supporting their changing needs and capabilities while keeping them engaged during the whole rehabilitation process. The proposed multilevel framework specialization has been successfully used, as valuable answer to those needs, within the context of the Biomot European project. It has been, indeed, fundamental to face the challenges of closing the informative loop between the user and the device, and providing valuable quantitative information to the external observers. Within this research project, we developed an innovative compliant wearable exoskeleton prototype for gait rehabilitation capable of adjusting, at run-time, the provided support according to different cooperation strategies and to user needs and capabilities. At the same time, the wearer is also engaged in the rehabilitation process by intuitive visual feedbacks about his performances in the achievement of the rehabilitation targets and about the exoskeleton support. Both researchers and clinical experts evaluating the final rehabilitation application of the multilevel framework provided enthusiastic feedbacks about the proposed solutions and the obtained results. To conclude, the modular and generic multilevel framework developed in this thesis has the potential to push forward the current state of the art in the applications where a symbiotic cooperation between robotic devices and humans is required. Indeed, it effectively endorses the development of a new generation of robotic devices capable to perform challenging cooperative tasks in highly unpredictable environments while complying with the current needs, intentions, and capabilities of the human

An INNOVATIVE USE of TECHNOLOGY and ASSOCIATIVE LEARNING to ASSESS PRONE MOTOR LEARNING and DESIGN INTERVENTIONS to ENHANCE MOTOR DEVELOPMENT in INFANTS

Since the introduction of the American Academy of Pediatrics Back to Sleep Campaign infants have not met the recommendation to “incorporate supervised, awake “prone play” in their infant’s daily routine to support motor development and minimize the risk of plagiocephaly”. Interventions are needed to increase infants’ tolerance for prone position and prone playtime to reduce the risk of plagiocephaly and motor delays. Associative learning is the ability to understand causal relationship between events. Operant conditioning is a form of associative learning that occurs by associating a behavior with positive or negative consequences. Operant conditions has been utilized to encourage behaviors such as kicking, reaching and sucking in infants by associating these behaviors with positive reinforcement. This dissertation is a compilation of three papers that each represent a study used to investigate a potential play based interventions to encourage prone motor skills in infants. The first paper describes a series of experiment used to develop the Prone Play Activity Center (PPAC) and experimental protocols used in the other studies. The purpose of the second study was to determine the feasibility of a clinical trial comparing usual care (low tech) to a high-tech intervention based on the principles of operant conditioning to increase tolerance for prone and improve prone motor skills. Ten infants participated in the study where parents of infants in the high tech intervention group (n=5) used the PPAC for 3 weeks to practice prone play. Findings from this study suggested the proposed intervention is feasible with some modifications for a future large-scale clinical trial. The purpose of the third study evaluated the ability of 3-6 months old infants to demonstrate AL in prone and remember the association learned a day later. Findings from this study suggested that a majority of infants demonstrated AL in prone with poor retention of the association, 24 hours later. Taken together these 3 papers provide preliminary evidence that a clinical trial of an intervention is feasible and that associative learning could be used to reinforce specific prone motor behaviors in the majority of infants