Joint Trajectory Generation and High-level Control for Patient-tailored Robotic Gait Rehabilitation

This dissertation presents a group of novel methods for robot-based gait rehabilitation which were developed aiming to offer more individualized therapies based on the specific condition of each patient, as well as to improve the overall rehabilitation experience for both patient and therapist. A novel methodology for gait pattern generation is proposed, which offers estimated hip and knee joint trajectories corresponding to healthy walking, and allows the therapist to graphically adapt the reference trajectories in order to fit better the patient's needs and disabilities. Additionally, the motion controllers for the hip and knee joints, mobile platform, and pelvic mechanism of an over-ground gait rehabilitation robotic system are also presented, as well as some proposed methods for assist as needed therapy. Two robot-patient synchronization approaches are also included in this work, together with a novel algorithm for online hip trajectory adaptation developed to reduce obstructive forces applied to the patient during therapy with compliant robotic systems. Finally, a prototype graphical user interface for the therapist is also presented

A novel approach to user controlled ambulation of lower extremity exoskeletons using admittance control paradigm

The robotic lower extremity exoskeletons address the ambulatory problems confronting individuals with paraplegia. Paraplegia due to spinal cord injury (SCI) can cause motor deficit to the lower extremities leading to inability to walk. Though wheelchairs provide mobility to the user, they do not provide support to all activities of everyday living to individuals with paraplegia. Current research is addressing the issue of ambulation through the use of wearable exoskeletons that are pre-programmed. There are currently four exoskeletons in the U.S. market: Ekso, Rewalk, REX and Indego. All of the currently available exoskeletons have 2 active Degrees of Freedom (DOF) except for REX which has 5 active DOF. All of them have pre-programmed gait giving the user the ability to initiate a gait but not the ability to control the stride amplitude (height), stride frequency or stride length, and hence restricting users’ ability to navigate across different surfaces and obstacles that are commonly encountered in the community. Most current exoskeletons do not have motors for abduction or adduction to provide users with the option for movement in coronal plane, hence restricting user’s ability to effectively use the exoskeletons. These limitations of currently available pre-programmed exoskeleton models are sought to be overcome by an intuitive, real time user-controlled control mechanism employing admittance control by using hand-trajectory as a surrogate for foot trajectory. Preliminary study included subjects controlling the trajectory of the foot in a virtual environment using their contralateral hand. The study proved that hands could produce trajectories similar to human foot trajectories when provided with haptic and visual feedback. A 10 DOF 1/2 scale biped robot was built to test the control paradigm. The robot has 5 DOF on each leg with 2 DOF at the hip to provide flexion/extension and abduction/adduction, 1 DOF at the knee to provide flexion and 2 DOF at the ankle to provide flexion/extension and inversion/eversion. The control mechanism translates the trajectory of each hand into the trajectory of the ipsilateral foot in real time, thus providing the user with the ability to control each leg in both sagittal and coronal planes using the admittance control paradigm. The efficiency of the control mechanism was evaluated in a study using healthy subjects controlling the robot on a treadmill. A trekking pole was attached to each foot of the biped. The subjects controlled the trajectory of the foot of the biped by applying small forces in the direction of the required movement to the trekking pole through a force sensor. The algorithm converted the forces to Cartesian position of the foot in real time using admittance control; the Cartesian position was converted to joint angles of the hip and knee using inverse kinematics. The kinematics, synchrony and smoothness of the trajectory produced by the biped robot was evaluated at different speeds, with and without obstacles, and compared with typical walking by human subjects on the treadmill. Further, the cognitive load required to control the biped on the treadmill was evaluated and the effect of speed and obstacles with cognitive load on the kinematics, synchrony and smoothness was analyzed

분산된 로터로 구동되는 비행 스켈레톤 시스템의 디자인 상태추정 및 제어

학위논문(박사)--서울대학교 대학원 :공과대학 기계항공공학부,2020. 2. 이동준.In this thesis, we present key theoretical components for realizing flying aerial skeleton system called LASDRA (large-size aerial skeleton with distributed rotor actuation). Aerial skeletons are articulated aerial robots actuated by distributed rotors including both ground connected type and flying type. These systems have recently attracted interest and are being actively researched in several research groups, with the expectation of applying those for aerial manipulation in distant/narrow places, or for the performance with entertaining purpose such as drone shows. Among the aerial skeleton systems, LASDRA system, proposed by our group has some significant advantages over the other skeleton systems that it is capable of free SE(3) motion by omni-directional wrench generation of each link, and also the system can be operated with wide range of configuration because of the 3DOF (degrees of freedom) inter-link rotation enabled by cable connection among the link modules. To realize this LASDRA system, following three components are crucial: 1) a link module that can produce omni-directional force and torque and enough feasible wrench space; 2) pose and posture estimation algorithm for an articulated system with high degrees of freedom; and 3) a motion generation framework that can provide seemingly natural motion while being able to generate desired motion (e.g., linear and angular velocity) for the entire body. The main contributions of this thesis is theoretically developing these three components, and verifying these through outdoor flight experiment with a real LASDRA system. First of all, a link module for the LASDRA system is designed with proposed constrained optimization problem, maximizing the guaranteed feasible force and torque for any direction while also incorporating some constraints (e.g., avoiding inter-rotor air-flow interference) to directly obtain feasible solution. Also, an issue of ESC-induced (electronic speed control) singularity is first introduced in the literature which is inevitably caused by bi-directional thrust generation with sensorless actuators, and handled with a novel control allocation called selective mapping. Then for the state estimation of the entire LASDRA system, constrained Kalman filter based estimation algorithm is proposed that can provide estimation result satisfying kinematic constraint of the system, also along with a semi-distributed version of the algorithm to endow with system scalability. Lastly, CPG-based motion generation framework is presented that can generate natural biomimetic motion, and by exploiting the inverse CPG model obtained with machine learning method, it becomes possible to generate certain desired motion while still making CPG generated natural motion.본 논문에서는 비행 스켈레톤 시스템 LASDRA (large-size aerial skeleton with distributed rotor actuation) 의 구현을 위해 요구되는 핵심 기법들을 제안하며, 이를 실제 LASDRA 시스템의 실외 비행을 통해 검증한다. 제안된 기법은 1) 전방향으로 힘과 토크를 낼 수 있고 충분한 가용 렌치공간을 가진 링크 모듈, 2) 높은 자유도의 다관절구조 시스템을 위한 위치 및 자세 추정 알고리즘, 3) 자연스러운 움직임을 내는 동시에 전체 시스템이 속도, 각속도 등 원하는 움직임을 내도록 할 수 있는 모션 생성 프레임워크로 구성된다. 본 논문에서는 우선 링크 모듈의 디자인을 위해 전방향으로 보장되는 힘과 토크의 크기를 최대화하는 구속 최적화를 사용하고, 실제 적용가능한 해를 얻기 위해 몇가지 구속조건(로터 간 공기 흐름 간섭의 회피 등)을 고려한다. 또한 센서가 없는 액츄에이터로 양방향 추력을 내는 것에서 야기되는 ESC 유발 특이점 (ESC-induced singularity) 이라는 문제를 처음으로 소개하고, 이를 해결하기 위해 선택적 맵핑 (selective mapping) 이라는 기법을 제시한다. 전체 LASDRA 시스템의 상태추정을 위해 시스템의 기구학적 구속조건을 만족하는 결과를 얻을 수 있도록 구속 칼만 필터 기반의 상태추정 기법을 제시하고, 시스템 확장성을 고려하여 반 분산 (semi-distributed) 개념의 알고리즘을 함께 제시한다. 마지막으로 본 논문에서는 자연스러운 움직임의 생성을 위하여 CPG 기반의 모션 생성 프레임워크를 제안하며, 기계 학습 방법을 통해 CPG 역연산 모델을 얻음으로써 전체 시스템이 원하는 움직임을 낼 수 있도록 한다.1 Introduction 1 1.1 Motivation and Background 1 1.2 Research Problems and Approach 3 1.3 Preview of Contributions 5 2 Omni-Directional Aerial Robot 7 2.1 Introduction 7 2.2 Mechanical Design 12 2.2.1 Design Description 12 2.2.2 Wrench-Maximizing Design Optimization 13 2.3 System Modeling and Control Design 20 2.3.1 System Modeling 20 2.3.2 Pose Trajectory Tracking Control 22 2.3.3 Hybrid Pose/Wrench Control 22 2.3.4 PSPM-Based Teleoperation 24 2.4 Control Allocation with Selective Mapping 27 2.4.1 Infinity-Norm Minimization 27 2.4.2 ESC-Induced Singularity and Selective Mapping 29 2.5 Experiment 38 2.5.1 System Setup 38 2.5.2 Experiment Results 41 2.6 Conclusion 49 3 Pose and Posture Estimation of an Aerial Skeleton System 51 3.1 Introduction 51 3.2 Preliminary 53 3.3 Pose and Posture Estimation 55 3.3.1 Estimation Algorithm via SCKF 55 3.3.2 Semi-Distributed Version of Algorithm 59 3.4 Simulation 62 3.5 Experiment 65 3.5.1 System Setup 65 3.5.2 Experiment of SCKF-Based Estimation Algorithm 66 3.6 Conclusion 69 4 CPG-Based Motion Generation 71 4.1 Introduction 71 4.2 Description of Entire Framework 75 4.2.1 LASDRA System 75 4.2.2 Snake-Like Robot & Pivotboard 77 4.3 CPG Model 79 4.3.1 LASDRA System 79 4.3.2 Snake-Like Robot 80 4.3.3 Pivotboard 83 4.4 Target Pose Calculation with Expected Physics 84 4.5 Inverse Model Learning 86 4.5.1 LASDRA System 86 4.5.2 Snake-Like Robot 89 4.5.3 Pivotboard 90 4.6 CPG Parameter Adaptation 93 4.7 Simulation 94 4.7.1 LASDRA System 94 4.7.2 Snake-Like Robot & Pivotboard 97 4.8 Conclusion 101 5 Outdoor Flight Experiment of the F-LASDRA System 103 5.1 System Setup 103 5.2 Experiment Results 104 6 Conclusion 111 6.1 Summary 111 6.2 Future Works 112Docto

Human adaptive haptic sensing

How do humans physically interact with the environment or with other humans? It is well known that the nervous system can modify the body’s stiffness by selectively cocontracting muscles to shape the mechanical interaction with the environment, but how this influences haptic perception is not known. This thesis examines whether humans can adapt muscles’ activation to influence their perception of the physical interaction with the environment. This question is investigated by conducting behavioural experiments using dedicated robotic interfaces to study sensorimotor interactions in the presence of haptic and visual perturbations. Hypotheses about the underlying mechanism are then tested through mathematical modelling and simulations. Chapter 1 reviews related frameworks and introduces the most relevant questions addressed in this work. Chapter 2 then shows that the central nervous system (CNS) can voluntarily adapt muscle cocontraction to increase haptic sensitivity. In an experiment, participants tracked a randomly moving target with visual noise while being physically guided by a virtual elastic band, where the band’s stiffness was controlled by their muscle coactivation. The results show that participants learned to increase cocontraction with visual noise and decrease it when the guidance is incongruent with the visual target. The adaptation law governing the regulation of the body’s stiffness by the CNS is then derived through computational modelling. This model is designed to maximise visuo-haptic information while minimising metabolic cost, thus trading off sensory information with energy. Further, it is shown in Chapter 3 that when the subjects are coupled via a tuneable connection to a robotic guidance designed to hinder their tracking through perturbations at the turning points (where participants physiologically increase cocontraction), they adapted cocontraction to reduce the impact of perturbations on performance. These results highlight the CNS ability to modify the muscle activation patterns to improve performance with minimal effort. Chapter 4 tests the robustness of human adaptive haptic sensing introduced in the previous chapters for human-human physical interaction. For example, in tango dancing physical contact provides haptic information of the partner’s action required to coordinate the movements. During such physical interactions, should one keep the arms compliant so that the partner can correct the motion, or should one stiffen them to better keep along the planned movement? Using a tracking task in which a dyad is coupled via a rigid connection, subjects readily adapted the compliance of their limb depending on both the accuracy of the partner’s and their own movement. The same computational model introduced in Chapter 2 could explain these results and predict the experimentally observed cocontraction adaptation. This suggests that the minimisation of prediction error and energy is a general principle also holding in interpersonal interactions. Altogether, these findings shed light on how humans can adapt haptic sensing by changing body properties, and propose a novel framework to interpret visuo-haptic perception for interaction with the environment and other humans.Open Acces

Passive Motion Paradigm: An Alternative to Optimal Control

In the last years, optimal control theory (OCT) has emerged as the leading approach for investigating neural control of movement and motor cognition for two complementary research lines: behavioral neuroscience and humanoid robotics. In both cases, there are general problems that need to be addressed, such as the “degrees of freedom (DoFs) problem,” the common core of production, observation, reasoning, and learning of “actions.” OCT, directly derived from engineering design techniques of control systems quantifies task goals as “cost functions” and uses the sophisticated formal tools of optimal control to obtain desired behavior (and predictions). We propose an alternative “softer” approach passive motion paradigm (PMP) that we believe is closer to the biomechanics and cybernetics of action. The basic idea is that actions (overt as well as covert) are the consequences of an internal simulation process that “animates” the body schema with the attractor dynamics of force fields induced by the goal and task-specific constraints. This internal simulation offers the brain a way to dynamically link motor redundancy with task-oriented constraints “at runtime,” hence solving the “DoFs problem” without explicit kinematic inversion and cost function computation. We argue that the function of such computational machinery is not only restricted to shaping motor output during action execution but also to provide the self with information on the feasibility, consequence, understanding and meaning of “potential actions.” In this sense, taking into account recent developments in neuroscience (motor imagery, simulation theory of covert actions, mirror neuron system) and in embodied robotics, PMP offers a novel framework for understanding motor cognition that goes beyond the engineering control paradigm provided by OCT. Therefore, the paper is at the same time a review of the PMP rationale, as a computational theory, and a perspective presentation of how to develop it for designing better cognitive architectures

Steering control for haptic feedback and active safety functions

Steering feedback is an important element that defines driver–vehicle interaction. It strongly affects driving performance and is primarily dependent on the steering actuator\u27s control strategy. Typically, the control method is open loop, that is without any reference tracking; and its drawbacks are hardware dependent steering feedback response and attenuated driver–environment transparency. This thesis investigates a closed-loop control method for electric power assisted steering and steer-by-wire systems. The advantages of this method, compared to open loop, are better hardware impedance compensation, system independent response, explicit transparency control and direct interface to active safety functions.The closed-loop architecture, outlined in this thesis, includes a reference model, a feedback controller and a disturbance observer. The feedback controller forms the inner loop and it ensures: reference tracking, hardware impedance compensation and robustness against the coupling uncertainties. Two different causalities are studied: torque and position control. The two are objectively compared from the perspective of (uncoupled and coupled) stability, tracking performance, robustness, and transparency.The reference model forms the outer loop and defines a torque or position reference variable, depending on the causality. Different haptic feedback functions are implemented to control the following parameters: inertia, damping, Coulomb friction and transparency. Transparency control in this application is particularly novel, which is sequentially achieved. For non-transparent steering feedback, an environment model is developed such that the reference variable is a function of virtual dynamics. Consequently, the driver–steering interaction is independent from the actual environment. Whereas, for the driver–environment transparency, the environment interaction is estimated using an observer; and then the estimated signal is fed back to the reference model. Furthermore, an optimization-based transparency algorithm is proposed. This renders the closed-loop system transparent in case of environmental uncertainty, even if the initial condition is non-transparent.The steering related active safety functions can be directly realized using the closed-loop steering feedback controller. This implies, but is not limited to, an angle overlay from the vehicle motion control functions and a torque overlay from the haptic support functions.Throughout the thesis, both experimental and the theoretical findings are corroborated. This includes a real-time implementation of the torque and position control strategies. In general, it can be concluded that position control lacks performance and robustness due to high and/or varying system inertia. Though the problem is somewhat mitigated by a robust H-infinity controller, the high frequency haptic performance remains compromised. Whereas, the required objectives are simultaneously achieved using a torque controller

Stability and Performance Improvement in Haptic Human-Robot Interaction

The goal of this research is to develop theories, methods, and tools to understand the mechanisms of neuromotor adaptation in human-robot physical interaction, in order to improve the stability and performance of the interaction. Human power-assisting systems (e.g., powered lifting devices that aid human operators in manipulating heavy or bulky loads) require physical contact between the operator and machine, creating a coupled dynamic system. This dynamic coupling has been shown to introduce inherent instabilities and performance degradation due to a change in human stiffness; when instability is encountered, a human operator often attempts to control the oscillation by stiffening their arm, which leads to a stiffer system with more instability. Robot co-worker controllers must account for this issue. In this work we set out to 1) understand the association between neuromuscular adaptations and system performance limits, 2) develop probabilistic methods to classify and predict the transition of operator’s cognitive and physical states from physiological measures, and 3) integrate this knowledge into a structure of shared human-robot control. We developed a model of the human operator endpoint stiffness, characterized at the musculoskeletal level, that can account for deliberate stiffness increase at the endpoint through the incorporation of muscle coactivation. We also developed a switching admittance control approach which can account for changes in the operator’s muscle coactivation and is able to generate cognitive states in an unsupervised manner, given a relevant training dataset. Finally, a novel variable admittance control approach, which significantly reduces grasp contact instability commonly encountered in fixed admittance control settings, was developed, analytically derived, and provides solutions for both constant mass and variable mass parameter cases.Ph.D

Contact force regulation in physical human-machine interaction based on model predictive control

With increasing attention to physical human-machine interaction (pHMI), new control methods involving contact force regulation in collaborative and coexistence scenarios have spread in recent years. Thanks to its internal robustness, high dynamic performance, and capabilities to avoid constraint violations, a Model Predictive Control (MPC) action can pose a viable solution to manage the uncertainties involved in those applications. This paper uses an MPC-driven control method that aims to apply a well-defined and tunable force impulse on a human subject. After describing a general control design suitable to achieve this goal, a practical implementation of such a logic, based on an MPC controller, is shown. In particular, the physical interaction considered is the one occurring between the body of a patient and an external perturbation device in a dynamic posturography trial. The device prototype is presented in both its hardware architecture and software design. The MPC-based main control parameters are thus tuned inside hardware-in-the-loop and human-in-the-loop environments to get optimal behaviors. Finally, the device performance is analyzed to assess the MPC algorithm’s accuracy, repeatability, flexibility, and robustness concerning the several uncertainties due to the specific pHMI environment considered

Haptic dokunma hisli ve kuvvet geri beslemeli arayüz sistem tasarımı

TÜBİTAK MAG01.07.2008“Haptic” cihaz, kullanıcı ile bilgisayar arasında üç boyutlu veri transferi sağlayan bir cihazdır. Bu cihaz kullanıcıya sanal ortamdaki bir nesneyi görmenin yanında, bu sanal nesneye dokunma olanağı da sağlar. Aynı zamanda bu cihaz ile gerçek bir nesneden veriler alınarak bu verilerin sanal ortamda kullanımı da gerçekleştirilebilir. Günümüzde sanal gerçeklik teknolojisinin gelişimine paralel olarak “haptic” cihazlarının kullanımı, sanayi, eğitim ve tıp alanlarında gittikçe artmaktadır. Bu teknolojinin değişik uygulama alanları, farklı bilim dallarına ait disiplinlerarası çalışmaları gerektirdiğinden, kendi içinde özgün çalışma konularını da yaratmaktadır (“haptic” arayüz tasarımı, serbest modelleme, sanal ortamda ameliyat eğitimi, vb.). Medikal uygulamalarda önemli bir potansiyele sahip bu teknoloji ile daha önceleri el işçiliği ile yapılmak zorunda kalınan karmaşık modifikasyonlar gerçekleştirilebilmektedir (beyin cerrahisinde hata kabul etmeyen, ustalık ve ameliyat öncesi uzun uğraşlar gerektiren operasyonlar). Bu örnek “haptic” cihazlar ile dijital ortamda gerçekleştirilebilecek olan uygulamalardan sadece bir tanesidir. Bu proje kapsamında, bahsedilen uygulama alanlarına yönelik 7 serbestlik dereceli bir “haptic” cihaz tasarlanıp üretilmiş ve cihazın kullanılabilirliğini göstermek için bir arayüz tasarlamıştır. Günümüze kadar tasarlanmış olan bütün “haptic” cihazlar maksimum 6 serbestlik derecesine sahiptirler. Tasarlanan 7 serbestlik dereceli “haptic” cihaz, bugüne kadar üretilmiş aynı uzuv uzunlukları ve eklem değişkenlerine sahip diğer “haptic” cihazlar ile karşılaştırıldığında yaklaşık %20 daha büyük bir çalışma hacmine ve daha esnek bir çalışma kabiliyetine sahiptir. Bu çalışma, Dünya’daki “haptic” cihazların geliştirilmesi ve Türkiye’deki “haptic” cihazların ve uygulama alanlarının yaygınlaşması açısından son derece önemlidir.Haptic devices are used to provide multi-modal data transfer between haptic users and computers in virtual reality applications. They enable humans to take force and tactile feedback from any virtual or remote objects. Haptic devices also facilitate the use of data collected from a real object in the virtual environment. Usage of the haptic devices increase more and more in industrial, educational and medical applications in parallel with development of virtual reality technology. As virtual reality technology requires interdisciplinary study with related to its application areas, it creates a lot of different specific working areas (Haptic interface design, freeform model, surgical operations in virtual environment etc.). Especially, some complex modifications which require hand-working can be performed with the system having great potential in medical applications (Brain surgery without error and operations which require great skill etc.). This is only one of the implementations of haptic devices in digital environment. Aim of this project is to design and manufacture a novel haptic device which serves the mentioned application areas and to improve an interface to implement the device. Most of the haptic devices in literature have maximum 6 DOF. The 7 DOF haptic device designed has about 20% extra working space and more flexible working capability compared to the other haptic devices with the similar link lengths and joint limitations. This project is important in terms of the development of haptic devices in the world as well as spreading of haptic devices and its applications in Turkey

Rehabilitation Engineering

Population ageing has major consequences and implications in all areas of our daily life as well as other important aspects, such as economic growth, savings, investment and consumption, labour markets, pensions, property and care from one generation to another. Additionally, health and related care, family composition and life-style, housing and migration are also affected. Given the rapid increase in the aging of the population and the further increase that is expected in the coming years, an important problem that has to be faced is the corresponding increase in chronic illness, disabilities, and loss of functional independence endemic to the elderly (WHO 2008). For this reason, novel methods of rehabilitation and care management are urgently needed. This book covers many rehabilitation support systems and robots developed for upper limbs, lower limbs as well as visually impaired condition. Other than upper limbs, the lower limb research works are also discussed like motorized foot rest for electric powered wheelchair and standing assistance device