Learning Algorithm Design for Human-Robot Skill Transfer

In this research, we develop an intelligent learning scheme for performing human-robot skills transfer. Techniques adopted in the scheme include the Dynamic Movement Prim- itive (DMP) method with Dynamic Time Warping (DTW), Gaussian Mixture Model (G- MM) with Gaussian Mixture Regression (GMR) and the Radical Basis Function Neural Networks (RBFNNs). A series of experiments are conducted on a Baxter robot, a NAO robot and a KUKA iiwa robot to verify the effectiveness of the proposed design.During the design of the intelligent learning scheme, an online tracking system is de- veloped to control the arm and head movement of the NAO robot using a Kinect sensor. The NAO robot is a humanoid robot with 5 degrees of freedom (DOF) for each arm. The joint motions of the operator’s head and arm are captured by a Kinect V2 sensor, and this information is then transferred into the workspace via the forward and inverse kinematics. In addition, to improve the tracking performance, a Kalman filter is further employed to fuse motion signals from the operator sensed by the Kinect V2 sensor and a pair of MYO armbands, so as to teleoperate the Baxter robot. In this regard, a new strategy is developed using the vector approach to accomplish a specific motion capture task. For instance, the arm motion of the operator is captured by a Kinect sensor and programmed through a processing software. Two MYO armbands with embedded inertial measurement units are worn by the operator to aid the robots in detecting and replicating the operator’s arm movements. For this purpose, the armbands help to recognize and calculate the precise velocity of motion of the operator’s arm. Additionally, a neural network based adaptive controller is designed and implemented on the Baxter robot to illustrate the validation forthe teleoperation of the Baxter robot.Subsequently, an enhanced teaching interface has been developed for the robot using DMP and GMR. Motion signals are collected from a human demonstrator via the Kinect v2 sensor, and the data is sent to a remote PC for teleoperating the Baxter robot. At this stage, the DMP is utilized to model and generalize the movements. In order to learn from multiple demonstrations, DTW is used for the preprocessing of the data recorded on the robot platform, and GMM is employed for the evaluation of DMP to generate multiple patterns after the completion of the teaching process. Next, we apply the GMR algorithm to generate a synthesized trajectory to minimize position errors in the three dimensional (3D) space. This approach has been tested by performing tasks on a KUKA iiwa and a Baxter robot, respectively.Finally, an optimized DMP is added to the teaching interface. A character recombination technology based on DMP segmentation that uses verbal command has also been developed and incorporated in a Baxter robot platform. To imitate the recorded motion signals produced by the demonstrator, the operator trains the Baxter robot by physically guiding it to complete the given task. This is repeated five times, and the generated training data set is utilized via the playback system. Subsequently, the DTW is employed to preprocess the experimental data. For modelling and overall movement control, DMP is chosen. The GMM is used to generate multiple patterns after implementing the teaching process. Next, we employ the GMR algorithm to reduce position errors in the 3D space after a synthesized trajectory has been generated. The Baxter robot, remotely controlled by the user datagram protocol (UDP) in a PC, records and reproduces every trajectory. Additionally, Dragon Natural Speaking software is adopted to transcribe the voice data. This proposed approach has been verified by enabling the Baxter robot to perform a writing task of drawing robot has been taught to write only one character

Markerless Kinematics of Pediatric Manual Wheelchair Mobility

Pediatric manual wheelchair users face substantial risk of orthopaedic injury to the upper extremities, particularly the shoulders, during transition to wheelchair use and during growth and development. Propulsion strategy can influence mobility efficiency, activity participation, and quality of life. The current forefront of wheelchair biomechanics research includes translating findings from adult to pediatric populations, improving the quality and efficiency of care under constrained clinical funding, and understanding injury mechanisms and risk factors. Typically, clinicians evaluate wheelchair mobility using marker-based motion capture and instrumentation systems that are precise and accurate but also time-consuming, inconvenient, and expensive for repeated assessments. There is a substantial need for technology that evaluates and improves wheelchair mobility outside of the laboratory to provide better outcomes for wheelchair users, enhancing clinical data. Advancement in this area gives physical therapists better tools and the supporting research necessary to improve treatment efficacy, mobility, and quality of life in pediatric wheelchair users. This dissertation reports on research studies that evaluate the effect of physiotherapeutic training on manual wheelchair mobility. In particular, these studies (1) develop and characterize a novel markerless motion capture-musculoskeletal model systems interface for kinematic assessment of manual wheelchair propulsion biomechanics, (2) conduct a longitudinal investigation of pediatric manual wheelchair users undergoing intensive community-based therapy to determine predictors of kinematic response, and (3) evaluate propulsion pattern-dependent training efficacy and musculoskeletal behavior using visual biofeedback.Results of the research studies show that taking a systems approach to the kinematic interface produces an effective and reliable system for kinematic assessment and training of manual wheelchair propulsion. The studies also show that the therapeutic outcomes and orthopaedic injury risk of pediatric manual wheelchair users are significantly related to the propulsion pattern employed. Further, these subjects can change their propulsion pattern in response to therapy even in the absence of wheelchair-based training, and have pattern-dependent differences in joint kinematics, musculotendon excursion, and training response. Further clinical research in this area is suggested, with a focus on refining physiotherapeutic training strategies for pediatric manual wheelchair users to develop safer and more effective propulsion patterns

Using the Microsoft Kinect to assess human bimanual coordination

Optical marker-based systems are the gold-standard for capturing three-dimensional (3D) human kinematics. However, these systems have various drawbacks including time consuming marker placement, soft tissue movement artifact, and are prohibitively expensive and non-portable. The Microsoft Kinect is an inexpensive, portable, depth camera that can be used to capture 3D human movement kinematics. Numerous investigations have assessed the Kinect\u27s ability to capture postural control and gait, but to date, no study has evaluated it\u27s capabilities for measuring spatiotemporal coordination. In order to investigate human coordination and coordination stability with the Kinect, a well-studied bimanual coordination paradigm (Kelso, 1984, Kelso; Scholz, & Schöner, 1986) was adapted. ^ Nineteen participants performed ten trials of coordinated hand movements in either in-phase or anti-phase patterns of coordination to the beat of a metronome which was incrementally sped up and slowed down. Continuous relative phase (CRP) and the standard deviation of CRP were used to assess coordination and coordination stability, respectively.^ Data from the Kinect were compared to a Vicon motion capture system using a mixed-model, repeated measures analysis of variance and intraclass correlation coefficients (2,1) (ICC(2,1)).^ Kinect significantly underestimated CRP for the the anti-phase coordination pattern (p \u3c.0001) and overestimated the in-phase pattern (p\u3c.0001). However, a high ICC value (r=.097) was found between the systems. For the standard deviation of CRP, the Kinect exhibited significantly higher variability than the Vicon (p \u3c .0001) but was able to distinguish significant differences between patterns of coordination with anti-phase variability being higher than in-phase (p \u3c .0001). Additionally, the Kinect was unable to accurately capture the structure of coordination stability for the anti-phase pattern. Finally, agreement was found between systems using the ICC (r=.37).^ In conclusion, the Kinect was unable to accurately capture mean CRP. However, the high ICC between the two systems is promising and the Kinect was able to distinguish between the coordination stability of in-phase and anti-phase coordination. However, the structure of variability as movement speed increased was dissimilar to the Vicon, particularly for the anti-phase pattern. Some aspects of coordination are nicely captured by the Kinect while others are not. Detecting differences between bimanual coordination patterns and the stability of those patterns can be achieved using the Kinect. However, researchers interested in the structure of coordination stability should exercise caution since poor agreement was found between systems

Force Sensorless Admittance Control for Teleoperation of Uncertain Robot Manipulator Using Neural Networks

In this paper, a force sensorless control scheme based on neural networks (NNs) is developed for interaction between robot manipulators and human arms in physical collision. In this scheme, the trajectory is generated by using geometry vector method with Kinect sensor. To comply with the external torque from the environment, this paper presents a sensorless admittance control approach in joint space based on an observer approach, which is used to estimate external torques applied by the operator. To deal with the tracking problem of the uncertain manipulator, an adaptive controller combined with the radial basis function NN (RBFNN) is designed. The RBFNN is used to compensate for uncertainties in the system. In order to achieve the prescribed tracking precision, an error transformation algorithm is integrated into the controller. The Lyapunov functions are used to analyze the stability of the control system. The experiments on the Baxter robot are carried out to demonstrate the effectiveness and correctness of the proposed control scheme