Nonlinear Modeling and Control of Driving Interfaces and Continuum Robots for System Performance Gains

With the rise of (semi)autonomous vehicles and continuum robotics technology and applications, there has been an increasing interest in controller and haptic interface designs. The presence of nonlinearities in the vehicle dynamics is the main challenge in the selection of control algorithms for real-time regulation and tracking of (semi)autonomous vehicles. Moreover, control of continuum structures with infinite dimensions proves to be difficult due to their complex dynamics plus the soft and flexible nature of the manipulator body. The trajectory tracking and control of automobile and robotic systems requires control algorithms that can effectively deal with the nonlinearities of the system without the need for approximation, modeling uncertainties, and input disturbances. Control strategies based on a linearized model are often inadequate in meeting precise performance requirements. To cope with these challenges, one must consider nonlinear techniques. Nonlinear control systems provide tools and methodologies for enabling the design and realization of (semi)autonomous vehicle and continuum robots with extended specifications based on the operational mission profiles. This dissertation provides an insight into various nonlinear controllers developed for (semi)autonomous vehicles and continuum robots as a guideline for future applications in the automobile and soft robotics field. A comprehensive assessment of the approaches and control strategies, as well as insight into the future areas of research in this field, are presented.First, two vehicle haptic interfaces, including a robotic grip and a joystick, both of which are accompanied by nonlinear sliding mode control, have been developed and studied on a steer-by-wire platform integrated with a virtual reality driving environment. An operator-in-the-loop evaluation that included 30 human test subjects was used to investigate these haptic steering interfaces over a prescribed series of driving maneuvers through real time data logging and post-test questionnaires. A conventional steering wheel with a robust sliding mode controller was used for all the driving events for comparison. Test subjects operated these interfaces for a given track comprised of a double lane-change maneuver and a country road driving event. Subjective and objective results demonstrate that the driver’s experience can be enhanced up to 75.3% with a robotic steering input when compared to the traditional steering wheel during extreme maneuvers such as high-speed driving and sharp turn (e.g., hairpin turn) passing. Second, a cellphone-inspired portable human-machine-interface (HMI) that incorporated the directional control of the vehicle as well as the brake and throttle functionality into a single holistic device will be presented. A nonlinear adaptive control technique and an optimal control approach based on driver intent were also proposed to accompany the mechatronic system for combined longitudinal and lateral vehicle guidance. Assisting the disabled drivers by excluding extensive arm and leg movements ergonomically, the device has been tested in a driving simulator platform. Human test subjects evaluated the mechatronic system with various control configurations through obstacle avoidance and city road driving test, and a conventional set of steering wheel and pedals were also utilized for comparison. Subjective and objective results from the tests demonstrate that the mobile driving interface with the proposed control scheme can enhance the driver’s performance by up to 55.8% when compared to the traditional driving system during aggressive maneuvers. The system’s superior performance during certain vehicle maneuvers and approval received from the participants demonstrated its potential as an alternative driving adaptation for disabled drivers. Third, a novel strategy is designed for trajectory control of a multi-section continuum robot in three-dimensional space to achieve accurate orientation, curvature, and section length tracking. The formulation connects the continuum manipulator dynamic behavior to a virtual discrete-jointed robot whose degrees of freedom are directly mapped to those of a continuum robot section under the hypothesis of constant curvature. Based on this connection, a computed torque control architecture is developed for the virtual robot, for which inverse kinematics and dynamic equations are constructed and exploited, with appropriate transformations developed for implementation on the continuum robot. The control algorithm is validated in a realistic simulation and implemented on a six degree-of-freedom two-section OctArm continuum manipulator. Both simulation and experimental results show that the proposed method could manage simultaneous extension/contraction, bending, and torsion actions on multi-section continuum robots with decent tracking performance (e.g. steady state arc length and curvature tracking error of 3.3mm and 130mm-1, respectively). Last, semi-autonomous vehicles equipped with assistive control systems may experience degraded lateral behaviors when aggressive driver steering commands compete with high levels of autonomy. This challenge can be mitigated with effective operator intent recognition, which can configure automated systems in context-specific situations where the driver intends to perform a steering maneuver. In this article, an ensemble learning-based driver intent recognition strategy has been developed. A nonlinear model predictive control algorithm has been designed and implemented to generate haptic feedback for lateral vehicle guidance, assisting the drivers in accomplishing their intended action. To validate the framework, operator-in-the-loop testing with 30 human subjects was conducted on a steer-by-wire platform with a virtual reality driving environment. The roadway scenarios included lane change, obstacle avoidance, intersection turns, and highway exit. The automated system with learning-based driver intent recognition was compared to both the automated system with a finite state machine-based driver intent estimator and the automated system without any driver intent prediction for all driving events. Test results demonstrate that semi-autonomous vehicle performance can be enhanced by up to 74.1% with a learning-based intent predictor. The proposed holistic framework that integrates human intelligence, machine learning algorithms, and vehicle control can help solve the driver-system conflict problem leading to safer vehicle operations

Soft Gloves: A Review on Recent Developments in Actuation, Sensing, Control and Applications

Interest in soft gloves, both robotic and haptic, has enormously grown over the past decade, due to their inherent compliance, which makes them particularly suitable for direct interaction with the human hand. Robotic soft gloves have been developed for hand rehabilitation, for ADLs assistance, or sometimes for both. Haptic soft gloves may be applied in virtual reality (VR) applications or to give sensory feedback in combination with prostheses or to control robots. This paper presents an updated review of the state of the art of soft gloves, with a particular focus on actuation, sensing, and control, combined with a detailed analysis of the devices according to their application field. The review is organized on two levels: a prospective review allows the highlighting of the main trends in soft gloves development and applications, and an analytical review performs an in-depth analysis of the technical solutions developed and implemented in the revised scientific research. Additional minor evaluations integrate the analysis, such as a synthetic investigation of the main results in the clinical studies and trials referred in literature which involve soft gloves

A novel training and collaboration integrated framework for human-agent teleoperation.

Human operators have the trend of increasing physical and mental workloads when performing teleoperation tasks in uncertain and dynamic environments. In addition, their performances are influenced by subjective factors, potentially leading to operational errors or task failure. Although agent-based methods offer a promising solution to the above problems, the human experience and intelligence are necessary for teleoperation scenarios. In this paper, a truncated quantile critics reinforcement learning-based integrated framework is proposed for human-agent teleoperation that encompasses training, assessment and agent-based arbitration. The proposed framework allows for an expert training agent, a bilateral training and cooperation process to realize the co-optimization of agent and human. It can provide efficient and quantifiable training feedback. Experiments have been conducted to train subjects with the developed algorithm. The performances of human-human and human-agent cooperation modes are also compared. The results have shown that subjects can complete the tasks of reaching and picking and placing with the assistance of an agent in a shorter operational time, with a higher success rate and less workload than human-human cooperation

Intent-Recognition-Based Traded Control for Telerobotic Assembly over High-Latency Telemetry

As we deploy robotic manipulation systems into unstructured real-world environments, the tasks which those robots are expected to perform grow very quickly in complexity. These tasks require a greater number of possible actions, more variable environmental conditions, and larger varieties of objects and materials which need to be manipulated. This in turn leads to a greater number of ways in which elements of a task can fail. When the cost of task failure is high, such as in the case of surgery or on-orbit robotic interventions, effective and efficient task recovery is essential. Despite ever-advancing capabilities, however, the current and near future state-of-the-art in fully autonomous robotic manipulation is still insufficient for many tasks in these critical applications. Thus, successful application of robotic manipulation in many application domains still necessitates a human operator to directly teleoperate the robots over some communications infrastructure. However, any such infrastructure always incurs some unavoidable round-trip telemetry latency depending on the distances involved and the type of remote environment. While direct teleoperation is appropriate when a human operator is physically close to the robots being controlled, there are still many applications in which such proximity is infeasible. In applications which require a robot to be far from its human operator, this latency can approach the speed of the relevant task dynamics, and performing the task with direct telemanipulation can become increasingly difficult, if not impossible. For example, round-trip delays for ground-controlled on-orbit robotic manipulation can reach multiple seconds depending on the infrastructure used and the location of the remote robot. The goal of this thesis is to advance the state-of-the art in semi-autonomous telemanipulation under multi-second round-trip communications latency between a human operator and remote robot in order to enable more telerobotic applications. We propose a new intent-recognition-based traded control (IRTC) approach which automatically infers operator intent and executes task elements which the human operator would otherwise be unable to perform. What makes our approach more powerful than the current approaches is that we prioritize preserving the operator's direct manual interaction with the remote environment while only trading control over to an autonomous subsystem when the operator-local intent recognition system automatically determines what the operator is trying to accomplish. This enables operators to perform unstructured and a priori unplanned actions in order to quickly recover from critical task failures. Furthermore, this thesis also describes a methodology for introducing and improving semi-autonomous control in critical applications. Specifically, this thesis reports (1) the demonstration of a prototype system for IRTC-based grasp assistance in the context of transatlantic telemetry delays, (2) the development of a systems framework for IRTC in semi-autonomous telemanipulation, and (3) an evaluation of the usability and efficacy of that framework with an increasingly complex assembly task. The results from our human subjects experiments show that, when incorporated with sufficient lower-level capabilities, IRTC is a promising approach to extend the reach and capabilities of on-orbit telerobotics and future in-space operations

Biomechatronics: Harmonizing Mechatronic Systems with Human Beings

This eBook provides a comprehensive treatise on modern biomechatronic systems centred around human applications. A particular emphasis is given to exoskeleton designs for assistance and training with advanced interfaces in human-machine interaction. Some of these designs are validated with experimental results which the reader will find very informative as building-blocks for designing such systems. This eBook will be ideally suited to those researching in biomechatronic area with bio-feedback applications or those who are involved in high-end research on manmachine interfaces. This may also serve as a textbook for biomechatronic design at post-graduate level

Haptic Steering Interfaces for Semi-Autonomous Vehicles

Autonomous vehicles are predicted to significantly improve transportation quality by reducing traffic congestion, fuel expenditure and road accidents. However, until autonomous vehicles are reliable in all scenarios, human drivers will be asked to supervise automation behavior and intervene in automated driving when deemed necessary. Retaining the human driver in a strictly supervisory role, however, may make the driver complacent and reduce driver's situation awareness and driving skills which ironically, can further compromise the driver’s ability to intervene in safety-critical scenarios. Such issues can be alleviated by designing a human-automation interface that keeps the driver in-the-loop through constant interaction with automation and continuous feedback of automation's actions. This dissertation evaluates the utility of haptic feedback at the steering interface for enhancing driver awareness and enabling continuous human-automation interaction and performance improvement in semi-autonomous vehicles. In the first part of this dissertation, I investigate a driving scheme called Haptic Shared Control (HSC) in which the human driver and automation system share the steering control by simultaneously acting at the steering interface with finite mechanical impedances. I hypothesize that HSC can mitigate the human factors issues associated with semi-autonomous driving by allowing the human driver to continuously interact with automation and receive feedback about automation action. To test this hypothesis, I present two driving simulator experiments that are focused on the evaluation of HSC with respect to existing driving schemes during induced human and automation faults. In the first experiment, I compare obstacle avoidance performance of HSC with two existing control sharing schemes that support instantaneous transfers of control authority between human and automation. The results indicate that HSC outperforms both schemes in terms of obstacle avoidance, maneuvering efficiency, and driver engagement. In the second experiment, I consider emergency scenarios where I compare two HSC designs that provide high and low control authority to automation and an existing paradigm that decouples the driver input from the tires during collision avoidance. Results show that decoupling the driver invokes out-of-the-loop issues and misleads drivers to believe that they are in control. I also discover a `fault protection tradeoff': as the control authority provided to one agent increases, the protection against that agent's faults provided by the other agent reduces. In the second part of this dissertation, I focus on the problem of estimating haptic feedback from the road, or the road feedback. Road feedback is critical to making the driver aware of the state of the vehicle and road conditions, and its estimates are used in a variety of driver assist systems. However, conventional estimators only estimate road feedback on flat roads. To overcome this issue, I develop three estimators that enable road feedback estimation on uneven roads. I test and compare the performance of the three estimators by performing driving experiments on uneven roads such as road slopes and cleats. In the final part of this dissertation, I shift focus from physical human-automation interaction to human-human interaction. I present the evidence from the literature demonstrating that haptic feedback improves the performance of two humans physically collaborating on a shared task. I develop a control-theoretic model for haptic communication that can describe the mechanism by which haptic interaction facilitates performance improvement. The model creates a promising means to transfer the obtained insights to design robots or automation systems that can collaborate more efficiently with humans.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169975/1/akshaybh_1.pd

Augmenting user capabilities through an adaptive assistive manipulator

Mención Internacional en el título de doctorAssistive robot manipulators have the potential to increase the independence of disabled persons in activities of daily living. The current designs are mainly limited to pure teleoperation by the user, given the need for keeping the user in the control loop, and the complexity of the tasks and environments in which they operate. This thesis aims to augment the user’s capabilities for performing such tasks by adapting the robot, and its level of assistance, to the user. Methodologies for modeling and benchmarking the complete human-robot system were established, which helped drive the development of different approaches to adaptation. This included a task-oriented optimization of the robot physical structure, approaches for low-level adaptive shared control, and work on interactive learning of, and assistance on completing, simple object manipulation tasks. Three experimental platforms were used: The ASIBOT manipulator of Universidad Carlos III de Madrid (UC3M), the AMOR manipulator of Exact Dynamics, and the iCub humanoid robot.Los manipuladores asistenciales tienen el potencial de incrementar la independencia de personas discapacitadas en sus actividades de la vida diaria. Los diseños actuales se limitan principalmente a una pura teleoperación, pues dada la complejidad de las tareas y del entorno, se necesita mantener al usuario en el lazo de control. Esta tesis pretende mejorar las capacidades del usuario para realizar estas tareas, adaptando el robot y su nivel de asistencia a las necesidades del usuario. Se han establecido metodologías para el modelado y evaluación del comportamiento del sistema formado por humano y robot, lo que ha permitido el desarrollo de diferentes aproximaciones a la adaptación. Esto incluye desde la optimización de la estructura del robot atendiendo a las tareas, la evaluación de diversas aproximaciones al control compartido adaptativo a bajo nivel, al aprendizaje interactivo y el desarrollo de asistencias para completar tareas sencillas de manipulación. Se ha hecho uso de tres plataformas experimentales: el manipulador ASIBOT de la Universidad Carlos III de Madrid (UC3M), el manipulador AMOR de Exact Dynamics y el humanoide iCub.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Alberto Sanfeliú.- Secretario: Concepción Alicia Monje Micharet.- Vocal: Yiannis Demiri