Redesigning the human-robot interface : intuitive teleoperation of anthropomorphic robots

textA novel interface for robotic teleoperation was developed to enable accurate and highly efficient teleoperation of the Industrial Reconfigurable Anthropomorphic Dual-arm (IRAD) system and other robotic systems. In order to achieve a revolutionary increase in operator productivity, the bilateral/master-slave approach must give way to shared autonomy and unilateral control; autonomy must be employed where possible, and appropriate sensory feedback only where autonomy is impossible; and today’s low-information/high feedback model must be replaced by one that emphasizes feedforward precision and minimal corrective feedback. This is emphasized for task spaces outside of the traditional anthropomorphic scale such as mobile manipulation (i.e. large task spaces) and high precision tasks (i.e. very small task spaces). The system is demonstrated using an anthropomorphically dimensioned industrial manipulator working in task spaces from one meter to less than one millimeter, in both simulation and hardware. This thesis discusses the design requirements and philosophy of this interface, provides a summary of prototype teleoperation hardware, simulation environment, test-bed hardware, and experimental results.Mechanical Engineerin

Effects of different push-to-talk solutions on driving performance

Police officers have been using the Project54 system in their vehicles for a number of years. They have also started using the handheld version of Project54 outside their vehicles recently. There is a need to connect these two instances of the system into a continuous user interface. On the other hand, research has shown that the PTT button location affects driving performance. This thesis investigates the difference between the old, fixed PTT button and a new wireless PTT glove, that could be used in and outside of the car. The thesis describes the design of the glove and the driving simulator experiment that was conducted to investigate the glove\u27s merit. The main results show that the glove allows more freedom of operation, appears to be easier and more efficient to operate and it reduces the visual distraction of the drivers

The next challenge for world wide robotized tele-echography experiment (WORTEX 2012): from engineering success to healthcare delivery.

Access to good quality healthcare remains difficult for many patients whether they live in developed or developing countries. In developed countries, specialist medical expertise is concentrated in major hospitals in urban settings both to improve clinical outcomes and as a strategy to reduce the costs of specialist healthcare delivery. In developing countries, millions of people have limited, if any, routine access to a healthcare system and due to economic and cultural factors the accessibility of any services may be restricted. In both cases, geographical, socio-political, cultural and economic factors produce ‘medically isolated areas’ where patients find themselves disadvantaged in terms of timely diagnosis and expert and/or expensive treatment. The robotized teleechography approach, also referred to as robotized teleultrasound, offers a potential solution to diagnostic imaging in medically isolated areas. It is designed for patients requiring ultrasound scans for routine care (e.g., ante natal care) and for diagnostic imaging to investigate acute and medical emergencies conditions, including trauma care and responses to natural disasters such as earthquakes. The robotized teleechography system can hold any standard ultrasound probe; this lightweight system is positioned on the patient’s body by a healthcare assistant. The medical expert, a clinician with expertise in ultrasound imaging and diagnosis, is in a distant location and, using a dedicated joystick, remotely controls the scanning via any available communication link (Internet, satellite). The WORTEX2012 intercontinental trials of the system conducted last year successfully demonstrated the feasibility of remote robotized tele-echography in a range of cultural, technical and clinical contexts. In addition to the engineering success, these trials provided positive feedback from the participating clinicians and patients on using the system and on the system’s perceived potential to transform healthcare in medically isolated areas. The next challenge is to show evidence that this innovative technology can deliver on its promise if introduced into routine healthcare

AI enhanced collaborative human-machine interactions for home-based telerehabilitation

The use of robots in a telerehabilitation paradigm could facilitate the delivery of rehabilitation on demand while reducing transportation time and cost. As a result, it helps to motivate patients to exercise frequently in a more comfortable home environment. However, for such a paradigm to work, it is essential that the robustness of the system is not compromised due to network latency, jitter, and delay of the internet. This paper proposes a solution to data loss compensation to maintain the quality of the interaction between the user and the system. Data collected from a well-defined collaborative task using a virtual reality (VR) environment was used to train a robotic system to adapt to the users' behaviour. The proposed approach uses nonlinear autoregressive models with exogenous input (NARX) and long-short term memory (LSTM) neural networks to smooth out the interaction between the user and the predicted movements generated from the system. LSTM neural networks are shown to learn to act like an actual human. The results from this paper have shown that, with an appropriate training method, the artificial predictor can perform very well by allowing the predictor to complete the task within 25 s versus 23 s when executed by the human

Security framework for industrial collaborative robotic cyber-physical systems

The paper introduces a security framework for the application of human-robot collaboration in a futuristic industrial cyber-physical system (CPS) context of industry 4.0. The basic elements and functional requirements of a secure collaborative robotic cyber-physical system are explained and then the cyber-attack modes are discussed in the context of collaborative CPS whereas a defense mechanism strategy is proposed for such a complex system. The cyber-attacks are categorized according to the extent on controllability and the possible effects on the performance and efficiency of such CPS. The paper also describes the severity and categorization of such cyber-attacks and the causal effect on the human worker safety during human-robot collaboration. Attacks in three dimensions of availability, authentication and confidentiality are proposed as the basis of a consolidated mitigation plan. We propose a security framework based on a two-pronged strategy where the impact of this methodology is demonstrated on a teleoperation benchmark (NeCS-Car). The mitigation strategy includes enhanced data security at important interconnected adaptor nodes and development of an intelligent module that employs a concept similar to system health monitoring and reconfiguration

Evaluation of teleoperation system performance over a cellular network

The ubiquity of cellular networks has exploded over the last half decade making internet access a given when located in an urban settings. On top of this, new technologies like 4G LTE provide higher transfer speeds than ever, permitting streaming of video and other high bandwidth services. Though cellular networks are not new, few studies have leveraged this particular communications method when studying teleoperations, due to the significant bandwidth restrictions. As a result, this study seeks to understand whether teleoperation could be implemented over regular cellular networks where the bandwidth load that each cell tower is subject to cannot be controlled by the teleoperation system. For this, a prototype system is built using a remote controlled golf cart that hosts a multimedia link between the vehicle and a control station which communicate over the internet. The system is tested by measuring teleoperation for 3 different tasks of varying degrees of complexity. The results reveal that latency can be low enough to optimally control a remote vehicle. Nevertheless, the performance greatly depends on the network conditions that can vary significantly. The results also indicated that in-situ driving outperformed remote operation.M.S

The Shape of Damping: Optimizing Damping Coefficients to Improve Transparency on Bilateral Telemanipulation

This thesis presents a novel optimization-based passivity control algorithm for hapticenabled bilateral teleoperation systems involving multiple degrees of freedom. In particular, in the context of energy-bounding control, the contribution focuses on the implementation of a passivity layer for an existing time-domain scheme, ensuring optimal transparency of the interaction along subsets of the environment space which are preponderant for the given task, while preserving the energy bounds required for passivity. The involved optimization problem is convex and amenable to real-time implementation. The effectiveness of the proposed design is validated via an experiment performed on a virtual teleoperated environment. The interplay between transparency and stability is a critical aspect in haptic-enabled bilateral teleoperation control. While it is important to present the user with the true impedance of the environment, destabilizing factors such as time delays, stiff environments, and a relaxed grasp on the master device may compromise the stability and safety of the system. Passivity has been exploited as one of the the main tools for providing sufficient conditions for stable teleoperation in several controller design approaches, such as the scattering algorithm, timedomain passivity control, energy bounding algorithm, and passive set position modulation. In this work it is presented an innovative energy-based approach, which builds upon existing time-domain passivity controllers, improving and extending their effectiveness and functionality. The set of damping coefficients are prioritized in each degree of freedom, the resulting transparency presents a realistic force feedback in comparison to the other directions. Thus, the prioritization takes effect using a quadratic programming algorithm to find the optimal values for the damping. Finally, the energy tanks approach on passivity control is a solution used to ensure stability in a system for robotics bilateral manipulation. The bilateral telemanipulation must maintain the principle of passivity in all moments to preserve the system\u2019s stability. This work presents a brief introduction to haptic devices as a master component on the telemanipulation chain; the end effector in the slave side is a representation of an interactive object within an environment having a force sensor as feedback signal. The whole interface is designed into a cross-platform framework named ROS, where the user interacts with the system. Experimental results are presented

A Novel Predictor Based Framework to Improve Mobility of High Speed Teleoperated Unmanned Ground Vehicles

Teleoperated Unmanned Ground Vehicles (UGVs) have been widely used in applications when driver safety, mission eciency or mission cost is a major concern. One major challenge with teleoperating a UGV is that communication delays can significantly affect the mobility performance of the vehicle and make teleoperated driving tasks very challenging especially at high speeds. In this dissertation, a predictor based framework with predictors in a new form and a blended architecture are developed to compensate effects of delays through signal prediction, thereby improving vehicle mobility performance. The novelty of the framework is that minimal information about the governing equations of the system is required to compensate delays and, thus, the prediction is robust to modeling errors. This dissertation first investigates a model-free solution and develops a predictor that does not require information about the vehicle dynamics or human operators' motion for prediction. Compared to the existing model-free methods, neither assumptions about the particular way the vehicle moves, nor knowledge about the noise characteristics that drive the existing predictive filters are needed. Its stability and performance are studied and a predictor design procedure is presented. Secondly, a blended architecture is developed to blend the outputs of the model-free predictor with those of a steering feedforward loop that relies on minimal information about vehicle lateral response. Better prediction accuracy is observed based on open-loop virtual testing with the blended architecture compared to using either the model-free predictors or the model-based feedforward loop alone. The mobility performance of teleoperated vehicles with delays and the predictor based framework are evaluated in this dissertation with human-in-the-loop experiments using both simulated and physical vehicles in teleoperation mode. Predictor based framework is shown to provide a statistically significant improvement in vehicle mobility and drivability in the experiments performed.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/146026/1/zhengys_1.pd