Autonomy Infused Teleoperation with Application to BCI Manipulation

Robot teleoperation systems face a common set of challenges including latency, low-dimensional user commands, and asymmetric control inputs. User control with Brain-Computer Interfaces (BCIs) exacerbates these problems through especially noisy and erratic low-dimensional motion commands due to the difficulty in decoding neural activity. We introduce a general framework to address these challenges through a combination of computer vision, user intent inference, and arbitration between the human input and autonomous control schemes. Adjustable levels of assistance allow the system to balance the operator's capabilities and feelings of comfort and control while compensating for a task's difficulty. We present experimental results demonstrating significant performance improvement using the shared-control assistance framework on adapted rehabilitation benchmarks with two subjects implanted with intracortical brain-computer interfaces controlling a seven degree-of-freedom robotic manipulator as a prosthetic. Our results further indicate that shared assistance mitigates perceived user difficulty and even enables successful performance on previously infeasible tasks. We showcase the extensibility of our architecture with applications to quality-of-life tasks such as opening a door, pouring liquids from containers, and manipulation with novel objects in densely cluttered environments

Delay compensation for nonlinear teleoperators using predictor observers

This paper presents a delay compensation technique for nonlinear teleoperators by developing a predictor type sliding mode observer (SMO) that estimates future states of the slave operator. Predicted states are then used in control formulation. In the proposed scheme, disturbance observers (DOB) are also utilized to linearize nonlinear dynamics of the master and slave operators. It is shown that utilization of disturbance observers and predictor observer allow simple PD controllers to be used to provide stable position tracking for bilateral teleoperation. Proposed approach is verified with simulations where it is compared with two state-of-the-art methods. Successful experimental results with a bilateral teleoperation system consisting of a pair of pantograph robots also validates the proposed method