Autonomy Infused Teleoperation with Application to BCI Manipulation

Robot teleoperation systems face a common set of challenges including latency, low-dimensional user commands, and asymmetric control inputs. User control with Brain-Computer Interfaces (BCIs) exacerbates these problems through especially noisy and erratic low-dimensional motion commands due to the difficulty in decoding neural activity. We introduce a general framework to address these challenges through a combination of computer vision, user intent inference, and arbitration between the human input and autonomous control schemes. Adjustable levels of assistance allow the system to balance the operator's capabilities and feelings of comfort and control while compensating for a task's difficulty. We present experimental results demonstrating significant performance improvement using the shared-control assistance framework on adapted rehabilitation benchmarks with two subjects implanted with intracortical brain-computer interfaces controlling a seven degree-of-freedom robotic manipulator as a prosthetic. Our results further indicate that shared assistance mitigates perceived user difficulty and even enables successful performance on previously infeasible tasks. We showcase the extensibility of our architecture with applications to quality-of-life tasks such as opening a door, pouring liquids from containers, and manipulation with novel objects in densely cluttered environments

Telescience Testbed Pilot Program

The Telescience Testbed Pilot Program is developing initial recommendations for requirements and design approaches for the information systems of the Space Station era. During this quarter, drafting of the final reports of the various participants was initiated. Several drafts are included in this report as the University technical reports

Scaled bilateral teleoperation using discrete-time sliding mode controller

In this paper, the design of a discrete-time slidingmode controller based on Lyapunov theory is presented along with a robust disturbance observer and is applied to a piezostage for high-precision motion. A linear model of a piezostage was used with nominal parameters to compensate the disturbance acting on the system in order to achieve nanometer accuracy. The effectiveness of the controller and disturbance observer is validated in terms of closed-loop position performance for nanometer references. The control structure has been applied to a scaled bilateral structure for the custom-built telemicromanipulation setup. A piezoresistive atomic force microscope cantilever with a built-in Wheatstone bridge is utilized to achieve the nanonewtonlevel interaction forces between the piezoresistive probe tip and the environment. Experimental results are provided for the nanonewton-range force sensing, and good agreement between the experimental data and the theoretical estimates has been demonstrated. Force/position tracking and transparency between the master and the slave has been clearly demonstrated after necessary scalin

Semi-autonomous scheme for pushing micro-objects

-In many microassembly applications, it is often desirable to position and orient polygonal micro-objects lying on a planar surface. Pushing micro-objects using point contact provides more flexibility and less complexity compared to pick and place operation. Due to the fact that in micro-world surface forces are much more dominant than inertial forces and these forces are distributed unevenly, pushing through the center of mass of the micro-object will not yield a pure translational motion. In order to translate a micro-object, the line of pushing should pass through the center of friction. In this paper, a semi-autonomous scheme based on hybrid vision/force feedback is proposed to push microobjects with human assistance using a custom built telemicromanipulation setup to achieve pure translational motion. The pushing operation is divided into two concurrent processes: In one process human operator who acts as an impedance controller alters the velocity of the pusher while in contact with the micro-object through scaled bilateral teleoperation with force feedback. In the other process, the desired line of pushing for the micro-object is determined continuously using visual feedback procedures so that it always passes through the varying center of friction. Experimental results are demonstrated to prove nanoNewton range force sensing, scaled bilateral teleoperation with force feedback and pushing microobjects

Diverse applications of advanced man-telerobot interfaces

Advancements in man-machine interfaces and control technologies used in space telerobotics and teleoperators have potential application wherever human operators need to manipulate multi-dimensional spatial relationships. Bilateral six degree-of-freedom position and force cues exchanged between the user and a complex system can broaden and improve the effectiveness of several diverse man-machine interfaces

Teleoperation of passivity-based model reference robust control over the internet

This dissertation offers a survey of a known theoretical approach and novel experimental results in establishing a live communication medium through the internet to host a virtual communication environment for use in Passivity-Based Model Reference Robust Control systems with delays. The controller which is used as a carrier to support a robust communication between input-to-state stability is designed as a control strategy that passively compensates for position errors that arise during contact tasks and strives to achieve delay-independent stability for controlling of aircrafts or other mobile objects. Furthermore the controller is used for nonlinear systems, coordination of multiple agents, bilateral teleoperation, and collision avoidance thus maintaining a communication link with an upper bound of constant delay is crucial for robustness and stability of the overall system. For utilizing such framework an elucidation can be formulated by preparing site survey for analyzing not only the geographical distances separating the nodes in which the teleoperation will occur but also the communication parameters that define the virtual topography that the data will travel through. This survey will first define the feasibility of the overall operation since the teleoperation will be used to sustain a delay based controller over the internet thus obtaining a hypothetical upper bound for the delay via site survey is crucial not only for the communication system but also the delay is required for the design of the passivity-based model reference robust control. Following delay calculation and measurement via site survey, bandwidth tests for unidirectional and bidirectional communication is inspected to ensure that the speed is viable to maintain a real-time connection. Furthermore from obtaining the results it becomes crucial to measure the consistency of the delay throughout a sampled period to guarantee that the upper bound is not breached at any point within the communication to jeopardize the robustness of the controller. Following delay analysis a geographical and topological overview of the communication is also briefly examined via a trace-route to understand the underlying nodes and their contribution to the delay and round-trip consistency. To accommodate the communication channel for the controller the input and output data from both nodes need to be encapsulated within a transmission control protocol via a multithreaded design of a robust program within the C language. The program will construct a multithreaded client-server relationship in which the control data is transmitted. For added stability and higher level of security the channel is then encapsulated via an internet protocol security by utilizing a protocol suite for protecting the communication by authentication and encrypting each packet of the session using negotiation of cryptographic keys during each session

Force feedback pushing scheme for micromanipulation applications

Pushing micro-objects using point contact provides more flexibility and less complexity compared to pick and place operation. Due to the fact that in micro-world surface forces are much more dominant than inertial forces and these forces are distributed unevenly, pushing through the center of mass of the micro-object may not yield a pure translational motion. In order to translate a micro-object, the line of pushing should pass through the center of friction. In this paper, a semi-autonomous scheme based on hybrid vision/force feedback procedure is proposed to push micro-objects with human assistance using a custom built tele-micromanipulation setup to achieve translational motion. In the semi-autonomous pushing process, velocity controlled pushing with force feedback is realized along x-axis by the human operator while y-axis orientation is undertaken automatically using visual feedback. This way the desired line of pushing for the micro-object is controlled to pass through the varying center of friction. Experimental results are shown to prove nano-Newton range force sensing, scaled bilateral teleoperation with force feedback and snapshot of pushing operation