Position and Velocity Control for Telemanipulation with Interoperability Protocol

International audienceIn this paper we describe how a generic interoperability teler-obotics protocol can be applied for master-slave robotic systems operating in position-position, position-speed and hybrid control modes. The interoperability protocol allows robust and efficient data exchange for teleoperation systems, however it was not shown how it can fit switching position and rate control modes. Here we propose the general framework of hybrid position and rate control modes with interoperability protocol. Furthermore, we demonstrate experimentally that the framework is suitable for robotics teleoperation systems in which a human-operator can switch between position-position and position-speed master and slave robots' workspace mapping

Exploring Robot Teleoperation in Virtual Reality

This thesis presents research on VR-based robot teleoperation with a focus on remote environment visualisation in virtual reality, the effects of remote environment reconstruction scale in virtual reality on the human-operator's ability to control the robot and human-operator's visual attention patterns when teleoperating a robot from virtual reality. A VR-based robot teleoperation framework was developed, it is compatible with various robotic systems and cameras, allowing for teleoperation and supervised control with any ROS-compatible robot and visualisation of the environment through any ROS-compatible RGB and RGBD cameras. The framework includes mapping, segmentation, tactile exploration, and non-physically demanding VR interface navigation and controls through any Unity-compatible VR headset and controllers or haptic devices. Point clouds are a common way to visualise remote environments in 3D, but they often have distortions and occlusions, making it difficult to accurately represent objects' textures. This can lead to poor decision-making during teleoperation if objects are inaccurately represented in the VR reconstruction. A study using an end-effector-mounted RGBD camera with OctoMap mapping of the remote environment was conducted to explore the remote environment with fewer point cloud distortions and occlusions while using a relatively small bandwidth. Additionally, a tactile exploration study proposed a novel method for visually presenting information about objects' materials in the VR interface, to improve the operator's decision-making and address the challenges of point cloud visualisation. Two studies have been conducted to understand the effect of virtual world dynamic scaling on teleoperation flow. The first study investigated the use of rate mode control with constant and variable mapping of the operator's joystick position to the speed (rate) of the robot's end-effector, depending on the virtual world scale. The results showed that variable mapping allowed participants to teleoperate the robot more effectively but at the cost of increased perceived workload. The second study compared how operators used a virtual world scale in supervised control, comparing the virtual world scale of participants at the beginning and end of a 3-day experiment. The results showed that as operators got better at the task they as a group used a different virtual world scale, and participants' prior video gaming experience also affected the virtual world scale chosen by operators. Similarly, the human-operator's visual attention study has investigated how their visual attention changes as they become better at teleoperating a robot using the framework. The results revealed the most important objects in the VR reconstructed remote environment as indicated by operators' visual attention patterns as well as their visual priorities shifts as they got better at teleoperating the robot. The study also demonstrated that operators’ prior video gaming experience affects their ability to teleoperate the robot and their visual attention behaviours

Models for force control in telesurgical robot systems

Surgical robotics is one of the most rapidly developing fields within robotics. Besides general motion control issues, control engineers often find it challenging to design robotic telesurgery systems, as these have to deal with complex environmental constrains. The unique behavior of soft tissues requires special approaches in both robot control and system modeling in the case of robotic tissue manipulation. Precise control depends on the appropriate modeling of the interaction between the manipulated tissues and the instruments held by the robotic arm, frequently referred to as the tool–tissue interaction. Due to the nature of the physiological environment, the mechatronics of the systems and the time delays, it is difficult to introduce a universal model or a general modeling approach. This paper gives an overview of the emerging problems in the design and modeling of telesurgical systems, analyzing each component, and introducing the most widely employed models. The arising control problems are reviewed in the frames of master–slave type teleoperation, proposing a novel oft tissue model and providing an overview of the possible control approaches

Exploring Robot Teleoperation in Virtual Reality

This thesis presents research on VR-based robot teleoperation with a focus on remote environment visualisation in virtual reality, the effects of remote environment reconstruction scale in virtual reality on the human-operator's ability to control the robot and human-operator's visual attention patterns when teleoperating a robot from virtual reality. A VR-based robot teleoperation framework was developed, it is compatible with various robotic systems and cameras, allowing for teleoperation and supervised control with any ROS-compatible robot and visualisation of the environment through any ROS-compatible RGB and RGBD cameras. The framework includes mapping, segmentation, tactile exploration, and non-physically demanding VR interface navigation and controls through any Unity-compatible VR headset and controllers or haptic devices. Point clouds are a common way to visualise remote environments in 3D, but they often have distortions and occlusions, making it difficult to accurately represent objects' textures. This can lead to poor decision-making during teleoperation if objects are inaccurately represented in the VR reconstruction. A study using an end-effector-mounted RGBD camera with OctoMap mapping of the remote environment was conducted to explore the remote environment with fewer point cloud distortions and occlusions while using a relatively small bandwidth. Additionally, a tactile exploration study proposed a novel method for visually presenting information about objects' materials in the VR interface, to improve the operator's decision-making and address the challenges of point cloud visualisation. Two studies have been conducted to understand the effect of virtual world dynamic scaling on teleoperation flow. The first study investigated the use of rate mode control with constant and variable mapping of the operator's joystick position to the speed (rate) of the robot's end-effector, depending on the virtual world scale. The results showed that variable mapping allowed participants to teleoperate the robot more effectively but at the cost of increased perceived workload. The second study compared how operators used a virtual world scale in supervised control, comparing the virtual world scale of participants at the beginning and end of a 3-day experiment. The results showed that as operators got better at the task they as a group used a different virtual world scale, and participants' prior video gaming experience also affected the virtual world scale chosen by operators. Similarly, the human-operator's visual attention study has investigated how their visual attention changes as they become better at teleoperating a robot using the framework. The results revealed the most important objects in the VR reconstructed remote environment as indicated by operators' visual attention patterns as well as their visual priorities shifts as they got better at teleoperating the robot. The study also demonstrated that operators’ prior video gaming experience affects their ability to teleoperate the robot and their visual attention behaviours

Towards Robotic Laboratory Automation Plug & Play: The "LAPP" Framework

Increasing the level of automation in pharmaceutical laboratories and production facilities plays a crucial role in delivering medicine to patients. However, the particular requirements of this field make it challenging to adapt cutting-edge technologies present in other industries. This article provides an overview of relevant approaches and how they can be utilized in the pharmaceutical industry, especially in development laboratories. Recent advancements include the application of flexible mobile manipulators capable of handling complex tasks. However, integrating devices from many different vendors into an end-to-end automation system is complicated due to the diversity of interfaces. Therefore, various approaches for standardization are considered in this article, and a concept is proposed for taking them a step further. This concept enables a mobile manipulator with a vision system to "learn" the pose of each device and - utilizing a barcode - fetch interface information from a universal cloud database. This information includes control and communication protocol definitions and a representation of robot actions needed to operate the device. In order to define the movements in relation to the device, devices have to feature - besides the barcode - a fiducial marker as standard. The concept will be elaborated following appropriate research activities in follow-up papers

1st year EFAST annual report

The present report provides information about the activities conducted during the 1st year of the EFAST project. The first chapter is dedicated to describe the inquiries conducted at the beginning of the project and to briefly summarise the main results. The second chapter is dedicated to the first EFAST workshop where some of the leading scientists in the field of earthquake engineering have met to discuss about the need and the technologies related to earthquake engineering. The third chapter contains a state of the art and future direction in seismic testing and simulation. The final chapter is dedicated to describe the preliminary design of the web portal of the future testing facility.JRC.DG.G.5-European laboratory for structural assessmen

An Evaluation Schema for the Ethical Use of Autonomous Robotic Systems in Security Applications

We propose a multi-step evaluation schema designed to help procurement agencies and others to examine the ethical dimensions of autonomous systems to be applied in the security sector, including autonomous weapons systems

Validation of design artefacts for blockchain-enabled precision healthcare as a service.

Healthcare systems around the globe are currently experiencing a rapid wave of digital disruption. Current research in applying emerging technologies such as Big Data (BD), Artificial Intelligence (AI), Machine Learning (ML), Deep Learning (DL), Augmented Reality (AR), Virtual Reality (VR), Digital Twin (DT), Wearable Sensor (WS), Blockchain (BC) and Smart Contracts (SC) in contact tracing, tracking, drug discovery, care support and delivery, vaccine distribution, management, and delivery. These disruptive innovations have made it feasible for the healthcare industry to provide personalised digital health solutions and services to the people and ensure sustainability in healthcare. Precision Healthcare (PHC) is a new inclusion in digital healthcare that can support personalised needs. It focuses on supporting and providing precise healthcare delivery. Despite such potential, recent studies show that PHC is ineffectual due to the lower patient adoption in the system. Anecdotal evidence shows that people are refraining from adopting PHC due to distrust. This thesis presents a BC-enabled PHC ecosystem that addresses ongoing issues and challenges regarding low opt-in. The designed ecosystem also incorporates emerging information technologies that are potential to address the need for user-centricity, data privacy and security, accountability, transparency, interoperability, and scalability for a sustainable PHC ecosystem. The research adopts Soft System Methodology (SSM) to construct and validate the design artefact and sub-artefacts of the proposed PHC ecosystem that addresses the low opt-in problem. Following a comprehensive view of the scholarly literature, which resulted in a draft set of design principles and rules, eighteen design refinement interviews were conducted to develop the artefact and sub-artefacts for design specifications. The artefact and sub-artefacts were validated through a design validation workshop, where the designed ecosystem was presented to a Delphi panel of twenty-two health industry actors. The key research finding was that there is a need for data-driven, secure, transparent, scalable, individualised healthcare services to achieve sustainability in healthcare. It includes explainable AI, data standards for biosensor devices, affordable BC solutions for storage, privacy and security policy, interoperability, and usercentricity, which prompts further research and industry application. The proposed ecosystem is potentially effective in growing trust, influencing patients in active engagement with real-world implementation, and contributing to sustainability in healthcare

Large space structures and systems in the space station era: A bibliography with indexes

Bibliographies and abstracts are listed for 1219 reports, articles, and other documents introduced into the NASA scientific and technical information system between July 1, 1990 and December 31, 1990. The purpose is to provide helpful information to the researcher, manager, and designer in technology development and mission design according to system, interactive analysis and design, structural and thermal analysis and design, structural concepts and control systems, electronics, advanced materials, assembly concepts, propulsion, and solar power satellite systems

Haptic data reduction through dynamic perceptual analysis and event-based communication

This research presents an adjustable and flexible framework for haptic data compression and communication that can be used in a robotic teleoperation session. The framework contains a customized event-driven transmission control protocol, several dynamically adaptive perceptual and prediction methods for haptic sample reduction, and last but not the least, an architecture for the data flow