A gaze-contingent framework for perceptually-enabled applications in healthcare

Patient safety and quality of care remain the focus of the smart operating room of the future. Some of the most influential factors with a detrimental effect are related to suboptimal communication among the staff, poor flow of information, staff workload and fatigue, ergonomics and sterility in the operating room. While technological developments constantly transform the operating room layout and the interaction between surgical staff and machinery, a vast array of opportunities arise for the design of systems and approaches, that can enhance patient safety and improve workflow and efficiency. The aim of this research is to develop a real-time gaze-contingent framework towards a "smart" operating suite, that will enhance operator's ergonomics by allowing perceptually-enabled, touchless and natural interaction with the environment. The main feature of the proposed framework is the ability to acquire and utilise the plethora of information provided by the human visual system to allow touchless interaction with medical devices in the operating room. In this thesis, a gaze-guided robotic scrub nurse, a gaze-controlled robotised flexible endoscope and a gaze-guided assistive robotic system are proposed. Firstly, the gaze-guided robotic scrub nurse is presented; surgical teams performed a simulated surgical task with the assistance of a robot scrub nurse, which complements the human scrub nurse in delivery of surgical instruments, following gaze selection by the surgeon. Then, the gaze-controlled robotised flexible endoscope is introduced; experienced endoscopists and novice users performed a simulated examination of the upper gastrointestinal tract using predominately their natural gaze. Finally, a gaze-guided assistive robotic system is presented, which aims to facilitate activities of daily living. The results of this work provide valuable insights into the feasibility of integrating the developed gaze-contingent framework into clinical practice without significant workflow disruptions.Open Acces

Three-Axis Fiber-Optic Body Force Sensor for Flexible Manipulators

This paper proposes a force/torque sensor structure that can be easily integrated with a flexible manipulator structure. The sensor’s ring-like structure with its hollow inner section provides ample space for auxiliary components, such as cables and tubes, to be passed through and, hence, is very suitable for integration with tendon-driven and fluid-actuated manipulators. The sensor structure can also accommodate the wiring for a distributed sensor system as well as for diagnostic instruments that may be incorporated in the manipulator. Employing a sensing approach based on optical fibers as done here allows for the creation of sensors that are free of electrical currents at the point of sensing and immune to magnetic fields. These sensors are inherently safe when used in the close vicinity of humans and their measuring performance is not impaired when they are operated in or nearby machines such as magnetic resonance imaging (MRI) scanners. This type of sensor concept is particularly suitable for inclusion in instruments and robotic tools for minimally invasive surgery (MIS). The paper summarizes the design, integration challenges and calibration of the proposed optical three-axis force sensor. The experimental results confirm the effectiveness of our optical sensing approach and show that after calibrating its stiffness matrix, force and momentum components can be determined accurately

Three-Axis Fiber-Optic Body Force Sensor for Flexible Manipulators

This paper proposes a force/torque sensor structure that can be easily integrated into a flexible manipulator structure. The sensor's ring-like structure with its hollow inner section provides ample space for auxiliary components, such as cables and tubes, to be passed through and, hence, is very suitable for integration with tendon-driven and fluid-actuated manipulators. The sensor structure can also accommodate the wiring for a distributed sensor system as well as for diagnostic instruments that may be incorporated in the manipulator. Employing a sensing approach based on optical fibers as done here allows for the creation of sensors that are free of electrical currents at the point of sensing and immune to magnetic fields. These sensors are inherently safe when used in the close vicinity of humans and their measuring performance is not impaired when they are operated in or nearby machines, such as magnetic resonance imaging scanners. This type of sensor concept is particularly suitable for inclusion in instruments and robotic tools for minimally invasive surgery. This paper summarizes the design, integration challenges, and calibration of the proposed optical three-axis force sensor. The experimental results confirm the effectiveness of our optical sensing approach and show that after calibrating its stiffness matrix, force and momentum components can be determined accurately

Gaze-contingent perceptually enabled interactions in the operating theatre.

PURPOSE: Improved surgical outcome and patient safety in the operating theatre are constant challenges. We hypothesise that a framework that collects and utilises information -especially perceptually enabled ones-from multiple sources, could help to meet the above goals. This paper presents some core functionalities of a wider low-cost framework under development that allows perceptually enabled interaction within the surgical environment. METHODS: The synergy of wearable eye-tracking and advanced computer vision methodologies, such as SLAM, is exploited. As a demonstration of one of the framework's possible functionalities, an articulated collaborative robotic arm and laser pointer is integrated and the set-up is used to project the surgeon's fixation point in 3D space. RESULTS: The implementation is evaluated over 60 fixations on predefined targets, with distances between the subject and the targets of 92-212 cm and between the robot and the targets of 42-193 cm. The median overall system error is currently 3.98 cm. Its real-time potential is also highlighted. CONCLUSIONS: The work presented here represents an introduction and preliminary experimental validation of core functionalities of a larger framework under development. The proposed framework is geared towards a safer and more efficient surgical theatre

Smart Camera Robotic Assistant for Laparoscopic Surgery

The cognitive architecture also includes learning mechanisms to adapt the behavior of the robot to the different ways of working of surgeons, and to improve the robot behavior through experience, in a similar way as a human assistant would do. The theoretical concepts of this dissertation have been validated both through in-vitro experimentation in the labs of medical robotics of the University of Malaga and through in-vivo experimentation with pigs in the IACE Center (Instituto Andaluz de Cirugía Experimental), performed by expert surgeons.In the last decades, laparoscopic surgery has become a daily practice in operating rooms worldwide, which evolution is tending towards less invasive techniques. In this scenario, robotics has found a wide field of application, from slave robotic systems that replicate the movements of the surgeon to autonomous robots able to assist the surgeon in certain maneuvers or to perform autonomous surgical tasks. However, these systems require the direct supervision of the surgeon, and its capacity of making decisions and adapting to dynamic environments is very limited. This PhD dissertation presents the design and implementation of a smart camera robotic assistant to collaborate with the surgeon in a real surgical environment. First, it presents the design of a novel camera robotic assistant able to augment the capacities of current vision systems. This robotic assistant is based on an intra-abdominal camera robot, which is completely inserted into the patient’s abdomen and it can be freely moved along the abdominal cavity by means of magnetic interaction with an external magnet. To provide the camera with the autonomy of motion, the external magnet is coupled to the end effector of a robotic arm, which controls the shift of the camera robot along the abdominal wall. This way, the robotic assistant proposed in this dissertation has six degrees of freedom, which allow providing a wider field of view compared to the traditional vision systems, and also to have different perspectives of the operating area. On the other hand, the intelligence of the system is based on a cognitive architecture specially designed for autonomous collaboration with the surgeon in real surgical environments. The proposed architecture simulates the behavior of a human assistant, with a natural and intuitive human-robot interface for the communication between the robot and the surgeon