New Geometric Data Structures for Collision Detection

We present new geometric data structures for collision detection and more, including: Inner Sphere Trees - the first data structure to compute the peneration volume efficiently. Protosphere - an new algorithm to compute space filling sphere packings for arbitrary objects. Kinetic AABBs - a bounding volume hierarchy that is optimal in the number of updates when the objects deform. Kinetic Separation-List - an algorithm that is able to perform continuous collision detection for complex deformable objects in real-time. Moreover, we present applications of these new approaches to hand animation, real-time collision avoidance in dynamic environments for robots and haptic rendering, including a user study that exploits the influence of the degrees of freedom in complex haptic interactions. Last but not least, we present a new benchmarking suite for both, peformance and quality benchmarks, and a theoretic analysis of the running-time of bounding volume-based collision detection algorithms

Novel Bidirectional Body - Machine Interface to Control Upper Limb Prosthesis

Objective. The journey of a bionic prosthetic user is characterized by the opportunities and limitations involved in adopting a device (the prosthesis) that should enable activities of daily living (ADL). Within this context, experiencing a bionic hand as a functional (and, possibly, embodied) limb constitutes the premise for mitigating the risk of its abandonment through the continuous use of the device. To achieve such a result, different aspects must be considered for making the artificial limb an effective support for carrying out ADLs. Among them, intuitive and robust control is fundamental to improving amputees’ quality of life using upper limb prostheses. Still, as artificial proprioception is essential to perceive the prosthesis movement without constant visual attention, a good control framework may not be enough to restore practical functionality to the limb. To overcome this, bidirectional communication between the user and the prosthesis has been recently introduced and is a requirement of utmost importance in developing prosthetic hands. Indeed, closing the control loop between the user and a prosthesis by providing artificial sensory feedback is a fundamental step towards the complete restoration of the lost sensory-motor functions. Within my PhD work, I proposed the development of a more controllable and sensitive human-like hand prosthesis, i.e., the Hannes prosthetic hand, to improve its usability and effectiveness. Approach. To achieve the objectives of this thesis work, I developed a modular and scalable software and firmware architecture to control the Hannes prosthetic multi-Degree of Freedom (DoF) system and to fit all users’ needs (hand aperture, wrist rotation, and wrist flexion in different combinations). On top of this, I developed several Pattern Recognition (PR) algorithms to translate electromyographic (EMG) activity into complex movements. However, stability and repeatability were still unmet requirements in multi-DoF upper limb systems; hence, I started by investigating different strategies to produce a more robust control. To do this, EMG signals were collected from trans-radial amputees using an array of up to six sensors placed over the skin. Secondly, I developed a vibrotactile system to implement haptic feedback to restore proprioception and create a bidirectional connection between the user and the prosthesis. Similarly, I implemented an object stiffness detection to restore tactile sensation able to connect the user with the external word. This closed-loop control between EMG and vibration feedback is essential to implementing a Bidirectional Body - Machine Interface to impact amputees’ daily life strongly. For each of these three activities: (i) implementation of robust pattern recognition control algorithms, (ii) restoration of proprioception, and (iii) restoration of the feeling of the grasped object's stiffness, I performed a study where data from healthy subjects and amputees was collected, in order to demonstrate the efficacy and usability of my implementations. In each study, I evaluated both the algorithms and the subjects’ ability to use the prosthesis by means of the F1Score parameter (offline) and the Target Achievement Control test-TAC (online). With this test, I analyzed the error rate, path efficiency, and time efficiency in completing different tasks. Main results. Among the several tested methods for Pattern Recognition, the Non-Linear Logistic Regression (NLR) resulted to be the best algorithm in terms of F1Score (99%, robustness), whereas the minimum number of electrodes needed for its functioning was determined to be 4 in the conducted offline analyses. Further, I demonstrated that its low computational burden allowed its implementation and integration on a microcontroller running at a sampling frequency of 300Hz (efficiency). Finally, the online implementation allowed the subject to simultaneously control the Hannes prosthesis DoFs, in a bioinspired and human-like way. In addition, I performed further tests with the same NLR-based control by endowing it with closed-loop proprioceptive feedback. In this scenario, the results achieved during the TAC test obtained an error rate of 15% and a path efficiency of 60% in experiments where no sources of information were available (no visual and no audio feedback). Such results demonstrated an improvement in the controllability of the system with an impact on user experience. Significance. The obtained results confirmed the hypothesis of improving robustness and efficiency of a prosthetic control thanks to of the implemented closed-loop approach. The bidirectional communication between the user and the prosthesis is capable to restore the loss of sensory functionality, with promising implications on direct translation in the clinical practice

Exploring AI-enhanced Shared Control for an Assistive Robotic Arm

Assistive technologies and in particular assistive robotic arms have the potential to enable people with motor impairments to live a self-determined life. More and more of these systems have become available for end users in recent years, such as the Kinova Jaco robotic arm. However, they mostly require complex manual control, which can overwhelm users. As a result, researchers have explored ways to let such robots act autonomously. However, at least for this specific group of users, such an approach has shown to be futile. Here, users want to stay in control to achieve a higher level of personal autonomy, to which an autonomous robot runs counter. In our research, we explore how Artifical Intelligence (AI) can be integrated into a shared control paradigm. In particular, we focus on the consequential requirements for the interface between human and robot and how we can keep humans in the loop while still significantly reducing the mental load and required motor skills.Comment: Workshop on Engineering Interactive Systems Embedding AI Technologies (EIS-embedding-AI) at EICS'2

Anthropomorphic surgical system for soft tissue robot-assisted surgery

Over the past century, abdominal surgery has seen a rapid transition from open procedures to less invasive methods such as laparoscopy and robot-assisted minimally invasive surgery (R-A MIS). These procedures have significantly decreased blood loss, postoperative morbidity and length of hospital stay in comparison with open surgery. R-A MIS has offered refined accuracy and more ergonomic instruments for surgeons, further minimising trauma to the patient.This thesis aims to investigate, design and prototype a novel system for R-A MIS that will provide more natural and intuitive manipulation of soft tissues and, at the same time, increase the surgeon's dexterity. The thesis reviews related work on surgical systems and discusses the requirements for designing surgical instrumentation. From the background research conducted in this thesis, it is clear that training surgeons in MIS procedures is becoming increasingly long and arduous. Furthermore, most available systems adopt a design similar to conventional laparoscopic instruments or focus on different techniques with debatable benefits. The system proposed in this thesis not only aims to reduce the training time for surgeons but also to improve the ergonomics of the procedure.In order to achieve this, a survey was conducted among surgeons, regarding their opinions on surgical training, surgical systems, how satisfied they are with them and how easy they are to use. A concept for MIS robotic instrumentation was then developed and a series of focus group meetings with surgeons were run to discuss it. The proposed system, named microAngelo, is an anthropomorphic master-slave system that comprises a three-digit miniature hand that can be controlled using the master, a three-digit sensory exoskeleton. While multi-fingered robotic hands have been developed for decades, none have been used for surgical operations. As the system has a human centred design, its relation to the human hand is discussed. Prototypes of both the master and the slave have been developed and their design and mechanisms is demonstrated. The accuracy and repeatability of the master as well as the accuracy and force capabilities of the slave are tested and discussed

Robotic manipulators for single access surgery

This thesis explores the development of cooperative robotic manipulators for enhancing surgical precision and patient outcomes in single-access surgery and, specifically, Transanal Endoscopic Microsurgery (TEM). During these procedures, surgeons manipulate a heavy set of instruments via a mechanical clamp inserted in the patient’s body through a surgical port, resulting in imprecise movements, increased patient risks, and increased operating time. Therefore, an articulated robotic manipulator with passive joints is initially introduced, featuring built-in position and force sensors in each joint and electronic joint brakes for instant lock/release capability. The articulated manipulator concept is further improved with motorised joints, evolving into an active tool holder. The joints allow the incorporation of advanced robotic capabilities such as ultra-lightweight gravity compensation and hands-on kinematic reconfiguration, which can optimise the placement of the tool holder in the operating theatre. Due to the enhanced sensing capabilities, the application of the active robotic manipulator was further explored in conjunction with advanced image guidance approaches such as endomicroscopy. Recent advances in probe-based optical imaging such as confocal endomicroscopy is making inroads in clinical uses. However, the challenging manipulation of imaging probes hinders their practical adoption. Therefore, a combination of the fully cooperative robotic manipulator with a high-speed scanning endomicroscopy instrument is presented, simplifying the incorporation of optical biopsy techniques in routine surgical workflows. Finally, another embodiment of a cooperative robotic manipulator is presented as an input interface to control a highly-articulated robotic instrument for TEM. This master-slave interface alleviates the drawbacks of traditional master-slave devices, e.g., using clutching mechanics to compensate for the mismatch between slave and master workspaces, and the lack of intuitive manipulation feedback, e.g. joint limits, to the user. To address those drawbacks a joint-space robotic manipulator is proposed emulating the kinematic structure of the flexible robotic instrument under control.Open Acces

Control of Cooperative Haptics-Enabled Teleoperation Systems with Application to Minimally Invasive Surgery

Robot-Assisted Minimally Invasive Surgical (RAMIS) systems frequently have a structure of cooperative teleoperator systems where multiple master-slave pairs are used to collaboratively execute a task. Although multiple studies indicate that haptic feedback improves the realism of tool-tissue interaction to the surgeon and leads to better performance for surgical procedures, current telesurgical systems typically do not provide force feedback, mainly because of the inherent stability issues. The research presented in this thesis is directed towards the development of control algorithms for force reflecting cooperative surgical teleoperator systems with improved stability and transparency characteristics. In the case of cooperative force reflecting teleoperation over networks, conventional passivity based approaches may have limited applicability due to potentially non-passive slave-slave interactions and irregular communication delays imposed by the network. In this thesis, an alternative small gain framework for the design of cooperative network-based force reflecting teleoperator systems is developed. Using the small gain framework, control algorithms for cooperative force-reflecting teleoperator systems are designed that guarantee stability in the presence of multiple network-induced communication constraints. Furthermore, the design conservatism typically associated with the small-gain approach is eliminated by using the Projection-Based Force Reflection (PBFR) algorithms. Stability results are established for networked cooperative teleoperator systems under different types of force reflection algorithms in the presence of irregular communication delays. The proposed control approach is consequently implemented on a dual-arm (two masters/two slaves) robotic MIS testbed. The testbed consists of two Haptic Wand devices as masters and two PA10-7C robots as the slave manipulators equipped with da Vinci laparoscopic surgical instruments. The performance of the proposed control approach is evaluated in three different cooperative surgical tasks, which are knot tightening, pegboard transfer, and object manipulation. The experimental results obtained indicate that the PBFR algorithms demonstrate statistically significant performance improvement in comparison with the conventional direct force reflection algorithms. One possible shortcoming of using PBFR algorithms is that implementation of these algorithms may lead to attenuation of the high-frequency component of the contact force which is important, in particular, for haptic perception of stiff surfaces. In this thesis, a solution to this problem is proposed which is based on the idea of separating the different frequency bands in the force reflection signal and consequently applying the projection-based principle to the low-frequency component, while reflecting the high-frequency component directly. The experimental results demonstrate that substantial improvement in transient fidelity of the force feedback is achieved using the proposed method without negative effects on the stability of the system

Flexible robotic device for spinal surgery

Surgical robots have proliferated in recent years, with well-established benefits including: reduced patient trauma, shortened hospitalisation, and improved diagnostic accuracy and therapeutic outcome. Despite these benefits, many challenges in their development remain, including improved instrument control and ergonomics caused by rigid instrumentation and its associated fulcrum effect. Consequently, it is still extremely challenging to utilise such devices in cases that involve complex anatomical pathways such as the spinal column. The focus of this thesis is the development of a flexible robotic surgical cutting device capable of manoeuvring around the spinal column. The target application of the flexible surgical tool is the removal of cancerous tumours surrounding the spinal column, which cannot be excised completely using the straight surgical tools in use today; anterior and posterior sections of the spine must be accessible for complete tissue removal. A parallel robot platform with six degrees of freedom (6 DoFs) has been designed and fabricated to direct a flexible cutting tool to produce the necessary range of movements to reach anterior and posterior sections of the spinal column. A flexible water jet cutting system and a flexible mechanical drill, which may be assembled interchangeably with the flexible probe, have been developed and successfully tested experimentally. A model predicting the depth of cut by the water jet was developed and experimentally validated. A flexion probe that is able to guide the surgical cutting device around the spinal column has been fabricated and tested with human lumber model. Modelling and simulations show the capacity for the flexible surgical system to enable entering the posterior side of the human lumber model and bend around the vertebral body to reach the anterior side of the spinal column. A computer simulation with a full Graphical User Interface (GUI) was created and used to validate the system of inverse kinematic equations for the robot platform. The constraint controller and the inverse kinematics relations are both incorporated into the overall positional control structure of the robot, and have successfully established a haptic feedback controller for the 6 DoFs surgical probe, and effectively tested in vitro on spinal mock surgery. The flexible surgical system approached the surgery from the posterior side of the human lumber model and bend around the vertebral body to reach the anterior side of the spinal column. The flexible surgical robot removed 82% of mock cancerous tissue compared to 16% of tissue removed by the rigid tool.Open Acces

A Wearable Exoskeleton for Hand Kinesthetic Feedback in Virtual Reality

This paper presents a novel two-fingers exoskeleton kinesthetic in-teraction in Virtual Reality (VR): the proposed design of the exoskeleton priori-tizes the performance of the device in terms of low weight, good adaptability to different size of the human hand. This design made also the exoskeleton well wearable and allows strong force feedback which is an important parameter for a realistic kinesthesis of manipulated objects in VR

Advancing Medical Technology for Motor Impairment Rehabilitation: Tools, Protocols, and Devices

Excellent motor control skills are necessary to live a high-quality life. Activities such as walking, getting dressed, and feeding yourself may seem mundane, but injuries to the neuromuscular system can render these tasks difficult or even impossible to accomplish without assistance. Statistics indicate that well over 100 million people are affected by diseases or injuries, such as stroke, Parkinson’s Disease, Multiple Sclerosis, Cerebral Palsy, peripheral nerve injury, spinal cord injury, and amputation, that negatively impact their motor abilities. This wide array of injuries presents a challenge to the medical field as optimal treatment paradigms are often difficult to implement due to a lack of availability of appropriate assessment tools, the inability for people to access the appropriate medical centers for treatment, or altogether gaps in technology for treating the underlying impairments causing the disability. Addressing each of these challenges will improve the treatment of movement impairments, provide more customized and continuous treatment to a larger number of patients, and advance rehabilitative and assistive device technology. In my research, the key approach was to develop tools to assess and treat upper extremity movement impairment. In Chapter 2.1, I challenged a common biomechanical[GV1] modeling technique of the forearm. Comparing joint torque values through inverse dynamics simulation between two modeling platforms, I discovered that representing the forearm as a single cylindrical body was unable to capture the inertial parameters of a physiological forearm which is made up of two segments, the radius and ulna. I split the forearm segment into a proximal and distal segment, with the rationale being that the inertial parameters of the proximal segment could be tuned to those of the ulna and the inertial parameters of the distal segment could be tuned to those of the radius. Results showed a marked increase in joint torque calculation accuracy for those degrees of freedom that are affected by the inertial parameters of the radius and ulna. In Chapter 2.2, an inverse kinematic upper extremity model was developed for joint angle calculations from experimental motion capture data, with the rationale being that this would create an easy-to-use tool for clinicians and researchers to process their data. The results show accurate angle calculations when compared to algebraic solutions. Together, these chapters provide easy-to-use models and tools for processing movement assessment data. In Chapter 3.1, I developed a protocol to collect high-quality movement data in a virtual reality task that is used to assess hand function as part of a Box and Block Test. The goal of this chapter is to suggest a method to not only collect quality data in a research setting but can also be adapted for telehealth and at home movement assessment and rehabilitation. Results indicate that the data collected in this protocol are good and the virtual nature of this approach can make it a useful tool for continuous, data driven care in clinic or at home. In Chapter 3.2 I developed a high-density electromyography device for collecting motor unit action potentials of the arm. Traditional surface electromyography is limited by its ability to obtain signals from deep muscles and can also be time consuming to selectively place over appropriate muscles. With this high-density approach, muscle coverage is increased, placement time is decreased, and deep muscle activity can potentially be collected due to the high-density nature of the device[GV2] . Furthermore, the high-density electromyography device is built as a precursor to a high-density electromyography-electrical stimulation device for functional electrical stimulation. The customizable nature of the prototype in Chapter 3.2 allows for the implementation both recording and stimulating electrodes. Furthermore, signal results show that the electromyography data obtained from the device are of high quality and are correlated with gold standard surface electromyography sensors. One key factor in a device that can record and then stimulate based on the information from the recorded signals is an accurate movement intent decoder. High-quality movement decoders have been designed by closed-loop device controllers in the past, but they still struggle when the user interacts with objects of varying weight due to underlying alterations in muscle signals. In Chapter 4, I investigate this phenomenon by administering an experiment where participants perform a Box and Block Task with objects of 3 different weights, 0 kg, 0.02 kg, and 0.1 kg. Electromyography signals of the participants right arm were collected and co-contraction levels between antagonistic muscles were analyzed to uncover alterations in muscle forces and joint dynamics. Results indicated contraction differences between the conditions and also between movement stages (contraction levels before grabbing the block vs after touching the block) for each condition. This work builds a foundation for incorporating object weight estimates into closed-loop electromyography device movement decoders. Overall, we believe the chapters in this thesis provide a basis for increasing availability to movement assessment tools, increasing access to effective movement assessment and rehabilitation, and advance the medical device and technology field