Deep-learning-based instrument detection for intra-operative robotic assistance

Purpose: Robotic scrub nurses have the potential to become an attractive solution for the operating room. Surgical instrument detection is a fundamental task for these systems, which is the focus of this work. We address the detection of the complete surgery set for wisdom teeth extraction, and propose a data augmentation technique tailored for this task. Methods: Using a robotic scrub nurse system, we create a dataset of 369 unique multi-instrument images with manual annotations. We then propose the Mask-Based Object Insertion method, capable of automatically generating a large amount of synthetic images. By using both real and artificial data, different Mask R-CNN models are trained and evaluated. Results: Our experiments reveal that models trained on the synthetic data created with our method achieve comparable performance to that of models trained on real images. Moreover, we demonstrate that the combination of real and our artificial data can lead to a superior level of generalization. Conclusion: The proposed data augmentation technique is capable of dramatically reducing the labelling work required for training a deep-learning-based detection algorithm. A dataset for the complete instrument set for wisdom teeth extraction is made available for the scientific community, as well as the raw information required for the generation of the synthetic data (https://github.com/Jorebs/Deep-learning-based-instrument-detection-for-intra operative-robotic-assistance)

Hierarchical, informed and robust machine learning for surgical tool management

This thesis focuses on the development of a computer vision and deep learning based system for the intelligent management of surgical tools. The work accomplished included the development of a new dataset, creation of state of the art techniques to cope with volume, variety and vision problems, and designing or adapting algorithms to address specific surgical tool recognition issues. The system was trained to cope with a wide variety of tools, with very subtle differences in shapes, and was designed to work with high volumes, as well as varying illuminations and backgrounds. Methodology that was adopted in this thesis included the creation of a surgical tool image dataset and development of a surgical tool attribute matrix or knowledge-base. This was significant because there are no large scale publicly available surgical tool datasets, nor are there established annotations or datasets of textual descriptions of surgical tools that can be used for machine learning. The work resulted in the development of a new hierarchical architecture for multi-level predictions at surgical speciality, pack, set and tool level. Additional work evaluated the use of synthetic data to improve robustness of the CNN, and the infusion of knowledge to improve predictive performance

A gaze-contingent framework for perceptually-enabled applications in healthcare

Patient safety and quality of care remain the focus of the smart operating room of the future. Some of the most influential factors with a detrimental effect are related to suboptimal communication among the staff, poor flow of information, staff workload and fatigue, ergonomics and sterility in the operating room. While technological developments constantly transform the operating room layout and the interaction between surgical staff and machinery, a vast array of opportunities arise for the design of systems and approaches, that can enhance patient safety and improve workflow and efficiency. The aim of this research is to develop a real-time gaze-contingent framework towards a "smart" operating suite, that will enhance operator's ergonomics by allowing perceptually-enabled, touchless and natural interaction with the environment. The main feature of the proposed framework is the ability to acquire and utilise the plethora of information provided by the human visual system to allow touchless interaction with medical devices in the operating room. In this thesis, a gaze-guided robotic scrub nurse, a gaze-controlled robotised flexible endoscope and a gaze-guided assistive robotic system are proposed. Firstly, the gaze-guided robotic scrub nurse is presented; surgical teams performed a simulated surgical task with the assistance of a robot scrub nurse, which complements the human scrub nurse in delivery of surgical instruments, following gaze selection by the surgeon. Then, the gaze-controlled robotised flexible endoscope is introduced; experienced endoscopists and novice users performed a simulated examination of the upper gastrointestinal tract using predominately their natural gaze. Finally, a gaze-guided assistive robotic system is presented, which aims to facilitate activities of daily living. The results of this work provide valuable insights into the feasibility of integrating the developed gaze-contingent framework into clinical practice without significant workflow disruptions.Open Acces

Artificial intelligence applications and cataract management: A systematic review

Artificial intelligence (AI)-based applications exhibit the potential to improve the quality and efficiency of patient care in different fields, including cataract management. A systematic review of the different applications of AI-based software on all aspects of a cataract patient's management, from diagnosis to follow-up, was carried out in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. All selected articles were analyzed to assess the level of evidence according to the Oxford Centre for Evidence-Based Medicine 2011 guidelines, and the quality of evidence according to the Grading of Recommendations Assessment, Development and Evaluation system. Of the articles analyzed, 49 met the inclusion criteria. No data synthesis was possible for the heterogeneity of available data and the design of the available studies. The AI-driven diagnosis seemed to be comparable and, in selected cases, to even exceed the accuracy of experienced clinicians in classifying disease, supporting the operating room scheduling, and intraoperative and postoperative management of complications. Considering the heterogeneity of data analyzed, however, further randomized controlled trials to assess the efficacy and safety of AI application in the management of cataract should be highly warranted

Advanced Applications of Rapid Prototyping Technology in Modern Engineering

Rapid prototyping (RP) technology has been widely known and appreciated due to its flexible and customized manufacturing capabilities. The widely studied RP techniques include stereolithography apparatus (SLA), selective laser sintering (SLS), three-dimensional printing (3DP), fused deposition modeling (FDM), 3D plotting, solid ground curing (SGC), multiphase jet solidification (MJS), laminated object manufacturing (LOM). Different techniques are associated with different materials and/or processing principles and thus are devoted to specific applications. RP technology has no longer been only for prototype building rather has been extended for real industrial manufacturing solutions. Today, the RP technology has contributed to almost all engineering areas that include mechanical, materials, industrial, aerospace, electrical and most recently biomedical engineering. This book aims to present the advanced development of RP technologies in various engineering areas as the solutions to the real world engineering problems

Body sensor networks: smart monitoring solutions after reconstructive surgery

Advances in reconstructive surgery are providing treatment options in the face of major trauma and cancer. Body Sensor Networks (BSN) have the potential to offer smart solutions to a range of clinical challenges. The aim of this thesis was to review the current state of the art devices, then develop and apply bespoke technologies developed by the Hamlyn Centre BSN engineering team supported by the EPSRC ESPRIT programme to deliver post-operative monitoring options for patients undergoing reconstructive surgery. A wireless optical sensor was developed to provide a continuous monitoring solution for free tissue transplants (free flaps). By recording backscattered light from 2 different source wavelengths, we were able to estimate the oxygenation of the superficial microvasculature. In a custom-made upper limb pressure cuff model, forearm deoxygenation measured by our sensor and gold standard equipment showed strong correlations, with incremental reductions in response to increased cuff inflation durations. Such a device might allow early detection of flap failure, optimising the likelihood of flap salvage. An ear-worn activity recognition sensor was utilised to provide a platform capable of facilitating objective assessment of functional mobility. This work evolved from an initial feasibility study in a knee replacement cohort, to a larger clinical trial designed to establish a novel mobility score in patients recovering from open tibial fractures (OTF). The Hamlyn Mobility Score (HMS) assesses mobility over 3 activities of daily living: walking, stair climbing, and standing from a chair. Sensor-derived parameters including variation in both temporal and force aspects of gait were validated to measure differences in performance in line with fracture severity, which also matched questionnaire-based assessments. Monitoring the OTF cohort over 12 months with the HMS allowed functional recovery to be profiled in great detail. Further, a novel finding of continued improvements in walking quality after a plateau in walking quantity was demonstrated objectively. The methods described in this thesis provide an opportunity to revamp the recovery paradigm through continuous, objective patient monitoring along with self-directed, personalised rehabilitation strategies, which has the potential to improve both the quality and cost-effectiveness of reconstructive surgery services.Open Acces

Advanced Materials for Oral Application

This book consists of one editorial, 12 original research articles and two review papers from scientists across the world, with expertise in materials for dental application. The main subjects covered are: biomaterials and techniques for oral tissue engineering and regeneration; biomaterials for surgical reconstruction; CAD/CAM technologies and dedicated materials; novel restorative and endodontic materials and instruments

Narrative Review: Food Image Use for Machine Learnings’ Function in Dietary Assessment and Real Time Nutrition Feedback and Education

Technology has played a key role in advancing the health and agriculture sectors to improve obesity rates, diseasecontrol, food waste, and overall health disparities. However, these health and lifestyle determinants continue to plague theUnited States population. While new technologies have been and are currently being developed to address these concerns, they may not be practical for the general population. Utilizing machine learning advancement in food recognition using smartphone technology may be a means to improve the dietary component of nutrition assessments while providing valuable nutrition feedback. This narrative review was conducted to assess the current state of the literature on nutrition technology using image recognition for practical applications, while also proposing theoretical uses for the technology to improve quality of life through dietary feedback