Surgical Phase Recognition of Short Video Shots Based on Temporal Modeling of Deep Features

Recognizing the phases of a laparoscopic surgery (LS) operation form its video constitutes a fundamental step for efficient content representation, indexing and retrieval in surgical video databases. In the literature, most techniques focus on phase segmentation of the entire LS video using hand-crafted visual features, instrument usage signals, and recently convolutional neural networks (CNNs). In this paper we address the problem of phase recognition of short video shots (10s) of the operation, without utilizing information about the preceding/forthcoming video frames, their phase labels or the instruments used. We investigate four state-of-the-art CNN architectures (Alexnet, VGG19, GoogleNet, and ResNet101), for feature extraction via transfer learning. Visual saliency was employed for selecting the most informative region of the image as input to the CNN. Video shot representation was based on two temporal pooling mechanisms. Most importantly, we investigate the role of 'elapsed time' (from the beginning of the operation), and we show that inclusion of this feature can increase performance dramatically (69% vs. 75% mean accuracy). Finally, a long short-term memory (LSTM) network was trained for video shot classification based on the fusion of CNN features with 'elapsed time', increasing the accuracy to 86%. Our results highlight the prominent role of visual saliency, long-range temporal recursion and 'elapsed time' (a feature so far ignored), for surgical phase recognition.Comment: 6 pages, 4 figures, 6 table

Hierarchical, informed and robust machine learning for surgical tool management

This thesis focuses on the development of a computer vision and deep learning based system for the intelligent management of surgical tools. The work accomplished included the development of a new dataset, creation of state of the art techniques to cope with volume, variety and vision problems, and designing or adapting algorithms to address specific surgical tool recognition issues. The system was trained to cope with a wide variety of tools, with very subtle differences in shapes, and was designed to work with high volumes, as well as varying illuminations and backgrounds. Methodology that was adopted in this thesis included the creation of a surgical tool image dataset and development of a surgical tool attribute matrix or knowledge-base. This was significant because there are no large scale publicly available surgical tool datasets, nor are there established annotations or datasets of textual descriptions of surgical tools that can be used for machine learning. The work resulted in the development of a new hierarchical architecture for multi-level predictions at surgical speciality, pack, set and tool level. Additional work evaluated the use of synthetic data to improve robustness of the CNN, and the infusion of knowledge to improve predictive performance

Diagnosis of Cervical Cancer and Pre-Cancerous Lesions by Artificial Intelligence: A Systematic Review

The likelihood of timely treatment for cervical cancer increases with timely detection of abnormal cervical cells. Automated methods of detecting abnormal cervical cells were established because manual identification requires skilled pathologists and is time consuming and prone to error. The purpose of this systematic review is to evaluate the diagnostic performance of artificial intelligence (AI) technologies for the prediction, screening, and diagnosis of cervical cancer and pre-cancerous lesions

Recent Advances in Minimally Invasive Surgery

Minimally invasive surgery has become a common term in visceral as well as gynecologic surgery. It has almost evolved into its own surgical speciality over the past 20 years. Today, being firmly established in every subspeciality of visceral surgery, it is now no longer a distinct skillset, but a fixed part of the armamentarium of surgical options available. In every indication, the advantages of a minimally invasive approach include reduced intraoperative blood loss, less postoperative pain, and shorter rehabilitation times, as well as a marked reduction of overall and surgical postoperative morbidity. In the advent of modern oncologic treatment algorithms, these effects not only lower the immediate impact that an operation has on the patient, but also become important key steps in reducing the side-effects of surgery. Thus, they enable surgery to become a module in modern multi-disciplinary cancer treatment, which blends into multimodular treatment options at different times and prolongs and widens the possibilities available to cancer patients. In this quickly changing environment, the requirement to learn and refine not only open surgical but also different minimally invasive techniques on high levels deeply impact modern surgical training pathways. The use of modern elearning tools and new and praxis-based surgical training possibilities have been readily integrated into modern surgical education,which persists throughout the whole surgical career of modern gynecologic and visceral surgery specialists

Robot Assisted Object Manipulation for Minimally Invasive Surgery

Robotic systems have an increasingly important role in facilitating minimally invasive surgical treatments. In robot-assisted minimally invasive surgery, surgeons remotely control instruments from a console to perform operations inside the patient. However, despite the advanced technological status of surgical robots, fully autonomous systems, with decision-making capabilities, are not yet available. In 2017, a structure to classify the research efforts toward autonomy achievable with surgical robots was proposed by Yang et al. Six different levels were identified: no autonomy, robot assistance, task autonomy, conditional autonomy, high autonomy, and full autonomy. All the commercially available platforms in robot-assisted surgery is still in level 0 (no autonomy). Despite increasing the level of autonomy remains an open challenge, its adoption could potentially introduce multiple benefits, such as decreasing surgeons’ workload and fatigue and pursuing a consistent quality of procedures. Ultimately, allowing the surgeons to interpret the ample and intelligent information from the system will enhance the surgical outcome and positively reflect both on patients and society. Three main aspects are required to introduce automation into surgery: the surgical robot must move with high precision, have motion planning capabilities and understand the surgical scene. Besides these main factors, depending on the type of surgery, there could be other aspects that might play a fundamental role, to name some compliance, stiffness, etc. This thesis addresses three technological challenges encountered when trying to achieve the aforementioned goals, in the specific case of robot-object interaction. First, how to overcome the inaccuracy of cable-driven systems when executing fine and precise movements. Second, planning different tasks in dynamically changing environments. Lastly, how the understanding of a surgical scene can be used to solve more than one manipulation task. To address the first challenge, a control scheme relying on accurate calibration is implemented to execute the pick-up of a surgical needle. Regarding the planning of surgical tasks, two approaches are explored: one is learning from demonstration to pick and place a surgical object, and the second is using a gradient-based approach to trigger a smoother object repositioning phase during intraoperative procedures. Finally, to improve scene understanding, this thesis focuses on developing a simulation environment where multiple tasks can be learned based on the surgical scene and then transferred to the real robot. Experiments proved that automation of the pick and place task of different surgical objects is possible. The robot was successfully able to autonomously pick up a suturing needle, position a surgical device for intraoperative ultrasound scanning and manipulate soft tissue for intraoperative organ retraction. Despite automation of surgical subtasks has been demonstrated in this work, several challenges remain open, such as the capabilities of the generated algorithm to generalise over different environment conditions and different patients

CREATE: Concept Representation and Extraction from Heterogeneous Evidence

Traditional information retrieval methodology is guided by document retrieval paradigm, where relevant documents are returned in response to user queries. This paradigm faces serious drawback if the desired result is not explicitly present in a single document. The problem becomes more obvious when a user tries to obtain complete information about a real world entity, such as person, company, location etc. In such cases, various facts about the target entity or concept need to be gathered from multiple document sources. In this work, we present a method to extract information about a target entity based on the concept retrieval paradigm that focuses on extracting and blending information related to a concept from multiple sources if necessary. The paradigm is built around a generic notion of concept which is defined as any item that can be thought of as a topic of interest. Concepts may correspond to any real world entity such as restaurant, person, city, organization, etc, or any abstract item such as news topic, event, theory, etc. Web is a heterogeneous collection of data in different forms such as facts, news, opinions etc. We propose different models for different forms of data, all of which work towards the same goal of concept centric retrieval. We motivate our work based on studies about current trends and demands for information seeking. The framework helps in understanding the intent of content, i.e. opinion versus fact. Our work has been conducted on free text data in English. Nevertheless, our framework can be easily transferred to other languages

Qualitative Study of Factors Contributing to Fertility Service Use Among Cancer Survivors of Reproductive Age in the US

Cancer remains the second leading cause of death in the United States; however, there has been a decline in incidence and mortality due to advances in screening and treatment. Currently 16.9 million survivors are thriving within the United States, and the population of cancer survivors has been projected to grow to 22.2 million by 2030. Although cancer survivors report an increased surge of vitality and vigor, they often face physical, mental, psychosocial, or financial challenges that threaten their quality of life. A late treatment effect of particular concern for cancer survivors of reproductive age that has both physical and psychosocial implications is infertility. Current guidelines are in place to ensure that survivors are made aware of how treatment can impact their plans for building a family and pose viable options for preserving their fertility; however, studies indicate under-utilization of fertility preservation among cancer survivors. The aim of this study was to examine how a cancer diagnosis influenced parenthood motivation and family building strategies among individuals of reproductive age. The study identified contributors to access and use of fertility services by using a conceptual model based on Andersen’s Behavioral Model of Health Services, Lea’s model of Age-appropriate Care, and Kilbourne’s Health Services Research Framework to Advance Health Disparities. The conceptual model elucidates factors that impact cancer survivors’ use of fertility services, equitable access to services, and policy actions that assure equitable access to fertility services. A phenomenological qualitative study was conducted to allow deep exploration of these issues through the review and analysis of survivors’ stories as told through online narratives and in-depth interviews. Study findings showed a cancer diagnosis had not altered an individual’s desire to have children. Although fertility preservation was not always utilized by cancer survivors of reproductive age to conceive or sire a child, most survivors sought out some form of fertility service post treatment. Access and subsequent use of fertility services was grossly dependent on both survivor and provider factors as well as successful clinical encounters between the patient and the provider. These findings have implications for clinical care, public health policy, practice, and research