Real-time deep learning semantic segmentation during intra-operative surgery for 3D augmented reality assistance

The current study aimed to propose a Deep Learning (DL) and Augmented Reality (AR) based solution for a in-vivo robot-assisted radical prostatectomy (RARP), to improve the precision of a published work from our group. We implemented a two-steps automatic system to align a 3D virtual ad-hoc model of a patient's organ with its 2D endoscopic image, to assist surgeons during the procedure

6D object position estimation from 2D images: a literature review

The 6D pose estimation of an object from an image is a central problem in many domains of Computer Vision (CV) and researchers have struggled with this issue for several years. Traditional pose estimation methods (1) leveraged on geometrical approaches, exploiting manually annotated local features, or (2) relied on 2D object representations from different points of view and their comparisons with the original image. The two methods mentioned above are also known as Feature-based and Template-based, respectively. With the diffusion of Deep Learning (DL), new Learning-based strategies have been introduced to achieve the 6D pose estimation, improving traditional methods by involving Convolutional Neural Networks (CNN). This review analyzed techniques belonging to different research fields and classified them into three main categories: Template-based methods, Feature-based methods, and Learning-Based methods. In recent years, the research mainly focused on Learning-based methods, which allow the training of a neural network tailored for a specific task. For this reason, most of the analyzed methods belong to this category, and they have been in turn classified into three sub-categories: Bounding box prediction and Perspective-n-Point (PnP) algorithm-based methods, Classification-based methods, and Regression-based methods. This review aims to provide a general overview of the latest 6D pose recovery methods to underline the pros and cons and highlight the best-performing techniques for each group. The main goal is to supply the readers with helpful guidelines for the implementation of performing applications even under challenging circumstances such as auto-occlusions, symmetries, occlusions between multiple objects, and bad lighting conditions

X-Ray Bone Fracture Classification Using Deep Learning: A Baseline for Designing a Reliable Approach

In recent years, bone fracture detection and classification has been a widely discussed topic and many researchers have proposed different methods to tackle this problem. Despite this, a universal approach able to classify all the fractures in the human body has not yet been defined. We aim to analyze and evaluate a selection of papers, chosen according to their representative approach, where the authors applied different deep learning techniques to classify bone fractures, in order to select the strengths of each of them and try to delineate a generalized strategy. Each study is summarized and evaluated using a radar graph with six values: area under the curve (AUC), test accuracy, sensitivity, specificity, dataset size and labelling reliability. Plus, we defined the key points which should be taken into account when trying to accomplish this purpose and we compared each study with our baseline. In recent years, deep learning and, in particular, the convolution neural network (CNN), has achieved results comparable to those of humans in bone fracture classification. Adopting a correct generalization, we are reasonably sure that a computer-aided diagnosis (CAD) system, correctly designed to assist doctors, would save a considerable amount of time and would limit the number of wrong diagnoses

Computer-Aided Diagnosis System for Bone Fracture Detection and Classification: A Review on Deep Learning Techniques

Bone fracture detection and classification was a large discussed topic over the last few years and many researchers proposed different technological solutions to tackle this task. Despite this, a universal approach able to support the classification of fractures in the human body still does not exist today. We aim to provide a first discussion concerning a selection of research works done in the technological domain, with a specific focus on Deep Learning. The objective was to underline a picture on the most promising studies for stimulating a knowledge improvement in the specific focus of bone fracture classification, necessary to start the development of an optimal shared framework. The evaluation has been made involving a first qualitative assessment based on strengths and weaknesses, providing a usage scenario evaluation. This could support the development of a helpful Computer Aided Diagnosis (CAD) system able to drive doctors in diagnosis tasks reducing diagnosis time, especially in the most complex tasks, and supporting the reduction of wrong diagnosis issues, especially during stressful working conditions, as what frequently happens in many emergency departments

A deep learning framework for real-time 3D model registration in robot-assisted laparoscopic surgery

Introduction The current study presents a deep learning framework to determine, in real-time, position and rotation of a target organ from an endoscopic video. These inferred data are used to overlay the 3D model of patient's organ over its real counterpart. The resulting augmented video flow is streamed back to the surgeon as a support during laparoscopic robot-assisted procedures. Methods This framework exploits semantic segmentation and, thereafter, two techniques, based on Convolutional Neural Networks and motion analysis, were used to infer the rotation. Results The segmentation shows optimal accuracies, with a mean IoU score greater than 80% in all tests. Different performance levels are obtained for rotation, depending on the surgical procedure. Discussion Even if the presented methodology has various degrees of precision depending on the testing scenario, this work sets the first step for the adoption of deep learning and augmented reality to generalise the automatic registration process

Advanced deep learning comparisons for non-invasive tunnel lining assessment from ground penetrating radar profiles

Innovative, automated, and non-invasive techniques have been developed by scientific community to indirectly assess structural conditions and support the decision-making process for a worthwhile maintenance schedule. Nowadays, machine learning tools are in the spotlight because of their outstanding capabilities to deal with data coming from even heterogeneous sources and their ability to extract information from the structural systems, providing highly effective, reliable, and efficient damage classification tools. In the current study, a supervised multi-level damage classification strategy has been developed regarding Ground Penetrating Radar (GPR) profiles for the assessment of tunnel lining conditions. In previous research, the authors firstly considered a convolutional neural network (CNN), adopting the quite popular ResNet-50, initialized through transfer learning. In the present work, further enhancements have been attempted by adopting two configurations of the newest state-of-art advanced neural architectures: the neural transformers. The foremost is the original Vision Transformer (ViT), whose core is an encoder entirely based on the innovative self-attention mechanism and does not rely on convolution at all. The second is an improvement of ViT which merges convolution and self-attention, the Compact Convolution Transformer (CCT). In conclusion, a critical discussion of the different pros and cons of adopting the above-mentioned different architectures is finally provided, highlighting the actual powerfulness of these technologies in the future civil engineering paradigm nevertheless

Deep Learning and Augmented Reality for 3D human-machine interaction

L'abstract è presente nell'allegato / the abstract is in the attachmen

Intraoperative surgery room management: A deep learning perspective

The current study aimed to systematically review the literature addressing the use of deep learning (DL) methods in intraoperative surgery applications, focusing on the data collection, the objectives of these tools and, more technically, the DL-based paradigms utilized