Recent trends, technical concepts and components of computer-assisted orthopedic surgery systems: A comprehensive review

Computer-assisted orthopedic surgery (CAOS) systems have become one of the most important and challenging types of system in clinical orthopedics, as they enable precise treatment of musculoskeletal diseases, employing modern clinical navigation systems and surgical tools. This paper brings a comprehensive review of recent trends and possibilities of CAOS systems. There are three types of the surgical planning systems, including: systems based on the volumetric images (computer tomography (CT), magnetic resonance imaging (MRI) or ultrasound images), further systems utilize either 2D or 3D fluoroscopic images, and the last one utilizes the kinetic information about the joints and morphological information about the target bones. This complex review is focused on three fundamental aspects of CAOS systems: their essential components, types of CAOS systems, and mechanical tools used in CAOS systems. In this review, we also outline the possibilities for using ultrasound computer-assisted orthopedic surgery (UCAOS) systems as an alternative to conventionally used CAOS systems.Web of Science1923art. no. 519

Advancements and Breakthroughs in Ultrasound Imaging

Ultrasonic imaging is a powerful diagnostic tool available to medical practitioners, engineers and researchers today. Due to the relative safety, and the non-invasive nature, ultrasonic imaging has become one of the most rapidly advancing technologies. These rapid advances are directly related to the parallel advancements in electronics, computing, and transducer technology together with sophisticated signal processing techniques. This book focuses on state of the art developments in ultrasonic imaging applications and underlying technologies presented by leading practitioners and researchers from many parts of the world

Augmented reality (AR) for surgical robotic and autonomous systems: State of the art, challenges, and solutions

Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human-robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future

Development of 3D software for post-surgical shoulder arthroplasty implant position analysis

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Engenharia Clínica e Instrumentação Médica) Universidade de Lisboa, Faculdade de Ciências, 2020O ombro garante a articulação entre a cabeça do úmero e a parte glenoide da escapula, sendo responsável por diversas tarefas essenciais no nosso dia a dia. Esta articulação tem uma grande amplitude de movimento devido às ações dos músculos e às configurações articulares, permitindo movimentos como abdução, adução, rotação, elevação para a frente e para trás do tronco e mover-se em 360 ° no plano sagital. No entanto, esta grande amplitude de movimento torna o ombro mais instável e suscetível a lesões, o que poderá ter um impacto direto na qualidade de vida de uma pessoa, podendo condicionar significativamente a mobilidade desta articulação. Algumas das patologias do ombro incluem desgaste excessivo, inflamação, rutura do músculo da coifa dos rotadores, instabilidade e impacto que podem estar relacionados a doenças degenerativas, como a osteoartrite, fraturas ou uso excessivo da articulação (devido a movimentos repetitivos em desportos, no trabalho, etc.). Infelizmente, a degeneração da articulação do ombro é um problema particularmente frequente no envelhecimento da nossa população, afetando uma em cada cinco pessoas acima de 65 anos. Existem 2 tipos principais de degeneração do ombro: 1. osteoartrite, que é a degeneração primária da cartilagem articular relacionada com a idade, com um músculo da coifa dos rotadores intacto; 2. Artropatia por rutura da coifa dos rotadores, e, como o nome indica, caracterizada por uma grande rutura do músculo da coifa dos rotadores, levando a alterações na carga articular e danos articulares progressivos. Ambos os tipos de degeneração são caracterizados pela destruição da cartilagem articular que leva à dor, restrição de movimento e incapacidade funcional. Uma degeneração adicional pode levar à destruição óssea da glenoide, um fenómeno conhecido como erosão da glenoide. Existem diferentes tipos de erosões, dependendo do tipo de degeneração do ombro. Curiosamente, a osteoartrite é caracterizada principalmente por erosão óssea posterior e a artropatia por rutura da coifa dos rotadores por erosão óssea superior. A evolução da medicina permitiu a correção destas patologias ou consequências das mesmas, sendo agora uma cirurgia bastante comum na área da ortopedia, especialmente em idosos. Para tal, é necessário estudar cada caso e fazer um planeamento cirúrgico adequado através de imagens raio-X ou tomografias computadorizadas. O tratamento cirúrgico passa pela artroplastia do ombro (anatómica ou invertida) que ganhou popularidade devido à sua eficácia no alívio da dor e na restauração da função do ombro degenerado. Durante a cirurgia, o cirurgião precisa de estudar a inclinação, versão e correção do desvio e suporte ósseo mais adequados para a colocação do implante. A técnica mais utilizada é o alargamento assimétrico, onde a erosão da glenoide é corrigida com a remoção do lado mais alto do osso. Isso leva a uma perda óssea da medialização da glenoide e da linha articular, o que pode influenciar a função do implante do ombro e a duração de vida do mesmo a longo prazo. As causas mais importantes para falha da artroplastia do ombro são complicações do componente glenoide, como alargamento e desgaste, especialmente em glenoides com erosão pré-operatória. Esta erosão pode originar ângulos de inclinação e versão e desvio alterados. A principal causa de falha do componente glenoide em pacientes com erosão é carga anormal descentralizada quando a erosão não é adequadamente corrigida. Portanto, a glenoide erodida deve ser corrigida para uma inclinação, versão e desvio mais normais antes da implantação do componente glenoide. O posicionamento adequado do implante, juntamente com o equilíbrio dos tecidos moles, são questões-chave na artroplastia do ombro, tanto para a função pós-operatória quanto para a duração a longo prazo do implante. O mau posicionamento do componente glenoide pode levar a uma má função, instabilidade e enfraquecimento precoce do componente na artroplastia total e reversa do ombro. Devido às dificuldades com a colocação correta e às possíveis complicações que advém da inadequada da colocação do componente da glenoide, começou-se a investigar o planeamento computacional para a colocação do implante. O uso de ferramentas de planeamento (software) através de tomografia computadorizadas 3D e instrumentação específica do paciente é, neste momento, a prática mais comum. Para além da avaliação pré-cirúrgica de cada caso e do seu planeamento, o seguimento médico e avaliação pós-cirúrgica também é de extrema importância para averiguar se a colocação do implante foi adequada e se está de acordo com o plano pré-operatório. Para isso, são usados, de igual forma do planeamento, raios-X ou tomografias computadorizadas que são avaliadas pelos médicos. A maioria dos estudos em geral demonstra que as técnicas de tomografia computadorizada são mais precisas na determinação da posição pós-cirúrgica do componente glenoide do que as radiografias axiais padrão. Mais do que isso, o uso de destas acoplado a softwares 3D para análise da posição do implante demonstrou ser mais preciso, pois os modelos 3D refletem com maior precisão a verdadeira anatomia da morfologia da escápula. Assim, numa perspetiva de tentar encontrar uma solução idêntica à de software de planeamento, mas adaptado para análise pós cirúrgica do implante, foi proposto um projeto para desenvolvimento de um novo software capaz de realizar a análise da posição pós-cirúrgica do implante em termos de versão, inclinação e desvio do implante. Portanto, o objetivo deste sistema de software é determinar a confiabilidade e a precisão da colocação do implante. O software foi desenvolvido na plataforma MeVisLab e fornece uma descrição da posição do componente glenoide em termos de versão, inclinação e desvio do implante (componente glenóide) entre as posições reais e ideais e a rotação do implante. O MeVisLab é uma plataforma diversificada para processamento de imagens médicas e visualização científica. Inclui algoritmos avançados para registo, segmentação e análise quantitativa de imagens morfológicas e funcionais. A sua estrutura é baseada em módulos e com estes, redes podem ser criadas e diferentes aplicações podem ser construídas. A criação deste software nesta plataforma é considerada uma inovação, pois não há nenhuma referência a algo semelhante, com o mesmo objetivo e funcionalidades, na pesquisa bibliográfica feita. O modelo do implante utilizado ao longo do desenvolvimento do software e na sua análise foi ligeiramente modificado, fechando o lado aberto da glenoesfera (componente glenoide). Assim sendo, o desenvolvimento deste projeto incluiu etapas diferentes: a primeira foi a definição das medidas requisitadas pelos médicos, ângulos e pontos anatómicos de referência necessários, sendo que, neste caso, os pontos anatómicos de referência devem ser selecionados pelo utilizador e os outros dois serão calculados automaticamente; O próximo passo foi o desenvolvimento do software no MeVisLab com uma rede baseada em módulos que incluía módulos de visualização de imagens (2D e 3D), módulos que auxiliam na definição de planos e pontos de referência para os cálculos e módulos scripts em Python que contêm o código para todos os itens necessários para os cálculos; Em seguida, o desenvolvimento da interface para o utilizador foi feito de forma a que o mesmo tivesse uma experiência intuitiva e de fácil uso, e que conseguisse seguir todas as instruções necessárias, para a seleção de todos os pontos de referência e, posteriormente, o cálculo dos ângulos e medidas requisitadas; Finalmente, o último passo foi garantir que o software criado fosse confiável e consistente em todos os seus resultados no domínio intraobservador e interobservador e quando comparado com os resultados encontrados na literatura. Até ao momento, o software desenvolvido fornece os ângulos e medidas desejados com sucesso, no entanto, mostrou-se não ser tão confiável e consistente quanto era desejado. Assim, muito pode ainda ser feito para melhorar a precisão do software desenvolvido e atingir totalmente o objetivo final. Os resultados obtidos através do software podem ainda, mais tarde, ser usados para comparar se o implante colocado tem a mesma orientação que a planeada para um paciente específico antes da cirurgia. Além disso, a posição do implante pode ser correlacionada com a função pós-operatória do paciente.The shoulder has a high range of motion because of its muscles actions and joint configuration, allowing it to abduct, adduct, rotate, be raised in front of and behind the torso and move through a full 360° in the sagittal plane. This large range of motion makes the shoulder more unstable and susceptible to injuries. The most common shoulder pathologies include cuff tear arthropathy and osteoarthritis which are related to permanent loss of the rotator cuff tendons and a gradual wearing of the articular cartilage, respectively, that leads to pain, stiffness and, consequently, to loss of the shoulder function. Reverse shoulder arthroplasty (RSA) has been proven to be a successful treatment for cuff tear arthropathy and osteoarthritis in the elderly patients. RSA consist in a shoulder replacement with a prosthesis that aims to restore the best possible function to the joint by removing scar tissue balancing muscles and replacing the destroyed joint surface of the humerus. The glenosphere positioning during the procedure has a significant impact on outcomes in RSA because it determines the center of rotation and biomechanical traits of the new joint. Misalignment and/or displacement of the glenoid component with respect to the bone can be a cause of, or contribute to, failure of the implant. Reasons for displacementof the glenoid component include inaccurate assessment of the pathologic anatomy of the glenoid, incorrect choice of implant and/or position of the implant to correct the pathologic condition, and inaccurate surgical execution of the preoperative plan. The goal of this project was to create a software for post-surgical shoulder arthroplasty analysis that gives version, inclination and implant shift values of the glenoid component aiming to evaluate the precision and reliability of its placement. The development of this project included different steps: the first one was the definition of the desired measurements, angles and the needed landmarks being that landmarks must be selected by the user and then the other two will be calculated automatically; The next step was the development of the novel software in MeVisLab with a module based network that included image visualizing modules (2D and 3D), modules that assist the definition of planes and landmarks for the calculations and python script modules that contain the code for all needed calculations; Then, the development of the user interface took place with the necessary means for the user to have all instructions needed, for the selection of all landmarks and returning the angles and measurements calculated; Finally, the last step was to ensure that the created software was reliable and consistent in its results in both intraobserver and interobserver domain and when compared with literature findings. So far, the developed software provides the required version and inclination angles and implant shift measure successfully however has shown to not be as reliable and consistent as desired. Thus, a lot can still be done to improve the accuracy of the developed software and to achieve fully the final goal

3D approximation of scapula bone shape from 2D X-ray images using landmark-constrained statistical shape model fitting

Two-dimensional X-ray imaging is the dominant imaging modality in low-resource countries despite the existence of three-dimensional (3D) imaging modalities. This is because fewer hospitals in low-resource countries can afford the 3D imaging systems as their acquisition and operation costs are higher. However, 3D images are desirable in a range of clinical applications, for example surgical planning. The aim of this research was to develop a tool for 3D approximation of scapula bone from 2D X-ray images using landmark-constrained statistical shape model fitting. First, X-ray stereophotogrammetry was used to reconstruct the 3D coordinates of points located on 2D X-ray images of the scapula, acquired from two perspectives. A suitable calibration frame was used to map the image coordinates to their corresponding 3D realworld coordinates. The 3D point localization yielded average errors of (0.14, 0.07, 0.04) mm in the X, Y and Z coordinates respectively, and an absolute reconstruction error of 0.19 mm. The second phase assessed the reproducibility of the scapula landmarks reported by Ohl et al. (2010) and Borotikar et al. (2015). Only three (the inferior angle, acromion and the coracoid process) of the eight reproducible landmarks considered were selected as these were identifiable from the two different perspectives required for X-ray stereophotogrammetry in this project. For the last phase, an approximation of a scapula was produced with the aid of a statistical shape model (SSM) built from a training dataset of 84 CT scapulae. This involved constraining an SSM to the 3D reconstructed coordinates of the selected reproducible landmarks from 2D X-ray images. Comparison of the approximate model with a CT-derived ground truth 3D segmented volume resulted in surface-to-surface average distances of 4.28 mm and 3.20 mm, using three and sixteen landmarks respectively. Hence, increasing the number of landmarks produces a posterior model that makes better predictions of patientspecific reconstructions. An average Euclidean distance of 1.35 mm was obtained between the three selected landmarks on the approximation and the corresponding landmarks on the CT image. Conversely, a Euclidean distance of 5.99 mm was obtained between the three selected landmarks on the original SSM and corresponding landmarks on the CT image. The Euclidean distances confirm that a posterior model moves closer to the CT image, hence it reduces the search space for a more exact patient-specific 3D reconstruction by other fitting algorithms

End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention

Accurate real-time catheter segmentation is an important pre-requisite for robot-assisted endovascular intervention. Most of the existing learning-based methods for catheter segmentation and tracking are only trained on small-scale datasets or synthetic data due to the difficulties of ground-truth annotation. Furthermore, the temporal continuity in intraoperative imaging sequences is not fully utilised. In this paper, we present FW-Net, an end-to-end and real-time deep learning framework for endovascular intervention. The proposed FW-Net has three modules: a segmentation network with encoder-decoder architecture, a flow network to extract optical flow information, and a novel flow-guided warping function to learn the frame-to-frame temporal continuity. We show that by effectively learning temporal continuity, the network can successfully segment and track the catheters in real-time sequences using only raw ground-truth for training. Detailed validation results confirm that our FW-Net outperforms state-of-the-art techniques while achieving real-time performance.Comment: ICRA 202

Development of a cognitive robotic system for simple surgical tasks

The introduction of robotic surgery within the operating rooms has significantly improved the quality of many surgical procedures. Recently, the research on medical robotic systems focused on increasing the level of autonomy in order to give them the possibility to carry out simple surgical actions autonomously. This paper reports on the development of technologies for introducing automation within the surgical workflow. The results have been obtained during the ongoing FP7 European funded project Intelligent Surgical Robotics (I-SUR). The main goal of the project is to demonstrate that autonomous robotic surgical systems can carry out simple surgical tasks effectively and without major intervention by surgeons. To fulfil this goal, we have developed innovative solutions (both in terms of technologies and algorithms) for the following aspects: fabrication of soft organ models starting from CT images, surgical planning and execution of movement of robot arms in contact with a deformable environment, designing a surgical interface minimizing the cognitive load of the surgeon supervising the actions, intra-operative sensing and reasoning to detect normal transitions and unexpected events. All these technologies have been integrated using a component-based software architecture to control a novel robot designed to perform the surgical actions under study. In this work we provide an overview of our system and report on preliminary results of the automatic execution of needle insertion for the cryoablation of kidney tumours

A Review of Virtual Reality Based Training Simulators for Orthopaedic Surgery

This review presents current virtual reality based training simulators for hip, knee and other orthopaedic surgery, including elective and trauma surgical procedures. There have not been any reviews focussing on hip and knee orthopaedic simulators. A comparison of existing simulator features is provided to identify what is missing and what is required to improve upon current simulators. In total 11 total hip replacement pre-operative planning tools were analysed, plus 9 hip trauma fracture training simulators. Additionally 9 knee arthroscopy simulators and 8 other orthopaedic simulators were included for comparison. The findings are that for orthopaedic surgery simulators in general, there is increasing use of patient-specific virtual models which reduce the learning curve. Modelling is also being used for patient-specific implant design and manufacture. Simulators are being increasingly validated for assessment as well as training. There are very few training simulators available for hip replacement, yet more advanced virtual reality is being used for other procedures such as hip trauma and drilling. Training simulators for hip replacement and orthopaedic surgery in general lag behind other surgical procedures for which virtual reality has become more common. Further developments are required to bring hip replacement training simulation up to date with other procedures. This suggests there is a gap in the market for a new high fidelity hip replacement and resurfacing training simulator

Augmented Reality: Mapping Methods and Tools for Enhancing the Human Role in Healthcare HMI

Background: Augmented Reality (AR) represents an innovative technology to improve data visualization and strengthen the human perception. Among Human–Machine Interaction (HMI), medicine can benefit most from the adoption of these digital technologies. In this perspective, the literature on orthopedic surgery techniques based on AR was evaluated, focusing on identifying the limitations and challenges of AR-based healthcare applications, to support the research and the development of further studies. Methods: Studies published from January 2018 to December 2021 were analyzed after a comprehensive search on PubMed, Google Scholar, Scopus, IEEE Xplore, Science Direct, and Wiley Online Library databases. In order to improve the review reporting, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used. Results: Authors selected sixty-two articles meeting the inclusion criteria, which were categorized according to the purpose of the study (intraoperative, training, rehabilitation) and according to the surgical procedure used. Conclusions: AR has the potential to improve orthopedic training and practice by providing an increasingly human-centered clinical approach. Further research can be addressed by this review to cover problems related to hardware limitations, lack of accurate registration and tracking systems, and absence of security protocols

A review of virtual reality based training simulators for orthopaedic surgery

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this recordThis review presents current virtual reality based training simulators for hip, knee and other orthopaedic surgery, including elective and trauma surgical procedures. There have not been any reviews focussing on hip and knee orthopaedic simulators. A comparison of existing simulator features is provided to identify what is missing and what is required to improve upon current simulators. In total 11 hip replacements pre-operative planning tools were analysed, plus 9 hip trauma fracture training simulators. Additionally 9 knee arthroscopy simulators and 8 other orthopaedic simulators were included for comparison. The findings are that for orthopaedic surgery simulators in general, there is increasing use of patient-specific virtual models which reduce the learning curve. Modelling is also being used for patient-specific implant design and manufacture. Simulators are being increasingly validated for assessment as well as training. There are very few training simulators available for hip replacement, yet more advanced virtual reality is being used for other procedures such as hip trauma and drilling. Training simulators for hip replacement and orthopaedic surgery in general lag behind other surgical procedures for which virtual reality has become more common. Further developments are required to bring hip replacement training simulation up to date with other procedures. This suggests there is a gap in the market for a new high fidelity hip replacement and resurfacing training simulator.Wessex Academic Health Science Network (Wessex AHSN) Innovation and Wealth Creation Accelerator Fund 2014/15Bournemouth Universit