AN AUTOMATED, DEEP LEARNING APPROACH TO SYSTEMATICALLY & SEQUENTIALLY DERIVE THREE-DIMENSIONAL KNEE KINEMATICS DIRECTLY FROM TWO-DIMENSIONAL FLUOROSCOPIC VIDEO

Total knee arthroplasty (TKA), also known as total knee replacement, is a surgical procedure to replace damaged parts of the knee joint with artificial components. It aims to relieve pain and improve knee function. TKA can improve knee kinematics and reduce pain, but it may also cause altered joint mechanics and complications. Proper patient selection, implant design, and surgical technique are important for successful outcomes. Kinematics analysis plays a vital role in TKA by evaluating knee joint movement and mechanics. It helps assess surgery success, guides implant and technique selection, informs implant design improvements, detects problems early, and improves patient outcomes. However, evaluating the kinematics of patients using conventional approaches presents significant challenges. The reliance on 3D CAD models limits applicability, as not all patients have access to such models. Moreover, the manual and time-consuming nature of the process makes it impractical for timely evaluations. Furthermore, the evaluation is confined to laboratory settings, limiting its feasibility in various locations. This study aims to address these limitations by introducing a new methodology for analyzing in vivo 3D kinematics using an automated deep learning approach. The proposed methodology involves several steps, starting with image segmentation of the femur and tibia using a robust deep learning approach. Subsequently, 3D reconstruction of the implants is performed, followed by automated registration. Finally, efficient knee kinematics modeling is conducted. The final kinematics results showed potential for reducing workload and increasing efficiency. The algorithms demonstrated high speed and accuracy, which could enable real-time TKA kinematics analysis in the operating room or clinical settings. Unlike previous studies that relied on sponsorships and limited patient samples, this algorithm allows the analysis of any patient, anywhere, and at any time, accommodating larger subject populations and complete fluoroscopic sequences. Although further improvements can be made, the study showcases the potential of machine learning to expand access to TKA analysis tools and advance biomedical engineering applications

A Computational Model to Predict \u3cem\u3eIn Vivo\u3c/em\u3e Kinetics in Implanted and Non-Implanted Shoulders

The purpose of this study was to develop and implement a computational model designed to input in vivo kinematic and predict in vivo forces and torques for the shoulder, elbow, and wrist in normal, rotator cuff-deficient (RCD), reverse shoulder arthroplasty (RSA) and total shoulder arthroplasty (TSA) shoulder subjects. Twenty subjects, divided evenly amongst the four shoulder types, performed a box-lift activity while under fluoroscopic surveillance. Three dimensional (3D) in vivo kinematics was determined for the subjects using implant models and bone models created from CT (computed tomography) scans in a 2D-to-3D registration process. The kinematics were used as input for an inverse dynamics mathematical model, and the subject-specific kinetics were derived. Average resultant shoulder forces were 78.3N (range: 70.4N to 117N, SD: 5.213), 102N (range: 90.2N to 180.2N, SD: 12.339), 94.9N (range: 84.9N to 149N, SD: 10.02), and 92.5N (range: 87.984N to 95.370N, SD: 1.848), for normal, RCD, RSA, and TSA subjects, respectively. Average resultant shoulder torques were 23.6Nm (range: 8.32Nm to 73.7Nm, SD: 11.227), 29.6Nm (range: 22.892Nm to 71.377Nm, SD: 7.581), 27.2Nm (range: 19.961Nm to 59.352Nm, SD: 6.664), 20.3Nm (range: 11.700Nm to 31.409Nm, SD: 6.496), for normal, RCD, RSA, and TSA shoulders, respectively. This study revealed that RCD subjects exhibited a decreased ROM (range of motion) of the humeral head with respect to the glenoid, as compared to the other groups. This study also showed that subjects having a rotator cuff-deficient shoulder and/or a replaced shoulder tend to use compensatory motions to perform the task of lifting a box, and, as a result, they experience greater forces at the glenohumeral joint. Paradoxically, the RCD subjects experienced the highest joint forces and torques among the different shoulder types

Parameterization of a Next Generation In-Vivo Forward Solution Physiological Model of the Human Lower Limb to Simulate and Predict Demographic and Pathology Specific Knee Mechanics

The human knee from a mechanical perspective is arguably one of the more complex of the joints of the human body and for this very reason there are a number of pathological factors that can adversely affect knee function, leading to pain, stiffness and an overall reduced quality of life. To rectify these disease conditions, a variety of intervention techniques exist, all of which are predicated on a thorough understanding of the forces and motions that occur at the knee.Various techniques have been developed to further the understanding of how the knee functions; however, many of these strategies involve time and cost consuming processes in order to assess functionality of the knee. Mathematical modeling is a methodology that uses mathematical equations of motion to solve for forces, or in the case of forward modeling, motions given a known set of forces. Such a model is capable of replicating the functionality of the knee in vivo.One application of such a model is in the context of total knee arthroplasty design. Intended for the restoration of functionality after late stage osteoarthritis, total knee arthroplasty devices are highly dependent on their associated design features and the use of a theoretical model affords the opportunity to test the performance of a device without ever needing to manufacture or implant it.In addition, there are also surgical applications where a mathematical model can test joints that otherwise cannot be evaluated under conventional means. This includes modeling of the healthy knee, as well as various functionality-limiting pathological conditions. Perhaps more importantly is the ability to evaluate different intervention techniques to determine the effectiveness in doing so identify which technique most effectively resolves the pathological issues.Advances to the model have focused on parameterization while contributing to a validated normal knee model, an enhancement on the efficiency of the muscles that drive flexion, facilitated methods to evaluate articular geometries and enhancements providing more realistic physiological motions. The model has also been enhanced to account for demographics, as well as abnormal pathology with additional parameters added to better understand gait mechanics at the knee

Development and Implementation of Mathematical Modeling, Vibration and Acoustic Emission Technique to Correlate \u3cem\u3eIn Vivo\u3c/em\u3e Kinematics, Kinetics and Sound in Total Hip Arthroplasty with Different Bearing Surfaces

The evaluation of Total Hip Arthroplasty (THA) outcome is difficult and invasive methods are often applied. Fluoroscopy has been used as an in vivo diagnostic technique to determine separation which may lead to vibration propagation and audible interactions. The objective of this study was to develop a new, non-invasive technique of digitally capturing vibration and sound emissions at the hip joint interface and to correlate those with the hip kinematics derived from fluoroscopy. Additionally, an examination of the role of hip mechanics on walking performance in THA subjects of various bearings surfaces was performed. In vivo kinematics, kinetics, corresponding vibration and sound measurements of THA were analyzed post-operatively using video-fluoroscopy, mathematical modeling, sound sensors and accelerometers during gait on a treadmill. Twenty-seven subjects (31 hips) with a metal-on-metal, metal-on-polyethylene, ceramic-on-ceramic, ceramicon- polyethylene or metal-on-metal polyethylene-sandwich THA were analyzed. A data acquisition system was used to amplify the signal and filter out associated frequencies attributed to noise. The sound measurements were correlated to in vivo kinematics. A mathematical model of the human extremity was derived to determine in vivo bearing and soft-tissue forces. For all bearings a distinct correlation of a high frequency sound occurring at the time when the femoral head slides back into the acetabular component was observed. Subjects having a hard-on-hard bearing seemed to attenuate a squeaking and/or impacting sound, while those having polyethylene liner only revealed a knocking sound attributed to impact loading conditions. For the first time, audible effects can be derived in vivo and the examined correlation brings valuable insight into the hip joint performance in an inexpensive and non-invasive manner. This research may allow for a further correlation to be derived between sound and different types of failure mechanisms. Results from this study will give surgeons and engineers a better understanding of in vivo mechanics of the hip joint and this way improve the quality of life of THA patients. In addition, the developed technique builds the first milestone in the design and implementation of a cost effective, non-invasive diagnostic technique which has the potential to become a routine diagnosis of joint conditions

DYNAMIC MEASUREMENT OF THREE-DIMENSIONAL MOTION FROM SINGLE-PERSPECTIVE TWO-DIMENSIONAL RADIOGRAPHIC PROJECTIONS

The digital evolution of the x-ray imaging modality has spurred the development of numerous clinical and research tools. This work focuses on the design, development, and validation of dynamic radiographic imaging and registration techniques to address two distinct medical applications: tracking during image-guided interventions, and the measurement of musculoskeletal joint kinematics. Fluoroscopy is widely employed to provide intra-procedural image-guidance. However, its planar images provide limited information about the location of surgical tools and targets in three-dimensional space. To address this limitation, registration techniques, which extract three-dimensional tracking and image-guidance information from planar images, were developed and validated in vitro. The ability to accurately measure joint kinematics in vivo is an important tool in studying both normal joint function and pathologies associated with injury and disease, however it still remains a clinical challenge. A technique to measure joint kinematics from single-perspective x-ray projections was developed and validated in vitro, using clinically available radiography equipmen

Augmented Reality: Mapping Methods and Tools for Enhancing the Human Role in Healthcare HMI

Background: Augmented Reality (AR) represents an innovative technology to improve data visualization and strengthen the human perception. Among Human–Machine Interaction (HMI), medicine can benefit most from the adoption of these digital technologies. In this perspective, the literature on orthopedic surgery techniques based on AR was evaluated, focusing on identifying the limitations and challenges of AR-based healthcare applications, to support the research and the development of further studies. Methods: Studies published from January 2018 to December 2021 were analyzed after a comprehensive search on PubMed, Google Scholar, Scopus, IEEE Xplore, Science Direct, and Wiley Online Library databases. In order to improve the review reporting, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used. Results: Authors selected sixty-two articles meeting the inclusion criteria, which were categorized according to the purpose of the study (intraoperative, training, rehabilitation) and according to the surgical procedure used. Conclusions: AR has the potential to improve orthopedic training and practice by providing an increasingly human-centered clinical approach. Further research can be addressed by this review to cover problems related to hardware limitations, lack of accurate registration and tracking systems, and absence of security protocols

Fusion of interventional ultrasound & X-ray

In einer immer älter werdenden Bevölkerung wird die Behandlung von strukturellen Herzkrankheiten zunehmend wichtiger. Verbesserte medizinische Bildgebung und die Einführung neuer Kathetertechnologien führten dazu, dass immer mehr herkömmliche chirurgische Eingriffe am offenen Herzen durch minimal invasive Methoden abgelöst werden. Diese modernen Interventionen müssen durch verschiedenste Bildgebungsverfahren navigiert werden. Hierzu werden hauptsächlich Röntgenfluoroskopie und transösophageale Echokardiografie (TEE) eingesetzt. Röntgen bietet eine gute Visualisierung der eingeführten Katheter, was essentiell für eine gute Navigation ist. TEE hingegen bietet die Möglichkeit der Weichteilgewebedarstellung und kann damit vor allem zur Darstellung von anatomischen Strukturen, wie z.B. Herzklappen, genutzt werden. Beide Modalitäten erzeugen Bilder in Echtzeit und werden für die erfolgreiche Durchführung minimal invasiver Herzchirurgie zwingend benötigt. Üblicherweise sind beide Systeme eigenständig und nicht miteinander verbunden. Es ist anzunehmen, dass eine Bildfusion beider Welten einen großen Vorteil für die behandelnden Operateure erzeugen kann, vor allem eine verbesserte Kommunikation im Behandlungsteam. Ebenso können sich aus der Anwendung heraus neue chirurgische Worfklows ergeben. Eine direkte Fusion beider Systeme scheint nicht möglich, da die Bilddaten eine zu unterschiedliche Charakteristik aufweisen. Daher kommt in dieser Arbeit eine indirekte Registriermethode zum Einsatz. Die TEE-Sonde ist während der Intervention ständig im Fluoroskopiebild sichtbar. Dadurch wird es möglich, die Sonde im Röntgenbild zu registrieren und daraus die 3D Position abzuleiten. Der Zusammenhang zwischen Ultraschallbild und Ultraschallsonde wird durch eine Kalibrierung bestimmt. In dieser Arbeit wurde die Methode der 2D-3D Registrierung gewählt, um die TEE Sonde auf 2D Röntgenbildern zu erkennen. Es werden verschiedene Beiträge präsentiert, welche einen herkömmlichen 2D-3D Registrieralgorithmus verbessern. Nicht nur im Bereich der Ultraschall-Röntgen-Fusion, sondern auch im Hinblick auf allgemeine Registrierprobleme. Eine eingeführte Methode ist die der planaren Parameter. Diese verbessert die Robustheit und die Registriergeschwindigkeit, vor allem während der Registrierung eines Objekts aus zwei nicht-orthogonalen Richtungen. Ein weiterer Ansatz ist der Austausch der herkömmlichen Erzeugung von sogenannten digital reconstructed radiographs. Diese sind zwar ein integraler Bestandteil einer 2D-3D Registrierung aber gleichzeitig sehr zeitaufwendig zu berechnen. Es führt zu einem erheblichen Geschwindigkeitsgewinn die herkömmliche Methode durch schnelles Rendering von Dreiecksnetzen zu ersetzen. Ebenso wird gezeigt, dass eine Kombination von schnellen lernbasierten Detektionsalgorithmen und 2D-3D Registrierung die Genauigkeit und die Registrierreichweite verbessert. Zum Abschluss werden die ersten Ergebnisse eines klinischen Prototypen präsentiert, welcher die zuvor genannten Methoden verwendet.Today, in an elderly community, the treatment of structural heart disease will become more and more important. Constant improvements of medical imaging technologies and the introduction of new catheter devices caused the trend to replace conventional open heart surgery by minimal invasive interventions. These advanced interventions need to be guided by different medical imaging modalities. The two main imaging systems here are X-ray fluoroscopy and Transesophageal  Echocardiography (TEE). While X-ray provides a good visualization of inserted catheters, which is essential for catheter navigation, TEE can display soft tissues, especially anatomical structures like heart valves. Both modalities provide real-time imaging and are necessary to lead minimal invasive heart surgery to success. Usually, the two systems are detached and not connected. It is conceivable that a fusion of both worlds can create a strong benefit for the physicians. It can lead to a better communication within the clinical team and can probably enable new surgical workflows. Because of the completely different characteristics of the image data, a direct fusion seems to be impossible. Therefore, an indirect registration of Ultrasound and X-ray images is used. The TEE probe is usually visible in the X-ray image during the described minimal-invasive interventions. Thereby, it becomes possible to register the TEE probe in the fluoroscopic images and to establish its 3D position. The relationship of the Ultrasound image to the Ultrasound probe is known by calibration. To register the TEE probe on 2D X-ray images, a 2D-3D registration approach is chosen in this thesis. Several contributions are presented, which are improving the common 2D-3D registration algorithm for the task of Ultrasound and X-ray fusion, but also for general 2D-3D registration problems. One presented approach is the introduction of planar parameters that increase robustness and speed during the registration of an object on two non-orthogonal views. Another approach is to replace the conventional generation of digital reconstructedradiographs, which is an integral part of 2D-3D registration but also a performance bottleneck, with fast triangular mesh rendering. This will result in a significant performance speed-up. It is also shown that a combination of fast learning-based detection algorithms with 2D-3D registration will increase the accuracy and the capture range, instead of employing them solely to the  registration/detection of a TEE probe. Finally, a first clinical prototype is presented which employs the presented approaches and first clinical results are shown

Exploiting Temporal Image Information in Minimally Invasive Surgery

Minimally invasive procedures rely on medical imaging instead of the surgeons direct vision. While preoperative images can be used for surgical planning and navigation, once the surgeon arrives at the target site real-time intraoperative imaging is needed. However, acquiring and interpreting these images can be challenging and much of the rich temporal information present in these images is not visible. The goal of this thesis is to improve image guidance for minimally invasive surgery in two main areas. First, by showing how high-quality ultrasound video can be obtained by integrating an ultrasound transducer directly into delivery devices for beating heart valve surgery. Secondly, by extracting hidden temporal information through video processing methods to help the surgeon localize important anatomical structures. Prototypes of delivery tools, with integrated ultrasound imaging, were developed for both transcatheter aortic valve implantation and mitral valve repair. These tools provided an on-site view that shows the tool-tissue interactions during valve repair. Additionally, augmented reality environments were used to add more anatomical context that aids in navigation and in interpreting the on-site video. Other procedures can be improved by extracting hidden temporal information from the intraoperative video. In ultrasound guided epidural injections, dural pulsation provides a cue in finding a clear trajectory to the epidural space. By processing the video using extended Kalman filtering, subtle pulsations were automatically detected and visualized in real-time. A statistical framework for analyzing periodicity was developed based on dynamic linear modelling. In addition to detecting dural pulsation in lumbar spine ultrasound, this approach was used to image tissue perfusion in natural video and generate ventilation maps from free-breathing magnetic resonance imaging. A second statistical method, based on spectral analysis of pixel intensity values, allowed blood flow to be detected directly from high-frequency B-mode ultrasound video. Finally, pulsatile cues in endoscopic video were enhanced through Eulerian video magnification to help localize critical vasculature. This approach shows particular promise in identifying the basilar artery in endoscopic third ventriculostomy and the prostatic artery in nerve-sparing prostatectomy. A real-time implementation was developed which processed full-resolution stereoscopic video on the da Vinci Surgical System

Advancement of a Forward Solution Mathematical Model of the Human Knee Joint

Sometimes called degenerative joint disease, osteoarthritis most often affects the knee, which is a leading cause of pain and reduced mobility. While early treatment is ideal, it is not always successful in combating osteoarthritis and improving joint function, therefore creating the need for total knee arthroplasty (TKA), which is a late-stage treatment where damaged bone and cartilage are replaced by artificial cartilage. Joint arthroplasty is a common and successful procedure for end-stage osteoarthritis. Unfortunately, TKA patient satisfaction rates lag behind those of total hip arthroplasty [1,2], which remains an impetus to create new designs. Due to ethical issues, time requirements, and prohibitive expenses of testing new designs in vivo, mathematical modeling may be an alternative tool to efficiently assess the kinetics and kinematics of new TKA designs. In general, the knee is one of the most complicated joints in the human body, including multiple articulating surfaces and the complexity of soft tissues encompassing the knee joint. Therefore, mathematically modeling the knee is a challenging and complex process. With increasing computational power and advanced knowledge and techniques, advanced mathematical models of the knee joint can be created utilizing various modeling techniques [3]. Furthermore, mathematical modeling can advance our knowledge related to knee biomechanics, especially those parameters that are otherwise challenging to obtain, such as soft tissue properties and effects pertaining to knee mechanics. Mathematical modeling allows the user to evaluate multiple designs and surgical approaches quickly and cost-efficiently without having to conduct lengthy clinical studies. Mathematical models can also provide insight into topics of clinical significance and can efficiently analyze outcome contributions that cannot be controlled in fluoroscopic studies, such as anatomical, mechanical, and kinematic alignment comparisons for the same subject. Furthermore, mathematical models can evaluate the effect of TKA design concerns such as changing conformity of the polyethylene or using femoral components with single or multi radius designs [3]. The objectives of this dissertation are to advance a forward solution model to create a more sophisticated and physiological representation of the knee joint.This is achieved by developing a muscle wrapping algorithm, integrating a validated inverse dynamics model, adding more muscles, incorporating several different TKA types including revision TKA designs, and expanding the model to include other daily activities. All these modifications are incorporated in a graphical user interface. These advancements increase both functionality and accuracy of the model. Several validation methods have been implemented to investigate the accuracy of the predicted kinetics and kinematics of this mathematical model