A Review on Segmentation of Knee Articular Cartilage: from Conventional Methods Towards Deep Learning

In this paper, we review the state-of-the-art approaches for knee articular cartilage segmentation from conventional techniques to deep learning (DL) based techniques. Knee articular cartilage segmentation on magnetic resonance (MR) images is of great importance in early diagnosis of osteoarthritis (OA). Besides, segmentation allows estimating the articular cartilage loss rate which is utilised in clinical practice for assessing the disease progression and morphological changes. Topics covered include various image processing algorithms and major features of different segmentation techniques, feature computations and the performance evaluation metrics. This paper is intended to provide researchers with a broad overview of the currently existing methods in the field, as well as to highlight the shortcomings and potential considerations in the application at clinical practice. The survey showed that the state-of-the-art techniques based on DL outperforms the other segmentation methods. The analysis of the existing methods reveals that integration of DL-based algorithms with other traditional model-based approaches have achieved the best results (mean Dice similarity cofficient (DSC) between 85:8% and 90%)

Automatic quantification of radiographic knee osteoarthritis severity and associated diagnostic features using deep convolutional neural networks

“Automatic Quantification of Radiographic Knee Osteoarthritis Severity and Associated Diagnostic Features using Deep Convolutional Neural Networks” A. Joseph Antony Due to the increasing prevalence of knee Osteoarthritis (OA), a debilitating kneejoint degradation, and total joint arthoplasty as a serious consequence, there is a need for effective clinical and scientific tools to assess knee OA in its early stages. This thesis investigates the use of machine learning algorithms and deep learning architectures, in particular convolutional neural networks (CNN), to quantify the severity and clinical radiographic features of knee OA. The goal is to offer novel and effective solutions to automatically assess the severity of knee OA achieving on par with human accuracy. Instead of conventional hand-crafted features, it is proposed in this thesis that automatically learning features in a supervised manner can be more effective for fine-grained knee OA image classification. The main contributions of this thesis are as follows. First, the use of off-the-shelf CNNs are investigated for classifying knee OA images through transfer learning by fine-tuning the CNNs. Second, CNNs are trained from scratch to quantify the knee OA severity optimising a weighted ratio of two loss functions: categorical cross entropy and mean-squared error. Third, CNNs are jointly trained to quantify the clinical features of knee OA: joint space narrowing (JSN) and osteophytes along with the KL grades. This improves the overall quantification of knee OA severity producing simultaneous predictions of KL grades, JSN and osteophytes. Two public datasets are used to evaluate the approaches, the OAI and the MOST, with extremely promising results that outperform existing approaches. In summary, this thesis primarily contributes to the field of automated methods for localisation and quantification of radiographic knee OA

Data-efficient knee anatomical landmark localization using deep learning

Abstract. Knee osteoarthritis (OA) is the most common musculoskeletal degenerative disease affecting the joints. OA is examined at a doctor’s visit and an X-ray image is often used to confirm the diagnosis. There is no treatment available for OA, therefore it is important to diagnose knee osteoarthritis at the earliest possible stage to preventpossible complications. Traditional methods used by a practitioners do not detect osteoarthritis as early as possible, therefore other methods are needed for early diagnosis. One possibility is to use novel quantitative imaging biomarkers, computation of which often requires precise understanding of the knee anatomy by a computer. More specifically, it is important to locate different areas of the knee according to anatomical atlases and place relevant regions of interest to compute the imaging biomarkers. A state-of-the-art approach for this problem is based on anatomical landmark localization. In this work, the localization of anatomical landmarks from knee X-rays using deep learning is investigated. To date, statistical methods have been used to localize landmarks, but this work focuses on identification based on deep learning and investigates how the amount of available training data affects performance. The method investigated in the present thesis is based on the KNEEL method developed earlier at the University of Oulu. The aim of this work was to improve this method by adjusting the training parameters and leveraging equivalent regularization for semi-supervised learning. Images from the Osteoarthritis Initiative database were used as material for training and validation. During the work, it was found that by adjusting the parameters used for training, anatomical landmarks can be localized more accurately than in the original KNEEL method. By adding the equivalent regularization, the accuracy of the localization was increased substantially, and a further enhancement in performance can be observed by utilizing unlabeled data in a semi-supervised learning manner. The results, developed in this thesis can layer be leveraged in OA research or even clinical practice, where the computation of quantitative imaging biomarkers is important. To our knowledge, this is the first work in OA where SSL and equivariant regularization were used.Datatehokas polven anatomisten maamerkkien paikantaminen käyttäen syväoppimista. Tiivistelmä. Polven nivelrikko on yleisin niveliin vaikuttava tuki- ja liikuntaelimistöä rappeuttava sairaus. Nivelrikko tutkitaan lääkärikäynnin yhteydessä ja diagnoosi vahvistetaan usein röntgenkuvantamisen avulla. Nivelrikkoon ei ole saatavilla hoitoa, joten on tärkeää diagnosoida polven nivelrikko mahdollisimman varhaisessa vaiheessa mahdollisten komplikaatioiden välttämiseksi. Perinteiset lääkäreiden käyttämät menetelmät eivät tunnista nivelrikkoa riittävän aikaisin, siksi tarvitaan muita menetelmiä varhaisempaan diagnostiikkaan. Yksi mahdollisuus on käyttää kvantitatiivisia kuvantamisbiomarkkereita, mutta näiden laskemiseksi tietokoneen täytyy ymmärtää anatomisia rakenteita tarkasti. Tarkemmin sanottuna on tärkeää paikantaa polven eri rakenteet ihmisen anatomiasta ja merkitä kiinnostavat rakenteet, jotta kuvantamisbiomarkkerit voidaan laskea. Nykyisin tätä ongelmaa lähestytään anatomisten maamerkkien paikantamisen avulla. Tässä työssä tutkittiin anatomisten maamerkkien paikantamista polven röntgenkuvista syväoppimisen avulla. Perinteisesti tähän on käytetty staattisia menetelmiä, mutta tässä työssä keskityttiin paikantamiseen käyttäen syväoppimista ja tutkittiin kuinka käytettävissä oleva opetusdatan määrä vaikuttaa suorituskykyyn. Työssä käytetty metodi perustuu aikaisemmin Oulun yliopistossa kehitettyyn KNEEL metodiin. Tämän työn tarkoituksena oli parantaa tätä metodia säätämällä opetusparametreja sekä hyödyntää ekvivalenttia regularisaatiota syväoppimisen yhteydessa. Kuvia The Osteoarthritis Initiative -tietokannasta käytettiin opetukseen ja validointiin. Työn aikana havaittiin, että säätämällä opetukseen käytettäviä parametrejä, voidaan anatomiset maamerkit paikantaa tarkemmin kuin alkuperäisellä KNEEL metodilla. Ekvivalentin regularisaation lisäämisellä paikantamisen tarkkuus lisääntyi huomattavasti. Suorituskyky parani entisestään käyttämällä annotoimatonta dataa puoli-ohjatun oppimisen yhteydessä. Tämän opinnäytetyön yhteydessä kehitettyä metodia voidaan käyttää nivelrikon tutkimuksen yhteydessä tai kliinisessä käytössä, missä kvantitatiivisten kuvantamisbiomarkkereiden käyttö on tärkeää. Tietojemme mukaan tämä työ on ensimmäinen, jossa käytetään puoliohjattua oppimista sekä ekvivalenttia regularisaatiota nivelrikon yhteydessä

High-Resolution Quantitative Cone-Beam Computed Tomography: Systems, Modeling, and Analysis for Improved Musculoskeletal Imaging

This dissertation applies accurate models of imaging physics, new high-resolution imaging hardware, and novel image analysis techniques to benefit quantitative applications of x-ray CT in in vivo assessment of bone health. We pursue three Aims: 1. Characterization of macroscopic joint space morphology, 2. Estimation of bone mineral density (BMD), and 3. Visualization of bone microstructure. This work contributes to the development of extremity cone-beam CT (CBCT), a compact system for musculoskeletal (MSK) imaging. Joint space morphology is characterized by a model which draws an analogy between the bones of a joint and the plates of a capacitor. Virtual electric field lines connecting the two surfaces of the joint are computed as a surrogate measure of joint space width, creating a rich, non-degenerate, adaptive map of the joint space. We showed that by using such maps, a classifier can outperform radiologist measurements at identifying osteoarthritic patients in a set of CBCT scans. Quantitative BMD accuracy is achieved by combining a polyenergetic model-based iterative reconstruction (MBIR) method with fast Monte Carlo (MC) scatter estimation. On a benchtop system emulating extremity CBCT, we validated BMD accuracy and reproducibility via a series of phantom studies involving inserts of known mineral concentrations and a cadaver specimen. High-resolution imaging is achieved using a complementary metal-oxide semiconductor (CMOS)-based x-ray detector featuring small pixel size and low readout noise. A cascaded systems model was used to performed task-based optimization to determine optimal detector scintillator thickness in nominal extremity CBCT imaging conditions. We validated the performance of a prototype scanner incorporating our optimization result. Strong correlation was found between bone microstructure metrics obtained from the prototype scanner and µCT gold standard for trabecular bone samples from a cadaver ulna. Additionally, we devised a multiresolution reconstruction scheme allowing fast MBIR to be applied to large, high-resolution projection data. To model the full scanned volume in the reconstruction forward model, regions outside a finely sampled region-of-interest (ROI) are downsampled, reducing runtime and cutting memory requirements while maintaining image quality in the ROI

Non-contact measures to monitor hand movement of people with rheumatoid arthritis using a monocular RGB camera

Hand movements play an essential role in a person’s ability to interact with the environment. In hand biomechanics, the range of joint motion is a crucial metric to quantify changes due to degenerative pathologies, such as rheumatoid arthritis (RA). RA is a chronic condition where the immune system mistakenly attacks the joints, particularly those in the hands. Optoelectronic motion capture systems are gold-standard tools to quantify changes but are challenging to adopt outside laboratory settings. Deep learning executed on standard video data can capture RA participants in their natural environments, potentially supporting objectivity in remote consultation. The three main research aims in this thesis were 1) to assess the extent to which current deep learning architectures, which have been validated for quantifying motion of other body segments, can be applied to hand kinematics using monocular RGB cameras, 2) to localise where in videos the hand motions of interest are to be found, 3) to assess the validity of 1) and 2) to determine disease status in RA. First, hand kinematics for twelve healthy participants, captured with OpenPose were benchmarked against those captured using an optoelectronic system, showing acceptable instrument errors below 10°. Then, a gesture classifier was tested to segment video recordings of twenty-two healthy participants, achieving an accuracy of 93.5%. Finally, OpenPose and the classifier were applied to videos of RA participants performing hand exercises to determine disease status. The inferred disease activity exhibited agreement with the in-person ground truth in nine out of ten instances, outperforming virtual consultations, which agreed only six times out of ten. These results demonstrate that this approach is more effective than estimated disease activity performed by human experts during video consultations. The end goal sets the foundation for a tool that RA participants can use to observe their disease activity from their home.Open Acces

Multiclass Bone Segmentation of PET/CT Scans for Automatic SUV Extraction

In this thesis I present an automated framework for segmentation of bone structures from dual modality PET/CT scans and further extraction of SUV measurements. The first stage of this framework consists of a variant of the 3D U-Net architecture for segmentation of three bone structures: vertebral body, pelvis, and sternum. The dataset for this model consists of annotated slices from the CT scans retrieved from the study of post-HCST patients and the 18F-FLT radiotracer, which are undersampled volumes due to the low-dose radiation used during the scanning. The mean Dice scores obtained by the proposed model are 0.9162, 0.9163, and 0.8721 for the vertebral body, pelvis, and sternum class respectively. The next step of the proposed framework consists of identifying the individual vertebrae, which is a particularly difficult task due to the low resolution of the CT scans in the axial dimension. To address this issue, I present an iterative algorithm for instance segmentation of vertebral bodies, based on anatomical priors of the spine for detecting the starting point of a vertebra. The spatial information contained in the CT and PET scans is used to translate the resulting masks to the PET image space and extract SUV measurements. I then present a CNN model based on the DenseNet architecture that, for the first time, classifies the spatial distribution of SUV within the marrow cavities of the vertebral bodies as normal engraftment or possible relapse. With an AUC of 0.931 and an accuracy of 92% obtained on real patient data, this method shows good potential as a future automated tool to assist in monitoring the recovery process of HSCT patients