Non-contact measures to monitor hand movement of people with rheumatoid arthritis using a monocular RGB camera

Hand movements play an essential role in a person’s ability to interact with the environment. In hand biomechanics, the range of joint motion is a crucial metric to quantify changes due to degenerative pathologies, such as rheumatoid arthritis (RA). RA is a chronic condition where the immune system mistakenly attacks the joints, particularly those in the hands. Optoelectronic motion capture systems are gold-standard tools to quantify changes but are challenging to adopt outside laboratory settings. Deep learning executed on standard video data can capture RA participants in their natural environments, potentially supporting objectivity in remote consultation. The three main research aims in this thesis were 1) to assess the extent to which current deep learning architectures, which have been validated for quantifying motion of other body segments, can be applied to hand kinematics using monocular RGB cameras, 2) to localise where in videos the hand motions of interest are to be found, 3) to assess the validity of 1) and 2) to determine disease status in RA. First, hand kinematics for twelve healthy participants, captured with OpenPose were benchmarked against those captured using an optoelectronic system, showing acceptable instrument errors below 10°. Then, a gesture classifier was tested to segment video recordings of twenty-two healthy participants, achieving an accuracy of 93.5%. Finally, OpenPose and the classifier were applied to videos of RA participants performing hand exercises to determine disease status. The inferred disease activity exhibited agreement with the in-person ground truth in nine out of ten instances, outperforming virtual consultations, which agreed only six times out of ten. These results demonstrate that this approach is more effective than estimated disease activity performed by human experts during video consultations. The end goal sets the foundation for a tool that RA participants can use to observe their disease activity from their home.Open Acces

DeepRA: Predicting Joint Damage From Radiographs Using CNN with Attention

Joint damage in Rheumatoid Arthritis (RA) is assessed by manually inspecting and grading radiographs of hands and feet. This is a tedious task which requires trained experts whose subjective assessment leads to low inter-rater agreement. An algorithm which can automatically predict the joint level damage in hands and feet can help optimize this process, which will eventually aid the doctors in better patient care and research. In this paper, we propose a two-staged approach which amalgamates object detection and convolution neural networks with attention which can efficiently and accurately predict the overall and joint level narrowing and erosion from patients radiographs. This approach has been evaluated on hands and feet radiographs of patients suffering from RA and has achieved a weighted root mean squared error (RMSE) of 1.358 and 1.404 in predicting joint level narrowing and erosion Sharp van der Heijde (SvH) scores which is 31% and 19% improvement with respect to the baseline SvH scores, respectively. The proposed approach achieved a weighted absolute error of 1.456 in predicting the overall damage in hands and feet radiographs for the patients which is a 79% improvement as compared to the baseline. Our method also provides an inherent capability to provide explanations for model predictions using attention weights, which is essential given the black box nature of deep learning models. The proposed approach was developed during the RA2 Dream Challenge hosted by Dream Challenges and secured 4th and 8th position in predicting overall and joint level narrowing and erosion SvH scores from radiographs

Visualizing and Predicting the Effects of Rheumatoid Arthritis on Hands

This dissertation was inspired by difficult decisions patients of chronic diseases have to make about about treatment options in light of uncertainty. We look at rheumatoid arthritis (RA), a chronic, autoimmune disease that primarily affects the synovial joints of the hands and causes pain and deformities. In this work, we focus on several parts of a computer-based decision tool that patients can interact with using gestures, ask questions about the disease, and visualize possible futures. We propose a hand gesture based interaction method that is easily setup in a doctor\u27s office and can be trained using a custom set of gestures that are least painful. Our system is versatile and can be used for operations like simple selections to navigating a 3D world. We propose a point distribution model (PDM) that is capable of modeling hand deformities that occur due to RA and a generalized fitting method for use on radiographs of hands. Using our shape model, we show novel visualization of disease progression. Using expertly staged radiographs, we propose a novel distance metric learning and embedding technique that can be used to automatically stage an unlabeled radiograph. Given a large set of expertly labeled radiographs, our data-driven approach can be used to extract different modes of deformation specific to a disease

Role of deep learning in predicting aging-related diseases:A scoping review

Aging refers to progressive physiological changes in a cell, an organ, or the whole body of an individual, over time. Aging-related diseases are highly prevalent and could impact an individual’s physical health. Recently, artificial intelligence (AI) methods have been used to predict aging-related diseases and issues, aiding clinical providers in decision-making based on patient’s medical records. Deep learning (DL), as one of the most recent generations of AI technologies, has embraced rapid progress in the early prediction and classification of aging-related issues. In this paper, a scoping review of publications using DL approaches to predict common aging-related diseases (such as age-related macular degeneration, cardiovascular and respiratory diseases, arthritis, Alzheimer’s and lifestyle patterns related to disease progression), was performed. Google Scholar, IEEE and PubMed are used to search DL papers on common aging-related issues published between January 2017 and August 2021. These papers were reviewed, evaluated, and the findings were summarized. Overall, 34 studies met the inclusion criteria. These studies indicate that DL could help clinicians in diagnosing disease at its early stages by mapping diagnostic predictions into observable clinical presentations; and achieving high predictive performance (e.g., more than 90% accurate predictions of diseases in aging)

Artificial intelligence in musculoskeletal ultrasound imaging

Ultrasonography (US) is noninvasive and offers real-time, low-cost, and portable imaging that facilitates the rapid and dynamic assessment of musculoskeletal components. Significant technological improvements have contributed to the increasing adoption of US for musculoskeletal assessments, as artificial intelligence (AI)-based computer-aided detection and computer-aided diagnosis are being utilized to improve the quality, efficiency, and cost of US imaging. This review provides an overview of classical machine learning techniques and modern deep learning approaches for musculoskeletal US, with a focus on the key categories of detection and diagnosis of musculoskeletal disorders, predictive analysis with classification and regression, and automated image segmentation. Moreover, we outline challenges and a range of opportunities for AI in musculoskeletal US practice.11Nsciescopu

Review of Wearable Devices and Data Collection Considerations for Connected Health

Wearable sensor technology has gradually extended its usability into a wide range of well-known applications. Wearable sensors can typically assess and quantify the wearer’s physiology and are commonly employed for human activity detection and quantified self-assessment. Wearable sensors are increasingly utilised to monitor patient health, rapidly assist with disease diagnosis, and help predict and often improve patient outcomes. Clinicians use various self-report questionnaires and well-known tests to report patient symptoms and assess their functional ability. These assessments are time consuming and costly and depend on subjective patient recall. Moreover, measurements may not accurately demonstrate the patient’s functional ability whilst at home. Wearable sensors can be used to detect and quantify specific movements in different applications. The volume of data collected by wearable sensors during long-term assessment of ambulatory movement can become immense in tuple size. This paper discusses current techniques used to track and record various human body movements, as well as techniques used to measure activity and sleep from long-term data collected by wearable technology devices

Ultrasound and fluorescence optical imaging biomarkers for early diagnosis and prediction of rheumatoid arthritis

Prevention of rheumatoid arthritis (RA) is most desirable together with curative treatment which is, however, not yet available. Today, the correct timely diagnosis and early treatment interventions to prevent disease progression remain the best options for our patients. In this thesis, I explore the diagnostic and predictive value of musculoskeletal ultrasound (MSUS) and fluorescence optical imaging (FOI) in identifying biological features on images indicative of existing or emerging joint inflammation (synovitis). In study 1, we tested and compared the diagnostic utility of FOI with clinical examination and musculoskeletal ultrasound (MSUS) to detect active synovitis in 872 joints of 26 patients with different rheumatic diseases (46% early RA). Fluorescence optical imaging proved to be 80% sensitive and 96% specific, having a 77% positive predictive value (PPV) and 97% negative predictive value (NPV) for detecting silent synovitis. In study 2 we showed FOI’s ability to quantify digital disease activity (DACT) scores of 1326 joints in 39 early RA patients to be 81% sensitive and 90% specific, with 96% PPV and 61% NPV. These results justify FOI use in clinical practice, to assist the rheumatologist to make an earlier diagnosis with greater confidence. Unsupervised cluster differences emerged for seropositive and seronegative RA patients showing FOI’s ability to objectively quantify hand joint inflammation using novel DACT scoring methods. In study 3 we report good association among the two ultrasound semi-quantitative scoring (SQS) methods to that of a novel quantitative scoring (QS) measure of color Doppler pixel counts in 37 established RA patients. Although SQS well correlated with QS to assess active synovitis, the SQS methods lacked visual perceptions of raters to distinguish between grade cut-offs which may help to further revise the criteria used to objectively quantify disease activity. In study 4, we show the value of ultrasound and immune-inflammatory biomarkers in predicting arthritis onset in individuals positive for Anti-CCP with musculoskeletal complaints at risk of RA development. We propose the recognition of a high-risk RA phase characterized by presence of certain ACPA reactivities, IL15-Rα, IL6; and ultrasound detected tenosynovitis, and possibilities to identify (low and high) risk groups for arthritis progression. Overall, our findings on imaging contribute towards a) silent synovitis detection despite negative clinical investigation, b) objective quantitative measures to monitor the effects of RA therapy and c) early identification of certain predictive imaging and biological features/biomarkers that precede arthritis development (tenosynovitis and/or bursitis) in individuals at risk for developing RA, enabling closer monitoring and early diagnosis

Novel Approaches to the Representation and Analysis of 3D Segmented Anatomical Districts

Nowadays, image processing and 3D shape analysis are an integral part of clinical practice and have the potentiality to support clinicians with advanced analysis and visualization techniques. Both approaches provide visual and quantitative information to medical practitioners, even if from different points of view. Indeed, shape analysis is aimed at studying the morphology of anatomical structures, while image processing is focused more on the tissue or functional information provided by the pixels/voxels intensities levels. Despite the progress obtained by research in both fields, a junction between these two complementary worlds is missing. When working with 3D models analyzing shape features, the information of the volume surrounding the structure is lost, since a segmentation process is needed to obtain the 3D shape model; however, the 3D nature of the anatomical structure is represented explicitly. With volume images, instead, the tissue information related to the imaged volume is the core of the analysis, while the shape and morphology of the structure are just implicitly represented, thus not clear enough. The aim of this Thesis work is the integration of these two approaches in order to increase the amount of information available for physicians, allowing a more accurate analysis of each patient. An augmented visualization tool able to provide information on both the anatomical structure shape and the surrounding volume through a hybrid representation, could reduce the gap between the two approaches and provide a more complete anatomical rendering of the subject. To this end, given a segmented anatomical district, we propose a novel mapping of volumetric data onto the segmented surface. The grey-levels of the image voxels are mapped through a volume-surface correspondence map, which defines a grey-level texture on the segmented surface. The resulting texture mapping is coherent to the local morphology of the segmented anatomical structure and provides an enhanced visual representation of the anatomical district. The integration of volume-based and surface-based information in a unique 3D representation also supports the identification and characterization of morphological landmarks and pathology evaluations. The main research contributions of the Ph.D. activities and Thesis are: \u2022 the development of a novel integration algorithm that combines surface-based (segmented 3D anatomical structure meshes) and volume-based (MRI volumes) information. The integration supports different criteria for the grey-levels mapping onto the segmented surface; \u2022 the development of methodological approaches for using the grey-levels mapping together with morphological analysis. The final goal is to solve problems in real clinical tasks, such as the identification of (patient-specific) ligament insertion sites on bones from segmented MR images, the characterization of the local morphology of bones/tissues, the early diagnosis, classification, and monitoring of muscle-skeletal pathologies; \u2022 the analysis of segmentation procedures, with a focus on the tissue classification process, in order to reduce operator dependency and to overcome the absence of a real gold standard for the evaluation of automatic segmentations; \u2022 the evaluation and comparison of (unsupervised) segmentation methods, finalized to define a novel segmentation method for low-field MR images, and for the local correction/improvement of a given segmentation. The proposed method is simple but effectively integrates information derived from medical image analysis and 3D shape analysis. Moreover, the algorithm is general enough to be applied to different anatomical districts independently of the segmentation method, imaging techniques (such as CT), or image resolution. The volume information can be integrated easily in different shape analysis applications, taking into consideration not only the morphology of the input shape but also the real context in which it is inserted, to solve clinical tasks. The results obtained by this combined analysis have been evaluated through statistical analysis

Deep Learning Paradigm for Cardiovascular Disease/Stroke Risk Stratification in Parkinson’s Disease Affected by COVID‐19: A Narrative Review

Background and Motivation: Parkinson’s disease (PD) is one of the most serious, non-curable, and expensive to treat. Recently, machine learning (ML) has shown to be able to predict cardiovascular/stroke risk in PD patients. The presence of COVID‐19 causes the ML systems to be-come severely non‐linear and poses challenges in cardiovascular/stroke risk stratification. Further, due to comorbidity, sample size constraints, and poor scientific and clinical validation techniques, there have been no well‐explained ML paradigms. Deep neural networks are powerful learning machines that generalize non‐linear conditions. This study presents a novel investigation of deep learning (DL) solutions for CVD/stroke risk prediction in PD patients affected by the COVID‐19 framework. Method: The PRISMA search strategy was used for the selection of 292 studies closely associated with the effect of PD on CVD risk in the COVID‐19 framework. We study the hypothesis that PD in the presence of COVID‐19 can cause more harm to the heart and brain than in non‐ COVID‐19 conditions. COVID‐19 lung damage severity can be used as a covariate during DL training model designs. We, therefore, propose a DL model for the estimation of, (i) COVID‐19 lesions in computed tomography (CT) scans and (ii) combining the covariates of PD, COVID‐19 lesions, office and laboratory arterial atherosclerotic image‐based biomarkers, and medicine usage for the PD patients for the design of DL point‐based models for CVD/stroke risk stratification. Results: We validated the feasibility of CVD/stroke risk stratification in PD patients in the presence of a COVID‐ 19 environment and this was also verified. DL architectures like long short‐term memory (LSTM), and recurrent neural network (RNN) were studied for CVD/stroke risk stratification showing powerful designs. Lastly, we examined the artificial intelligence bias and provided recommendations for early detection of CVD/stroke in PD patients in the presence of COVID‐19. Conclusion: The DL is a very powerful tool for predicting CVD/stroke risk in PD patients affected by COVID‐19. © 2022 by the authors. Licensee MDPI, Basel, Switzerland