A Survey of Multimodal Information Fusion for Smart Healthcare: Mapping the Journey from Data to Wisdom

Multimodal medical data fusion has emerged as a transformative approach in smart healthcare, enabling a comprehensive understanding of patient health and personalized treatment plans. In this paper, a journey from data to information to knowledge to wisdom (DIKW) is explored through multimodal fusion for smart healthcare. We present a comprehensive review of multimodal medical data fusion focused on the integration of various data modalities. The review explores different approaches such as feature selection, rule-based systems, machine learning, deep learning, and natural language processing, for fusing and analyzing multimodal data. This paper also highlights the challenges associated with multimodal fusion in healthcare. By synthesizing the reviewed frameworks and theories, it proposes a generic framework for multimodal medical data fusion that aligns with the DIKW model. Moreover, it discusses future directions related to the four pillars of healthcare: Predictive, Preventive, Personalized, and Participatory approaches. The components of the comprehensive survey presented in this paper form the foundation for more successful implementation of multimodal fusion in smart healthcare. Our findings can guide researchers and practitioners in leveraging the power of multimodal fusion with the state-of-the-art approaches to revolutionize healthcare and improve patient outcomes.Comment: This work has been submitted to the ELSEVIER for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Translational Research of Audiovisual Biofeedback: An investigation of respiratory-guidance in lung and liver cancer patient radiation therapy

Through the act of breathing, thoracic and abdominal anatomy is in constant motion and is typically irregular. This irregular motion can exacerbate errors in radiation therapy, breathing guidance interventions operate to minimise these errors. However, much of the breathing guidance investigations have not directly quantified the impact of regular breathing on radiation therapy accuracy. The first aim of this thesis was to critically appraise the literature in terms of the use of breathing guidance interventions via systematic review. This review found that 21 of the 27 identified studies yielded significant improvements from the use of breathing guidance. None of the studies were randomised and no studies quantified the impact on 4DCT image quality. The second aim of this thesis was to quantify the impact of audiovisual biofeedback breathing guidance on 4DCT. This study utilised data from an MRI study to program the motion of a digital phantom prior to then simulating 4DCT imaging. Audiovisual biofeedback demonstrated to significantly improved 4DCT image quality over free breathing. The third aim of this thesis was to assess the impact of audiovisual biofeedback on liver cancer patient breathing over a course of stereotactic body radiation therapy (SBRT). The findings of this study demonstrated the effectiveness of audiovisual biofeedback in producing consistent interfraction respiratory motion over a course of SBRT. The fourth aim of this thesis was to design and implement a phase II clinical trial investigating the use and impact of audiovisual biofeedback in lung cancer radiation therapy. The findings of a retrospective analysis were utilised to design and determine the statistics of the most comprehensive breathing guidance study to date: a randomised, stratified, multi-site, phase II clinical trial.. The fifth aim of this thesis was to explore the next stages of audiovisual biofeedback in terms of translating evidence into broader clinical use through commercialisation. This aim was achieved by investigating the the product-market fit of the audiovisual biofeedback technology. The culmination of these findings demonstrates the clinical benefit of the audiovisual biofeedback respiratory guidance system and the possibility to make breathing guidance systems more widely available to patients

Development of personalised optimisation of advanced-stage non-small cell lung cancer patients with volumetric modulated arc radiotherapy

Background and Aim of the study: Lung cancer is the third most common cancer in the UK. A significant number of these patients are diagnosed with inoperable advanced-stage non-small cell lung cancer. Until recently, the standard of care for these patients was radiotherapy with or without chemotherapy but the overall survival remained poor, with a 5-year survival of only 13%. Recent studies showed significant improvement in overallsurvival in patients who are suitable for, and have received, immunotherapy, in addition to chemotherapy and radiotherapy. Radiotherapy aims to deliver tumoricidal doses to the target volume whilst minimising doses to the surrounding organs at risk (OAR). However, achieving this goal could be challenging especially when treating advanced-stage tumours, as it could increase OAR doses and increase toxicities to an unacceptable level. Furthermore, several factors affect the achieved dose distribution including, patient’s geometry, treatment technique, planner’s experience, beam geometry, optimisation parameters, and interventions used during treatments. It is therefore important to develop methods to personalise treatment plan optimisation for advanced-stage non-small cell lung cancer (NSCLC) patients to achieve minimum OAR doses without compromising target doses using patient-specific parameters. This study aims to develop knowledge-based planning models (KBP) using patient-specific factors todetermine personalised treatment planning optimisation for advanced-stage NSCLC patients treated with volumetric modulated arc therapy (VMAT) to reduce OAR doses whilst delivering intended doses to the target volume.Methods: Four KBP models were developed using patient-specific dose and volume parameters to predict minimum achievable OAR doses, identify optimal arc parameters, trigger adaptive radiotherapy and estimate doses to adapted gross tumour volume using patients’ geometry. The KBP models were verified using independent patient data sets. Change in treatment plan optimisation could increase modulation and affect plandeliverability therefore, several modulation indexes were calculated and plans were measured on the clinical linear accelerator to assess the effect of change in optimisation on treatment plan delivery.Results: The KBP models developed showed that relatively simple models can predict OAR doses and arc parameters and help identify patients for adaptive radiotherapy. The models can accurately estimate personalised and progressive dose escalation. The KBP resulted in a significant reduction in plan variability in all three studied dosimetric parameters, volume of lungs receiving 5Gy (V5), 20Gy (V20) and mean lung dose (MLD)by 4.9% (p=0.007, 10.8% to 5.9%), 1.3% (p=0.038, 4.0% to 2.7%) and 0.9Gy (p=0.012, 2.5Gy to 1.6Gy), respectively. The individualised arc geometry resulted in a significant reduction in lungs (V5 = - 15.1%, MLD = - 1.0Gy) and heart (MHD = - 1.4Gy) doses without compromising target coverage. The models, which were developed to predict changes in PTV coverage (∆95) using a specific biomarker (Programmed death-ligand 1 (PD-L1 expression)) and the difference in ‘planning’ and ‘fraction’ planning target volume (PTV) centre of the mass (characterised by mean square difference, MSD), could predict change in PTV coverage within ± 1.0% for 77% of the total fractions. Furthermore, the models developed for predicting personalised and progressive dose escalation predicted doses within 0.4% and 0.7% respectively. Additionally, the plan complexity and deliverability measurements show that plan complexity could increase but may not affect treatment delivery significantly.Conclusion: The studies performed show that altering the ‘standard’ treatment planning optimisation approach could significantly reduce OAR doses and improve target coverage. This will help reduce toxicities and improve local control and overall survival and outcome for inoperable advanced-stage NSCLC patients

Advances towards the use of radical radiotherapy in malignant pleural mesothelioma

Background: The complex shape of the pleural cavity and the close proximity of normal radiosensitive structures render the delivery of radical radiotherapy in malignant pleural mesothelioma (MPM) challenging. However, the advent of conformal, intensity modulated radiotherapy (IMRT), where dose is selectively delivered to the tumour whilst sparing normal tissues, can facilitate safe dose escalation. SYSTEMS-2 is the only randomised controlled trial of radiotherapy dose escalation to be attempted in MPM and is comparing the palliative efficacy of two hypofractionated radiotherapy regimes to sites of pain using conformal techniques. Although traditionally associated with unacceptable late normal tissue toxicity, the success of stereotactic radiotherapy (SABR) and the discovery that two common malignancies exhibit low α/β ratios, has enhanced the popularity of hypofractionated regimes. While the radiobiology of MPM is not well understood, its slow growth and apparent radioresistance suggests that it may exhibit a low α/β ratio and therefore that it may respond more favourably to dose hypofractionation. Aims of thesis: To investigate the possibility of further radiotherapy dose escalation in MPM, beyond that delivered in the SYSTEMS-2 study. Methods: I. Novel radiotherapy dose constraints were generated for use in the SYSTEMS-2 study and tested on five patients from the SYSTEMS study. II. Multi criteria optimisation (MCO) software was used to assess whether the original dose escalated radiotherapy plans for the Glasgow cohort of SYSTEMS-2 could be improved, without compromising target volume coverage. III. A clinically relevant 3D in vitro spheroid model was used to investigate the radiobiology of two independent MPM cell lines (H2052 and 211H). Spheroids were established and exposed to the same total dose of ionising radiation (IR) delivered in different doses per fraction. Data was used to investigate response to dose fractionation and to estimate the α/β ratio of this tumour. IV. The response of H2052 and 211H spheroids to two radiosensiting agents was investigated in combination with fractionated radiotherapy. Spheroids were incubated with increasing concentrations of either NU7441 (a DNA-PKcs inhibitor) or A1331852 (a BH3 mimetic) before being exposed to fractionated IR. The immunohistochemical (IHC) expression of DNA-PKcs and Bcl-xL was explored in diagnostic biopsies obtained from MPM patients to investigate clinical validity of the targets. V. IHC expression of nine proteins, selected for their potential to impact on radioresponse, was analysed in diagnostic tumour tissue collected from SYSTEMS and SYSTEMS-2 patients. Expression data was correlated with baseline clinical trial data in all patients, and with clinical trial outcome data from SYSTEMS patients. Results: I. Initial planning studies showed that none of the five SYSTEMS patients met all of the SYSTEMS-2 dose constraints, but the plans were deemed to be potentially clinically acceptable and the constraints were taken forward in the trial. The value of familiarity with a planning technique was evidenced by the fact that all constraints were achieved when the cases were re-planned by the same staff member in April 2019. II. MCO re-planning of dose escalated SYSTEMS-2 plans achieved clinically significant dose reductions to organs at risk (OAR) without compromising target volume coverage in 13/20 cases. Plans which did not meet OAR constraints or conform to the prescribed target volume coverage may still have been clinically acceptable. III. In vitro studies confirmed that growth of MPM spheroids can be delayed by IR. Spheroids demonstrated sensitivity to changes in dose per fraction, with the greatest volume reductions observed in hypofractionated radiotherapy regimes. This data implies that these MPM cell lines may exhibit a low α/β ratio, a suggestion which was further supported by in vitro multi-fraction IR studies. IV. Data suggest that NU7441 and A1331852 are potent radiosensitisers of MPM spheroids and that both are valid clinical targets in MPM. The supposition that a BH3 mimetic may offer tumour specific radiosensitisation, combined with the observation that A1331852 demonstrated greatest efficacy with hypofractionated IR, suggests that this agent may be clinically valuable in the radiosensitisation of MPM. V. No statistically significant correlations were found between baseline clinical characteristics and expression of the proteins of interest and no potential biomarkers of radiosensitisation were identified in the SYSTEMS cohort. Conclusions: Novel dose constraints are being used to facilitate the delivery of hypofractionated, dose escalated palliative radiotherapy in the SYSTEMS-2 study. Results from this trial may guide future dose escalation in this disease and data from MCO planning studies suggest that further dose escalation to the target volume may be feasible without breaching OAR limits. In vitro studies suggest that MPM is sensitive to IR, responds more effectively to dose hypofractionation and may have a low α/β ratio. This data may be helpful in determining dose and fractionation regimes in future MPM radiotherapy trials. Combination of BH3 mimetics with IR may provide MPM specific radiosensitisation, achieving greatest efficacy with dose hypofractionation. Ongoing IHC analysis of tumour samples from the SYSTEMS-2 study may identify a biomarker of radiotherapy response which would be helpful in guiding radiotherapy treatment decisions for future patients. In summary, this thesis has investigated ways in which radiotherapy could be delivered with radical intent in MPM. Practical aspects of radiotherapy planning and delivery have been considered and are presented in conjunction with laboratory data to demonstrate how technical advances can be combined with an appreciation of disease radiobiology to facilitate radical treatment

Computational Modelling in the Management of Patients with Aortic Valve Stenosis

Background Stenosis of the aortic valve causes increased left ventricular pressure leading to adverse clinical outcomes. The selection and timing of intervention (surgical replacement or transcatheter implantation) is often unclear and is based upon limited data. Hypothesis A comprehensive and integrated personalised approach, including recognition of cardiac energetics parameters extracted from a personalised mathematical model, mapped to patient activity, has the potential to improve diagnosis and the planning and timing of interventions. Aims This project seeks to implement a simple, personalised, mathematical model of patients with aortic stenosis (AS), which can ‘measure’ cardiac work and power parameters that provide an effective characterisation of the demand on the heart in both rest and exercise conditions and can predict the changes of these parameters following an intervention. The specific aims of this project are: • to critically review current diagnostic methods • to evaluate the potential role of pre- and post-procedural measured patient activity • to implement a simple, personalised, mathematical model of patients with AS • to evaluate the potential role of a clinical decision support system Methods Twenty-two patients with severe AS according to ESC criteria were recruited. Relevant clinical, imaging, activity monitoring, six-minute walk test, and patient reported data were collected, before and early and after treatment. Novel imaging techniques were developed to help in the diagnosis of AS. A computational model was developed and executed using the data collected to create non-invasive pressure volume loops and study the global haemodynamic burden on the left ventricle. Simulations were run to predict the haemodynamic parameters both during exercise and following intervention. Modelled parameters were validated against clinically measured values. This information was then correlated with symptoms and activity data. A clinical decision support tool was created and populated with data obtained and its clinical utility evaluated. Outcomes The results of this project suggest that the combination of imaging and activity data with computational modelling provides a novel, patient-specific insight into patients’ haemodynamics and may help guide clinical decision making in patients with AS

Book of Abstracts 15th International Symposium on Computer Methods in Biomechanics and Biomedical Engineering and 3rd Conference on Imaging and Visualization

In this edition, the two events will run together as a single conference, highlighting the strong connection with the Taylor & Francis journals: Computer Methods in Biomechanics and Biomedical Engineering (John Middleton and Christopher Jacobs, Eds.) and Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization (JoãoManuel R.S. Tavares, Ed.). The conference has become a major international meeting on computational biomechanics, imaging andvisualization. In this edition, the main program includes 212 presentations. In addition, sixteen renowned researchers will give plenary keynotes, addressing current challenges in computational biomechanics and biomedical imaging. In Lisbon, for the first time, a session dedicated to award the winner of the Best Paper in CMBBE Journal will take place. We believe that CMBBE2018 will have a strong impact on the development of computational biomechanics and biomedical imaging and visualization, identifying emerging areas of research and promoting the collaboration and networking between participants. This impact is evidenced through the well-known research groups, commercial companies and scientific organizations, who continue to support and sponsor the CMBBE meeting series. In fact, the conference is enriched with five workshops on specific scientific topics and commercial software.info:eu-repo/semantics/draf