Predicting outcomes of smoking cessation interventions in novel scenarios using ontology-informed, interpretable machine learning

Background Systematic reviews of effectiveness estimate the relative average effects of interventions and comparators in a set of existing studies e.g., using rate ratios. However, policymakers, planners and practitioners require predictions about outcomes in novel scenarios where aspects of the interventions, populations or settings may differ. This study aimed to develop and evaluate an ontology-informed, interpretable machine learning algorithm to predict smoking cessation outcomes using detailed information about interventions, their contexts and evaluation study methods. This is the second of two linked papers on the use of machine learning in the Human Behaviour-Change Project. Methods The study used a corpus of 405 reports of randomised trials of smoking cessation interventions from the Cochrane Library database. These were annotated using the Behaviour Change Intervention Ontology to classify, for each of 971 study arms, 82 features representing details of intervention content and delivery, population, setting, outcome, and study methodology. The annotated data was used to train a novel machine learning algorithm based on a set of interpretable rules organised according to the ontology. The algorithm was evaluated for predictive accuracy by performance in five-fold 80:20 cross-validation, and compared with other approaches. Results The machine learning algorithm produced a mean absolute error in prediction percentage cessation rates of 9.15% in cross-validation, outperforming other approaches including an uninterpretable ‘black-box’ deep neural network (9.42%), a linear regression model (10.55%) and a decision tree-based approach (9.53%). The rules generated by the algorithm were synthesised into a consensus rule set to create a publicly available predictive tool to provide outcome predictions and explanations in the form of rules expressed in terms of predictive features and their combinations. Conclusions An ontologically-informed, interpretable machine learning algorithm, using information about intervention scenarios from reports of smoking cessation trials, can predict outcomes in new smoking cessation intervention scenarios with moderate accuracy.</ns3:p

Opening the Black Box: Explaining the Process of Basing a Health Recommender System on the I-Change Behavioral Change Model

Recommender systems are gaining traction in healthcare because they can tailor recommendations based on users' feedback concerning their appreciation of previous health-related messages. However, recommender systems are often not grounded in behavioral change theories, which may further increase the effectiveness of their recommendations. This paper's objective is to describe principles for designing and developing a health recommender system grounded in the I-Change behavioral change model that shall be implemented through a mobile app for a smoking cessation support clinical trial. We built upon an existing smoking cessation health recommender system that delivered motivational messages through a mobile app. A group of experts assessed how the system may be improved to address the behavioral change determinants of the I-Change behavioral change model. The resulting system features a hybrid recommender algorithm for computer tailoring smoking cessation messages. A total of 331 different motivational messages were designed using 10 health communication methods. The algorithm was designed to match 58 message characteristics to each user pro le by following the principles of the I-Change model and maintaining the bene ts of the recommender system algorithms. The mobile app resulted in a streamlined version that aimed to improve the user experience, and this system's design bridges the gap between health recommender systems and the use of behavioral change theories. This article presents a novel approach integrating recommender system technology, health behavior technology, and computer-tailored technology. Future researchers will be able to build upon the principles applied in this case study.European Union's Horizon 2020 Research and Innovation Programme under Grant 68112

Analyzing recommender systems for health promotion using a multidisciplinary taxonomy: A scoping review

Background: Recommender systems are information retrieval systems that provide users with relevant items (e.g., through messages). Despite their extensive use in the e-commerce and leisure domains, their application in healthcare is still in its infancy. These systems may be used to create tailored health interventions, thus reducing the cost of healthcare and fostering a healthier lifestyle in the population. Objective: This paper identifies, categorizes, and analyzes the existing knowledge in terms of the literature published over the past 10 years on the use of health recommender systems for patient interventions. The aim of this study is to understand the scientific evidence generated about health recommender systems, to identify any gaps in this field to achieve the United Nations Sustainable Development Goal 3 (SDG3) (namely, “Ensure healthy lives and promote well-being for all at all ages”), and to suggest possible reasons for these gaps as well as to propose some solutions. Methods: We conducted a scoping review, which consisted of a keyword search of the literature related to health recommender systems for patients in the following databases: ScienceDirect, PsycInfo, Association for Computing Machinery, IEEExplore, and Pubmed. Further, we limited our search to consider only English-lan-guage journal articles published in the last 10 years. The reviewing process comprised three researchers who filtered the results simultaneously. The quantitative synthesis was conducted in parallel by two researchers, who classified each paper in terms of four aspects—the domain, the methodological and procedural aspects, the health promotion theoretical factors and behavior change theories, and the technical aspects—using a new multidisciplinary taxonomy. Results: Nineteen papers met the inclusion criteria and were included in the data analysis, for which thirty-three features were assessed. The nine features associated with the health promotion theoretical factors and behavior change theories were not observed in any of the selected studies, did not use principles of tailoring, and did not assess (cost)-effectiveness. Discussion: Health recommender systems may be further improved by using relevant behavior change strategies and by implementing essential characteristics of tailored interventions. In addition, many of the features required to assess each of the domain aspects, the methodological and procedural aspects, and technical aspects were not reported in the studies. Conclusions: The studies analyzed presented few evidence in support of the positive effects of using health recommender systems in terms of cost-effectiveness and patient health outcomes. This is why future studies should ensure that all the proposed features are covered in our multidisciplinary taxonomy, including integration with electronic health records and the incorporation of health promotion theoretical factors and behavior change theories. This will render those studies more useful for policymakers since they will cover all aspects needed to determine their impact toward meeting SDG3.European Union's Horizon 2020 No 68112

Knowledge Extraction and Prediction from Behavior Science Randomized Controlled Trials: A Case Study in Smoking Cessation

Due to the fast pace at which randomized controlled trials are published in the health domain, researchers, consultants and policymakers would benefit from more automatic ways to process them by both extracting relevant information and automating the meta-analysis processes. In this paper, we present a novel methodology based on natural language processing and reasoning models to 1) extract relevant information from RCTs and 2) predict potential outcome values on novel scenarios, given the extracted knowledge, in the domain of behavior change for smoking cessation

Effect on smoking quit rate of telling patients their lung age: the Step2quit randomised controlled trial

Objective To evaluate the impact of telling patients their estimated spirometric lung age as an incentive to quit smoking.Design Randomised controlled trial.Setting Five general practices in Hertfordshire, England.Participants 561 current smokers aged over 35.Intervention All participants were offered spirometric assessment of lung function. Participants in intervention group received their results in terms of "lung age" (the age of the average healthy individual who would perform similar to them on spirometry). Those in the control group received a raw figure for forced expiratory volume at one second (FEV1). Both groups were advised to quit and offered referral to local NHS smoking cessation services.Main outcome measures The primary outcome measure was verified cessation of smoking by salivary cotinine testing 12 months after recruitment. Secondary outcomes were reported changes in daily consumption of cigarettes and identification of new diagnoses of chronic obstructive lung disease.Results Follow-up was 89%. Independently verified quit rates at 12 months in the intervention and control groups, respectively, were 13.6% and 6.4% (difference 7.2%, P=0.005, 95% confidence interval 2.2% to 12.1%; number needed to treat 14). People with worse spirometric lung age were no more likely to have quit than those with normal lung age in either group. Cost per successful quitter was estimated at 280 pound ((euro) 365, $556). A new diagnosis of obstructive lung disease was made in 17% in the intervention group and 14% in the control group; a total of 16% (89/561) of participants.Conclusion Telling smokers their lung age significantly improves the likelihood of them quitting smoking, but the mechanism by which this intervention achieves its effect is unclear.Trial registration National Research Register N0096173751

Continuous Estimation of Smoking Lapse Risk from Noisy Wrist Sensor Data Using Sparse and Positive-Only Labels

Estimating the imminent risk of adverse health behaviors provides opportunities for developing effective behavioral intervention mechanisms to prevent the occurrence of the target behavior. One of the key goals is to find opportune moments for intervention by passively detecting the rising risk of an imminent adverse behavior. Significant progress in mobile health research and the ability to continuously sense internal and external states of individual health and behavior has paved the way for detecting diverse risk factors from mobile sensor data. The next frontier in this research is to account for the combined effects of these risk factors to produce a composite risk score of adverse behaviors using wearable sensors convenient for daily use. Developing a machine learning-based model for assessing the risk of smoking lapse in the natural environment faces significant outstanding challenges requiring the development of novel and unique methodologies for each of them. The first challenge is coming up with an accurate representation of noisy and incomplete sensor data to encode the present and historical influence of behavioral cues, mental states, and the interactions of individuals with their ever-changing environment. The next noteworthy challenge is the absence of confirmed negative labels of low-risk states and adequate precise annotations of high-risk states. Finally, the model should work on convenient wearable devices to facilitate widespread adoption in research and practice. In this dissertation, we develop methods that account for the multi-faceted nature of smoking lapse behavior to train and evaluate a machine learning model capable of estimating composite risk scores in the natural environment. We first develop mRisk, which combines the effects of various mHealth biomarkers such as stress, physical activity, and location history in producing the risk of smoking lapse using sequential deep neural networks. We propose an event-based encoding of sensor data to reduce the effect of noises and then present an approach to efficiently model the historical influence of recent and past sensor-derived contexts on the likelihood of smoking lapse. To circumvent the lack of confirmed negative labels (i.e., annotated low-risk moments) and only a few positive labels (i.e., sensor-based detection of smoking lapse corroborated by self-reports), we propose a new loss function to accurately optimize the models. We build the mRisk models using biomarker (stress, physical activity) streams derived from chest-worn sensors. Adapting the models to work with less invasive and more convenient wrist-based sensors requires adapting the biomarker detection models to work with wrist-worn sensor data. To that end, we develop robust stress and activity inference methodologies from noisy wrist-sensor data. We first propose CQP, which quantifies wrist-sensor collected PPG data quality. Next, we show that integrating CQP within the inference pipeline improves accuracy-yield trade-offs associated with stress detection from wrist-worn PPG sensors in the natural environment. mRisk also requires sensor-based precise detection of smoking events and confirmation through self-reports to extract positive labels. Hence, we develop rSmoke, an orientation-invariant smoking detection model that is robust to the variations in sensor data resulting from orientation switches in the field. We train the proposed mRisk risk estimation models using the wrist-based inferences of lapse risk factors. To evaluate the utility of the risk models, we simulate the delivery of intelligent smoking interventions to at-risk participants as informed by the composite risk scores. Our results demonstrate the envisaged impact of machine learning-based models operating on wrist-worn wearable sensor data to output continuous smoking lapse risk scores. The novel methodologies we propose throughout this dissertation help instigate a new frontier in smoking research that can potentially improve the smoking abstinence rate in participants willing to quit

Artificial Intelligence and Behavioral Science Through the Looking Glass: Challenges for Real-World Application

Background: Artificial Intelligence (AI) is transforming the process of scientific research. AI, coupled with availability of large datasets and increasing computational power, is accelerating progress in areas such as genetics, climate change and astronomy [NeurIPS 2019 Workshop Tackling Climate Change with Machine Learning, Vancouver, Canada; Hausen R, Robertson BE. Morpheus: A deep learning framework for the pixel-level analysis of astronomical image data. Astrophys J Suppl Ser. 2020;248:20; Dias R, Torkamani A. AI in clinical and genomic diagnostics. Genome Med. 2019;11:70.]. The application of AI in behavioral science is still in its infancy and realizing the promise of AI requires adapting current practices. Purposes: By using AI to synthesize and interpret behavior change intervention evaluation report findings at a scale beyond human capability, the HBCP seeks to improve the efficiency and effectiveness of research activities. We explore challenges facing AI adoption in behavioral science through the lens of lessons learned during the Human Behaviour-Change Project (HBCP). Methods: The project used an iterative cycle of development and testing of AI algorithms. Using a corpus of published research reports of randomized controlled trials of behavioral interventions, behavioral science experts annotated occurrences of interventions and outcomes. AI algorithms were trained to recognize natural language patterns associated with interventions and outcomes from the expert human annotations. Once trained, the AI algorithms were used to predict outcomes for interventions that were checked by behavioral scientists. Results: Intervention reports contain many items of information needing to be extracted and these are expressed in hugely variable and idiosyncratic language used in research reports to convey information makes developing algorithms to extract all the information with near perfect accuracy impractical. However, statistical matching algorithms combined with advanced machine learning approaches created reasonably accurate outcome predictions from incomplete data. Conclusions: AI holds promise for achieving the goal of predicting outcomes of behavior change interventions, based on information that is automatically extracted from intervention evaluation reports. This information can be used to train knowledge systems using machine learning and reasoning algorithms

Precision public health – A critical review of the opportunities and obstacles

The term ‘precision public health’ (PPH) refers to a new approach in public health which involves the use of novel data sources and/or computer science-driven methods of data analysis to predict risk or outcomes, in order to improve how interventions are targeted or tailored, with the aim of making them more individualised and therefore more effective and cost-effective than methods currently in use. These data may include, for example, information from social media or devices, genomic or clinical data, and information from healthcare services. In this critical review, which was conducted between March and October 2019, we outline key assumptions underpinning the PPH approach and identify potential challenges in its application. We adopted a pragmatic, non-systematic review methodology to examine: (i) the general principles underlying PPH; (ii) the validity of claims made about PPH in empirical studies and commentaries; and (iii) the potential opportunities and challenges of adopting a PPH approach through examining two case studies: health checks and community-based interventions. Non-empirical studies (commentaries and think-pieces) were included in this review because PPH represents an emerging approach and many of the ideas around the potential of PPH are only described in such studies Commentary studies emphasise that precision can be achieved in targeting interventions towards narrow social profiles of people through the incorporation of data reflecting micro-level day-to-day insights into the lives of individuals. Structured analysis of commentary studies shows that (i) the PPH field may be highly influenced by commentary and non-systematic review pieces that lack transparent methods but make claims about the potential of PPH; (ii) commentators on PPH often attempt to provide evidence for claims but the link between the evidence and the claim is often unsubstantiated when critically examined; and (iii) many of the assumptions underlying PPH are not supported by empirical evidence suggesting that there needs to be a measured approach to adopting PPH approaches. Claims around the effectiveness of PPH and around PPH being an advance on current public health approaches tended not to be supported by empirical evidence. As a relatively new concept therefore, there is limited direct empirical evidence showing PPH to be effective, and the theoretical arguments in its favour are often not well supported by evidence. The more ambitious claims made for PPH in the literature often rest on questionable readings of the evidence – for example, citing the possibility of identifying subgroups of the population through better targeting as though this automatically promises greater effectiveness among interventions targeting those subgroups. In practice, it seems that PPH is less a radically new paradigm and more a range of incremental improvements to public health interventions. Hypothetical case studies outlining the potential of a PPH approach applied to health checks and community-based interventions indicate several ways in which new data or tools could be productively used to inform the design and implementation of public health interventions. Current evidence suggests the impact of these is likely to be fairly modest, although further focused research (e.g. exploring the utility of strategies for targeting or involving communities using PPH) may merit further exploration and evaluation. Defining PPH is contentious and our findings reflect the difficulty in assessing and operationalising the broad ambition of using emerging data and technologies to better understand profiles, predict risk and outcomes, and act upon this evidence. Future work in this area should seek to introduce more focus around the concept of PPH, including being clearer about the goals and breaking down the concept into a series of components that can each be evaluated. The bulk of the work presented here took place between March and October 2019. There is scope for further analysis to understand the potential of PPH in the future, as the number of studies adopting a PPH approach grows. This larger pool of studies may also lend itself to more systematic approaches to reviewing the evidence, particularly if there is an interest in evaluating a particular component or principle of PPH. In addition, the evidence examined in this report predates the COVID-19 global pandemic, and many of the measures taken to mitigate the spread of the pandemic may provide a further source of evidence and data to understand the potential role of PPH in public health decision-making

Predicting adolescent social networks to stop smoking in secondary schools

Social networks are increasingly being investigated in the context of individual behaviours. Research suggests that friendship connections have the ability to influence individual actions, change personal opinions and subsequently impact upon personal wellbeing. This paper explores the effect of individual friendship selection decisions, and the impact they may have on the overall evolution of a social network. Using data from a large smoking cessation programme in secondary schools, an agent based simulation aiming to predict the evolution of the adolescent social networks is created. The simulation uses existing friendship selection algorithms from link prediction literature, along with a new approach to link prediction, termed PageRank-Max. This new algorithm is based upon the optimisation of an individuals eigen-centrality, and is found to be more successful than existing methods at predicting the future state of an adolescent social network. This research highlights the importance of eigen-centrality in adolescent friendship decisions, and the use of agent-based simulation to conduct behavioural investigations. Furthermore, it provides a proof-of-concept for targeted interventions driven by social network analysis, demonstrating the utility of using emerging sources of social network data for public heath interventions such as with tobacco use which is a major global health challenge