Calibration of uncertainty in the active learning of machine learning force fields

FFLUX is a machine learning force field that uses the maximum expected prediction error (MEPE) active learning algorithm to improve the efficiency of model training. MEPE uses the predictive uncertainty of a Gaussian process (GP) to balance exploration and exploitation when selecting the next training sample. However, the predictive uncertainty of a GP is unlikely to be accurate or precise immediately after training. We hypothesize that calibrating the uncertainty quantification within MEPE will improve active learning performance. We develop and test two methods to improve uncertainty estimates: post-hoc calibration of predictive uncertainty using the CRUDE algorithm, and replacing the GP with a student- t process. We investigate the impact of these methods on MEPE for single sample and batch sample active learning. Our findings suggest that post-hoc calibration does not improve the performance of active learning using the MEPE method. However, we do find that the student- t process can outperform active learning strategies and random sampling using a GP if the training set is sufficiently large

Group interventions to improve health outcomes : a framework for their design and delivery

Peer reviewedPublisher PD

Screening families of patients with premature coronary heart disease to identify avoidable cardiovascular risk: a cross-sectional study of family members and a general population comparison group

<b>Background:</b> Primary prevention should be targeted at individuals with high global cardiovascular risk, but research is lacking on how best to identify such individuals in the general population. Family history is a good proxy measure of global risk and may provide an efficient mechanism for identifying high risk individuals. The aim was to test the feasibility of using patients with premature cardiovascular disease to recruit family members as a means of identifying and screening high-risk individuals. <b>Findings:</b> We recruited family members of 50 patients attending a cardiology clinic for premature coronary heart disease (CHD). We compared their cardiovascular risk with a general population control group, and determined their perception of their risk and current level of screening. 103 (36%) family members attended screening (27 siblings, 48 adult offspring and 28 partners). Five (5%) had prevalent CHD. A significantly higher percentage had an ASSIGN risk score >20% compared with the general population (13% versus 2%, p < 0.001). Only 37% of family members were aware they were at increased risk and only 50% had had their blood pressure and serum cholesterol level checked in the previous three years. <b>Conclusions:</b> Patients attending hospital for premature CHD provide a mechanism to contact family members and this can identify individuals with a high global risk who are not currently screened

Developing and implementing an integrated delirium prevention system of care:a theory driven, participatory research study

Background: Delirium is a common complication for older people in hospital. Evidence suggests that delirium incidence in hospital may be reduced by about a third through a multi-component intervention targeted at known modifiable risk factors. We describe the research design and conceptual framework underpinning it that informed the development of a novel delirium prevention system of care for acute hospital wards. Particular focus of the study was on developing an implementation process aimed at embedding practice change within routine care delivery. Methods: We adopted a participatory action research approach involving staff, volunteers, and patient and carer representatives in three northern NHS Trusts in England. We employed Normalization Process Theory to explore knowledge and ward practices on delirium and delirium prevention. We established a Development Team in each Trust comprising senior and frontline staff from selected wards, and others with a potential role or interest in delirium prevention. Data collection included facilitated workshops, relevant documents/records, qualitative one-to-one interviews and focus groups with multiple stakeholders and observation of ward practices. We used grounded theory strategies in analysing and synthesising data. Results: Awareness of delirium was variable among staff with no attention on delirium prevention at any level; delirium prevention was typically neither understood nor perceived as meaningful. The busy, chaotic and challenging ward life rhythm focused primarily on diagnostics, clinical observations and treatment. Ward practices pertinent to delirium prevention were undertaken inconsistently. Staff welcomed the possibility of volunteers being engaged in delirium prevention work, but existing systems for volunteer support were viewed as a barrier. Our evolving conception of an integrated model of delirium prevention presented major implementation challenges flowing from minimal understanding of delirium prevention and securing engagement of volunteers alongside practice change. The resulting Prevention of Delirium (POD) Programme combines a multi-component delirium prevention and implementation process, incorporating systems and mechanisms to introduce and embed delirium prevention into routine ward practices. Conclusions: Although our substantive interest was in delirium prevention, the conceptual and methodological strategies pursued have implications for implementing and sustaining practice and service improvements more broadly

Increasing community capacity to prevent childhood obesity: challenges, lessons learned and results from the Romp & Chomp intervention

BackgroundObesity is a major public health issue; however, only limited evidence is available about effective ways to prevent obesity, particularly in early childhood. Romp &amp; Chomp was a community-wide obesity prevention intervention conducted in Geelong Australia with a target group of 12,000 children aged 0-5 years. The intervention had an environmental and capacity building focus and we have recently demonstrated that the prevalence of overweight/obesity was lower in intervention children, post-intervention. Capacity building is defined as the development of knowledge, skills, commitment, structures, systems and leadership to enable effective health promotion and the aim of this study was to determine if the capacity of the Geelong community, represented by key stakeholder organisations, to support healthy eating and physical activity for young children was increased after Romp &amp; Chomp.MethodsA mixed methods evaluation with three data sources was utilised. 1) Document analysis comprised assessment of the documented formative and intervention activities against a capacity building framework (five domains: Partnerships, Leadership, Resource Allocation, Workforce Development, and Organisational Development); 2) Thematic analysis of key informant interviews (n = 16); and 3) the quantitative Community Capacity Index Survey.ResultsDocument analysis showed that the majority of the capacity building activities addressed the Partnerships, Resource Allocation and Organisational Development domains of capacity building, with a lack of activity in the Leadership and Workforce Development domains. The thematic analysis revealed the establishment of sustainable partnerships, use of specialist advice, and integration of activities into ongoing formal training for early childhood workers. Complex issues also emerged from the key informant interviews regarding the challenges of limited funding, high staff turnover, changing governance structures, lack of high level leadership and unclear communication strategies. The Community Capacity Index provided further evidence that the project implementation network achieved a moderate level of capacity.ConclusionsRomp &amp; Chomp increased the capacity of organisations, settings and services in the Geelong community to support healthy eating and physical activity for young children. Despite this success there are important learnings from this mixed methods evaluation that should inform current and future community-based public health and health promotion initiatives.Trial Registration Number: ANZCTRN12607000374460<br /

Implementing telephone triage in general practice: a process evaluation of a cluster randomised controlled trial

Background: Telephone triage represents one strategy to manage demand for face-to-face GP appointments in primary care. However, limited evidence exists of the challenges GP practices face in implementing telephone triage. We conducted a qualitative process evaluation alongside a UK-based cluster randomised trial (ESTEEM) which compared the impact of GP-led and nurse-led telephone triage with usual care on primary care workload, cost, patient experience, and safety for patients requesting a same-day GP consultation. The aim of the process study was to provide insights into the observed effects of the ESTEEM trial from the perspectives of staff and patients, and to specify the circumstances under which triage is likely to be successfully implemented. Here we report perspectives of staff. Methods: The intervention comprised implementation of either GP-led or nurse-led telephone triage for a period of 2-3 months. A qualitative evaluation was conducted using staff interviews recruited from eight general practices (4 GP triage, 4 Nurse triage) in the UK, implementing triage as part of the ESTEEM trial. Qualitative interviews were undertaken with 44 staff members in GP triage and nurse triage practices (16 GPs, 8 nurses, 7 practice managers, 13 administrative staff). Results: Staff reported diverse experiences and perceptions regarding the implementation of telephone triage, its effects on workload, and on the benefits of triage. Such diversity were explained by the different ways triage was organised, the staffing models used to support triage, how the introduction of triage was communicated across practice staff, and by how staff roles were reconfigured as a result of implementing triage. Conclusion: The findings from the process evaluation offer insight into the range of ways GP practices participating in ESTEEM implemented telephone triage, and the circumstances under which telephone triage can be successfully implemented beyond the context of a clinical trial. Staff experiences and perceptions of telephone triage are shaped by the way practices communicate with staff, prepare for and sustain the changes required to implement triage effectively, as well as by existing practice culture, and staff and patient behaviour arising in response to the changes made. Trial registration: Current Controlled Trials ISRCTN20687662. Registered 28 May 2009

Unexpected Consequences: Women’s experiences of a self-hypnosis intervention to help with pain relief during labour.

Background Self-hypnosis is becoming increasingly popular as a means of labour pain management. Previous studies have produced mixed results. There are very few data on women’s views and experiences of using hypnosis in this context. As part of a randomized controlled trial of self-hypnosis for intra-partum pain relief (the SHIP Trial) we conducted qualitative interviews with women randomized to the intervention arm to explore their views and experiences of using self-hypnosis during labour and birth. Methods Participants were randomly selected from the intervention arm of the study, which consisted of two antenatal self-hypnosis training sessions and a supporting CD that women were encouraged to listen to daily from 32 weeks gestation until the birth of their baby. Those who consented were interviewed in their own homes 8-12 weeks after birth. Following transcription, the interviews were analysed iteratively and emerging concepts were discussed amongst the authors to generate organizing themes. These were then used to develop a principal organizing metaphor or global theme, in a process known as thematic networks analysis. Results Of the 343 women in the intervention group, 48 were invited to interview, and 16 were interviewed over a 12 month period from February 2012 to January 2013. Coding of the data and subsequent analysis revealed a global theme of ‘unexpected consequences’, supported by 5 organising themes, ‘calmness in a climate of fear’, ‘from sceptic to believer’, ‘finding my space’, ‘delays and disappointments’ and ‘personal preferences’. Most respondents reported positive experiences of self-hypnosis and highlighted feelings of calmness, confidence and empowerment. They found the intervention to be beneficial and used a range of novel strategies to personalize their self-hypnosis practice. Occasionally women reported feeling frustrated or disappointed when their relaxed state was misinterpreted by midwives on admission or when their labour and birth experiences did not match their expectations. Conclusion The women in this study generally appreciated antenatal self-hypnosis training and found it to be beneficial during labour and birth. The state of focused relaxation experienced by women using the technique needs to be recognized by providers if the intervention is to be implemented into the maternity service

A qualitative understanding of the effects of reusable sanitary pads and puberty education: Implications for future research and practice

BACKGROUND: The management of menstruation has come to the fore as a barrier to girls’ education attainment in low income contexts. Interventions have been proposed and piloted, but the emerging nature of the field means limited evidence is available to understand their pathways of effect. // METHODS: This study describes and compares schoolgirls’ experiences of menstruation in rural Uganda at the conclusion of a controlled trial of puberty education and sanitary pad provision to elucidate pathways of effect in the interventions. Semi-structured interviews were undertaken with schoolgirls who participated in the Menstruation and the Cycle of Poverty trial concurrent with the final set of quantitative surveys. A framework approach and cross-case analysis were employed to describe and compare the experiences of 27 menstruating girls across the four intervention conditions; education (n = 8), reusable sanitary pads (n = 8), education with reusable sanitary pads (n = 6), and control (n = 5). // RESULTS: Themes included: menstrual hygiene, soiling, irritation and infection, physical experience, knowledge of menstruation, psychological, social and cultural factors, and support from others. Those receiving reusable pads experienced improvements in comfort and reliability. This translated into reduced fears around garment soiling and related school absenteeism. Other menstrual hygiene challenges of washing, drying and privacy remained prominent. Puberty education improved girls’ confidence to discuss menstruation and prompted additional support from teachers and peers. // CONCLUSIONS: Findings have important implications for the development and evaluation of future interventions. Results suggest the provision of menstrual absorbents addresses one core barrier to menstrual health, but that interventions addressing broader needs such as privacy may improve effectiveness. Puberty education sessions should increase attention to body awareness and include strategies to address a wider range of practical menstrual challenges, including pain management. Interviews revealed possibilities for improving quantitative surveys in future research

A typology of practice narratives during the implementation of a preventive, community intervention trial

<p>Abstract</p> <p>Background</p> <p>Traditional methods of process evaluation encompass what components were delivered, but rarely uncover how practitioners position themselves and act relative to an intervention being tested. This could be crucial for expanding our understanding of implementation and its contribution to intervention effectiveness.</p> <p>Methods</p> <p>We undertook a narrative analysis of in-depth, unstructured field diaries kept by nine community development practitioners for two years. The practitioners were responsible for implementing a multi-component, preventive, community-level intervention for mothers of new babies in eight communities, as part of a cluster randomised community intervention trial. We constructed a narrative typology of approaches to practice, drawing on the phenomenology of Alfred Schutz and Max Weber's Ideal Type theory.</p> <p>Results</p> <p>Five types of practice emerged, from a highly 'technology-based' type that was faithful to intervention specifications, through to a 'romantic' type that held relationships to be central to daily operations, with intact relationships being the final arbiter of intervention success. The five types also differed in terms of how others involved in the intervention were characterized, the narrative form (<it>e.g</it>., tragedy, satire) and where and how transformative change in communities was best created. This meant that different types traded-off or managed the priorities of the intervention differently, according to the deeply held values of their type.</p> <p>Conclusions</p> <p>The data set constructed for this analysis is unique. It revealed that practitioners not only exercise their agency within interventions, they do so systematically, that is, according to a pattern. The typology is the first of its kind and, if verified through replication, may have value for anticipating intervention dynamics and explaining implementation variation in community interventions.</p

Process evaluation for complex interventions in health services research: Analysing context, text trajectories and disruptions

Background: Process evaluations assess the implementation and sustainability of complex healthcare interventions within clinical trials, with well-established theoretical models available for evaluating intervention delivery within specific contexts. However, there is a need to translate conceptualisations of context into analytical tools which enable the dynamic relationship between context and intervention implementation to be captured and understood. Methods: In this paper I propose an alternative approach to the design, implementation and analysis of process evaluations for complex health interventions through a consideration of trial protocols as textual documents, distributed and enacted at multiple contextual levels. As an example, I conduct retrospective analysis of a sample of field notes and transcripts collected during the ESTEEM study - a cluster randomised controlled trial of primary care telephone triage. I draw on theoretical perspectives associated with Linguistic Ethnography to examine the delivery of ESTEEM through staff orientation to different texts. In doing so I consider what can be learned from examining the flow and enactment of protocols for notions of implementation and theoretical fidelity (i.e. intervention delivered as intended and whether congruent with the intervention theory). Results: Implementation of the triage intervention required staff to integrate essential elements of the protocol within everyday practice, seen through the adoption and use of different texts that were distributed across staff and within specific events. Staff were observed deploying texts in diverse ways (e.g. reinterpreting scripts, deviating from standard operating procedures, difficulty completing decision support software), providing numerous instances of disruption to maintaining intervention fidelity. Such observations exposed tensions between different contextual features in which the trial was implemented, offering theoretical explanations for the main trial findings. Conclusions: The value of following how trial protocols produce new texts is that we can observe the flow of 'the intervention as intended' across a series of events which are enacted to meet specific demands of intervention delivery. Such observations are not solely premised on identifying routines or practices of implementation, but where 'protocols as intended' breaks down. In doing so, I discuss whether it is here where we might expose the 'active ingredients' of interventions in action