Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method

Model validation is recognised as crucial to microsimulation modelling. However, modellers encounter difficulty in choosing the most meaningful methods to compare simulated and actual values. The aim of this paper is to introduce and demonstrate a method employed widely in healthcare calibration studies. The ‘Bland-Altman plot’ consists of a plot of the difference between two methods against the mean (x-y versus x+y/2). A case study is presented to illustrate the method in practice for spatial microsimulation validation. The study features a deterministic combinatorial model (SimObesity), which modelled a synthetic population for England at the ward level using survey (ELSA) and Census 2011 data. Bland-Altman plots were generated, plotting simulated and census ward-level totals for each category of all constraint (benchmark) variables. Other validation metrics, such as R2, SEI, TAE and RMSE, are also presented for comparison. The case study demonstrates how the Bland-Altman plots are interpreted. The simple visualisation of both individual- (ward-) level difference and total variation gives the method an advantage over existing tools used in model validation. There still remains the question of what constitutes a valid or well-fitting model. However, the Bland Altman method can usefully be added to the canon of calibration methods

Validation of spatial microsimulation models: a proposal to adopt the Bland-Altman method

Model validation is recognised as crucial to microsimulation modelling. However, modellers encounter difficulty in choosing the most meaningful methods to compare simulated and actual values. The aim of this paper is to introduce and demonstrate a method employed widely in healthcare calibration studies. The ‘Bland-Altman plot’ consists of a plot of the difference between two methods against the mean (x-y versus x+y/2). A case study is presented to illustrate the method in practice for spatial microsimulation validation. The study features a deterministic combinatorial model (SimObesity), which modelled a synthetic population for England at the ward level using survey (ELSA) and Census 2011 data. Bland-Altman plots were generated, plotting simulated and census ward-level totals for each category of all constraint (benchmark) variables. Other validation metrics, such as R2, SEI, TAE and RMSE, are also presented for comparison. The case study demonstrates how the Bland-Altman plots are interpreted. The simple visualisation of both individual- (ward-) level difference and total variation gives the method an advantage over existing tools used in model validation. There still remains the question of what constitutes a valid or well-fitting model. However, the Bland Altman method can usefully be added to the canon of calibration methods

Athletic population with spondylolysis: review of outcomes following surgical repair or conservative management

Objective The study aims to critically review the outcomes associated with the surgical repair or conservative management of spondylolysis in athletes. Methods The English literature listed in MEDLINE/PubMed was reviewed to identify related articles using the term “spondylolysis AND athlete.” The criteria for studies to be included were management of spondylolysis in athletes, English text, and no year, follow-up, or study design restrictions. The references of the retrieved articles were also evaluated. The primary outcome was time to return to sport. This search yielded 180 citations, and 25 publications were included in the review. Results Treatment methods were dichotomized as operative and nonoperative. In the nonoperative group, 390 athletes were included. A combination of bracing with physical therapy and restriction of activities was used. Conservative measures allowed athletes to return to sport in 3.7 months (weighted mean). One hundred seventy-four patients were treated surgically. The most common technique was Buck's, using a compression screw (91/174). All authors reported satisfactory outcomes. Time to return to play was 7.9 months (weighted mean). There were insufficient studies with suitably homogenous subgroups to conduct a meta-analysis. Conclusion There is no gold standard approach for the management of spondylolysis in the athletic population. The existing literature suggests initial therapy should be a course of conservative management with thoracolumbosacral orthosis brace, physiotherapy, and activity modification. If conservative management fails, surgical intervention should be considered. Two-sided clinical studies are needed to determine an optimal pathway for the management of athletes with spondylolysis

Increasing follow-up questionnaire response rates in a randomized controlled trial of telehealth for depression: three embedded controlled studies

Background: Attrition is problematic in trials, and may be exacerbated in longer studies, telehealth trials and participants with depression – three features of The Healthlines Study. Advance notification, including a photograph and using action-oriented email subject lines might increase response rates, but require further investigation. We examined the effectiveness of these interventions in three embedded Healthlines studies. Methods: Based in different trial sites, participants with depression were alternately allocated to be pre-called or not ahead of the 8-month follow-up questionnaire (Study 1), randomized to receive a research team photograph or not with their 12-month questionnaire (Study 2), and randomized to receive an action-oriented (‘ACTION REQUIRED’) or standard (‘Questionnaire reminder’) 12-month email reminder (Study 3). Participants could complete online or postal questionnaires, and received up to five questionnaire reminders. The primary outcome was completion of the Patient Health Questionnaire (PHQ-9). Secondary outcome measures were the number of reminders and time to questionnaire completion. Results: Of a total of 609 Healthlines depression participants, 190, 251 and 231 participants were included in Studies 1–3 (intervention: 95, 126 and 115), respectively. Outcome completion was ≥90 % across studies, with no differences between trial arms (Study 1: OR 0.38, 95 % CI 0.07–2.10; Study 2: OR 0.84, 95 % CI 0.26–2.66; Study 3: OR 0.53 95 % CI 0.19–1.49). Pre-called participants were less likely to require a reminder (48.4 % vs 62.1 %, OR 0.41, 95 % CI 0.21–0.78), required fewer reminders (adjusted difference in means −0.67, 95 % CI −1.13 to −0.20), and completed follow-up quicker (median 8 vs 15 days, HR 1.35, 95 % CI 1.00–1.82) than control subjects. There were no significant between-group differences in Studies 2 or 3. Conclusions: Eventual response rates in this trial were high, with no further improvement from these interventions. While the photograph and email interventions were ineffective, pre-calling participants reduced time to completion. This strategy might be helpful when the timing of study completion is important. Researchers perceived a substantial benefit from the reduction in reminders with pre-calling, despite no overall decrease in net effort after accounting for pre-notification

Oxygen isotope ratios of phytoliths as a proxy for past climates and environments

Past climate and vegetation records are essential to understand the natural range of environmental variability over timescales beyond the instrumental era. Tracers of past climate change are especially needed for arid regions, which are disproportionately sensitive to global climate warming, yet poorly represented in palaeoclimate archives. The oxygen isotope ratios (18O/16O) of phytoliths (silica microfossils produced by plants) hold the potential to serve as a new proxy for past climates in dry environments, namely for temperature and relative humidity. To use this novel proxy, it is necessary to extract phytoliths of sufficient purity for oxygen isotope analysis and to have a deep understanding of the processes controlling the oxygen isotope signature of the phytoliths in sediments. The aims of this thesis are to improve the methods for extraction and analysis of oxygen isotope ratios of phytoliths, and to quantify the relationship between climate and the oxygen isotope ratios to interpret variations. The oxygen isotope analysis of biogenic silica requires high purity samples. To improve and optimise phytolith extraction from plants and soils for oxygen isotope analysis, several methods were tested and compared to a novel approach using non thermal plasma ashing generated by radio frequency excitation of air under vacuum. Residual contamination was assessed using microscopy and Fourier Transform infrared-spectrometry modelling. Plasma ashing was found to have potential for organic material removal from plants, however the procedure is currently time consuming and requires further improvements to become a routine procedure. For extracting soil phytoliths, an added step of sulphuric acid and hydrogen peroxide (Piranha solution), normally reserved for extracting phytoliths from plants, produced samples with the required purity. Biogenic silica (e.g., diatoms and phytoliths) contains exchangeable oxygen that can affect the accuracy of isotopic analysis. The processes to account for this exchangeable oxygen require specialised equipment that is not always available. A novel approach for silica dehydroxylation was developed, involving sample heating and fluorination in an Elemental Analyser, a common piece of isotope analysis equipment. In situ high temperature dehydroxylation was found to adequately replicate established oxygen isotope ratios for international silicate standards and can be used for silica samples as small as 600 μg. To be able to use phytolith oxygen isotope ratios as a palaeoclimate proxy, the impact of climate needs to be understood. To do this, measured phytolith oxygen isotope ratios from plants and soils from a transect across Australia were analysed. Modern plant phytolith oxygen isotope ratios did not exhibit the expected strong correlations with climate, but oxygen isotopes in soil phytoliths did, correlating strongly with mean warmest period relative humidity at the maximum temperature and the mean annual temperature. The oxygen isotope ratios of phytoliths from plants and soils could be successfully predicted using Craig Gordon leaf water modelling and temperature-dependant silica fractionation, indicating that the key environmental processes are well characterised. The oxygen isotope composition of fossilised plant phytoliths provides the potential to make qualitative inference of climate change. However, the influence of multiple climate variables means that the method would be best employed within a multi-proxy framework.Thesis (MPhil) -- University of Adelaide, School of Physics, Chemistry and Earth Sciences, 202

A validation of the Nottingham Clavicle Score: a clavicle, acromioclavicular joint and sternoclavicular joint–specific patient-reported outcome measure

Background Patients with acromioclavicular joint (ACJ) and sternoclavicular joint (SCJ) injuries and with clavicle fractures are typically younger and more active than those with other shoulder pathologies. We developed the Nottingham Clavicle Score (NCS) specifically for this group of patients to improve sensitivity for assessing the outcomes of treatment of these conditions compared with the more commonly used Constant Score (CS) and Oxford Shoulder Score (OSS). Materials and methods This was a cohort study in which the preoperative and 6-month postoperative NCS evaluations of outcome in 90 patients were compared with the CS, OSS, Imatani Score (IS), and the EQ-5D scores. Reliability was assessed using the Cronbach α. Reproducibility of the NCS was assessed using the test/retest method. Effect sizes were calculated for each score to assess sensitivity to change. Validity was examined by correlations between the NCS and the CS, OSS, IS, and EQ-5D scores obtained preoperatively and postoperatively. Results Significant correlations were demonstrated preoperatively with the OSS (P = .025) and all subcategories of the EQ-5D (P < .05) and postoperatively with the OSS (P < .001), CS (P = .008), IS (P < .001), and all subcategories of EQ-5D (P < .02). The NCS had the largest effect size (1.92) of the compared scores. Internal consistency was excellent (Cronbach α = 0.87). Conclusion The NCS has been proven to be a valid, reliable and sensitive outcome measure that accurately measures the level of function and disability in the ACJ, SCJ and clavicle after traumatic injury and in degenerative disease

Chronic back pain prevalence at small area level in England - the design and validation of a 2-stage static spatial microsimulation model

Spatially disaggregated estimates provide valuable insights into the nature of a disease. They highlight inequalities, aid public health planning and identify avenues for further research. Spatial microsimulation is advantageous in that it can be used to create large microdata sets with intact microlevel relationships between variables, which allows analysis of relationships between variables locally. This methodological paper outlines the design and validation of a 2-stage static spatial microsimulation model for chronic back pain prevalence across England, suitable for policy modelling. Data used was obtained from the Health Survey for England and the 2011 Census. Microsimulation was performed using SimObesity, a previously validated static deterministic program, and the synthetic chronic back pain microdataset was internally validated. The paper also highlights modelling considerations for researchers embarking on similar work, as well as future directions for research in this area of microsimulation

Spatial microsimulation of osteoarthritis prevalence at the small area level in England – Constraint selection for a 2-stage microsimulation process

The presence of identical benchmark/constraint variables in both geographic and survey datasets is a principal requirement for static spatial microsimulation models, particularly in the field of medicine and health sciences. This is also a as key limitation of static spatial models because geographical datasets rarely contain all variables required to realistically simulate an outcome. We believe this challenge can be overcome by a multilevel approach to spatial microsimulation using a case study of estimating the small area level prevalence of knee osteoarthritis in England. In the paper, we describe constraint selection and demonstrate a novel two-stage spatial microsimulation procedure using SimObesity, a static deterministic combinatorial spatial microsimulation model. We also present the validation parameters of our synthetic data, important areas for consideration and avenues for future research. Our findings demonstrate that important benchmark variables absent from the geographical dataset can be incorporated into spatial microsimulation models without compromising model robustness

The Electron Transport Chain Sensitizes Staphylococcus aureus and Enterococcus faecalis to the Oxidative Burst

ABSTRACT Small-colony variants (SCVs) of Staphylococcus aureus typically lack a functional electron transport chain and cannot produce virulence factors such as leukocidins, hemolysins, or the antioxidant staphyloxanthin. Despite this, SCVs are associated with persistent infections of the bloodstream, bones, and prosthetic devices. The survival of SCVs in the host has been ascribed to intracellular residency, biofilm formation, and resistance to antibiotics. However, the ability of SCVs to resist host defenses is largely uncharacterized. To address this, we measured the survival of wild-type and SCV S. aureus in whole human blood, which contains high numbers of neutrophils, the key defense against staphylococcal infection. Despite the loss of leukocidin production and staphyloxanthin biosynthesis, SCVs defective for heme or menaquinone biosynthesis were significantly more resistant to the oxidative burst than wild-type bacteria in human blood or the presence of purified neutrophils. Supplementation of the culture medium of the heme-auxotrophic SCV with heme, but not iron, restored growth, hemolysin and staphyloxanthin production, and sensitivity to the oxidative burst. Since Enterococcus faecalis is a natural heme auxotroph and cause of bloodstream infection, we explored whether restoration of the electron transport chain in this organism also affected survival in blood. Incubation of E. faecalis with heme increased growth and restored catalase activity but resulted in decreased survival in human blood via increased sensitivity to the oxidative burst. Therefore, the lack of functional electron transport chains in SCV S. aureus and wild-type E. faecalis results in reduced growth rate but provides resistance to a key immune defense mechanism. </jats:p

Diploptene δ13C values from contemporary thermokarst lake sediments show complex spatial variation

Cryospheric changes in northern high latitudes are linked to significant greenhouse gas flux to the atmosphere, for example, methane that originates from organic matter decomposition in thermokarst lakes. The set of pathways that link methane production in sediments, via oxidation in the lake system, to the flux of residual methane to the atmosphere is complex and exhibits temporal and spatial variation. The isotopic signal of bacterial biomarkers (hopanoids, e.g. diploptene) in sediments has been used to identify contemporary ocean-floor methane seeps and, in the geological record, periods of enhanced methane production (e.g. the PETM). The biomarker approach could potentially be used to assess temporal changes in lake emissions through the Holocene via the sedimentary biomarker record. However, there are no data on the consistency of the signal of isotopic depletion in relation to source or on the amount of noise (unexplained variation) in biomarker values from modern lake sediments. We assessed methane oxidation as represented by the isotopic signal of biomarkers from methane oxidising bacteria (MOB) in multiple surface sediment samples in three distinct areas known to emit varying levels of methane in two shallow Alaskan thermokarst lakes. Diploptene was present and had δ13C values lower than -38g‰ in all sediments analysed, suggesting methane oxidation was widespread. However, there was considerable variation in δ13C values within each area. The most 13C-depleted diploptene was found in an area of high methane ebullition in Ace Lake (diploptene δ13C values between -68.2 and -50.1‰). In contrast, significantly higher diploptene δ13C values (between -42.9 and -38.8g‰) were found in an area of methane ebullition in Smith Lake. δ13C values of diploptene between -56.8 and -46.9g‰ were found in the centre of Smith Lake, where ebullition rates are low but diffusive methane efflux occurs. The small-scale heterogeneity of the samples may reflect patchy distribution of substrate and/or MOB within the sediments. The two ebullition areas differ in age and type of organic carbon substrate, which may affect methane production, transport, and subsequent oxidation. Given the high amount of variation in surface samples, a more extensive calibration of modern sediment properties, within and among lakes, is required before down-core records of hopanoid isotopic signatures are developed. © Author(s) 2016