Task-oriented evaluation of electronic medical records systems: development and validation of a questionnaire for physicians

BACKGROUND: Evaluation is a challenging but necessary part of the development cycle of clinical information systems like the electronic medical records (EMR) system. It is believed that such evaluations should include multiple perspectives, be comparative and employ both qualitative and quantitative methods. Self-administered questionnaires are frequently used as a quantitative evaluation method in medical informatics, but very few validated questionnaires address clinical use of EMR systems. METHODS: We have developed a task-oriented questionnaire for evaluating EMR systems from the clinician's perspective. The key feature of the questionnaire is a list of 24 general clinical tasks. It is applicable to physicians of most specialties and covers essential parts of their information-oriented work. The task list appears in two separate sections, about EMR use and task performance using the EMR, respectively. By combining these sections, the evaluator may estimate the potential impact of the EMR system on health care delivery. The results may also be compared across time, site or vendor. This paper describes the development, performance and validation of the questionnaire. Its performance is shown in two demonstration studies (n = 219 and 80). Its content is validated in an interview study (n = 10), and its reliability is investigated in a test-retest study (n = 37) and a scaling study (n = 31). RESULTS: In the interviews, the physicians found the general clinical tasks in the questionnaire relevant and comprehensible. The tasks were interpreted concordant to their definitions. However, the physicians found questions about tasks not explicitly or only partially supported by the EMR systems difficult to answer. The two demonstration studies provided unambiguous results and low percentages of missing responses. In addition, criterion validity was demonstrated for a majority of task-oriented questions. Their test-retest reliability was generally high, and the non-standard scale was found symmetric and ordinal. CONCLUSION: This questionnaire is relevant for clinical work and EMR systems, provides reliable and interpretable results, and may be used as part of any evaluation effort involving the clinician's perspective of an EMR system

"Did the trial kill the intervention?" experiences from the development, implementation and evaluation of a complex intervention

Background: The development, implementation and evaluation of any new health intervention is complex. This paper uses experiences from the design, implementation and evaluation of a rehabilitation programme to shed light on, and prompt discussion around, some of the complexities involved in such an undertaking. Methods: Semi-structured interviews were conducted with 15 trial participants and five members of staff at the conclusion of a trial evaluating a rehabilitation programme aimed at promoting recovery after stem cell transplantation. Results: This study identified a number of challenges relating to the development and evaluation of complex interventions. The difficulty of providing a standardised intervention that was acceptable to patients was highlighted in the participant interviews. Trial participants and some members of staff found the concept of equipoise and randomisation challenging and there was discord between the psychosocial nature of the intervention and the predominant bio-medical culture in which the research took place. Conclusions: A lack of scientific evidence as to the efficacy of an intervention does not preclude staff and patients holding strong views about the benefits of an intervention. The evaluation of complex interventions should, where possible, facilitate not restrict that complexity. Within the local environment where the trial is conducted, acquiescence from those in positions of authority is insufficient; commitment to the trial is required

Assessing the need for follow-up: The relationship of prognosis to posthospitalization adjustment

Parents of children (N-64) hospitalized at a psychiatric hospital between the years 1971 and 1976 rated their child's adjustment in a number of behavioral areas. Problems in school, making friends, acting age-appropriately, and handling aggression were reported to be the major sources of difficulty for 40% of the former patients. No statistical relationship was found between clinicians' prognosis at discharge and children's post-hospitalization adjustment. The authors discuss some of the variables that mitigate against accurate outcome pre diction and argue for the integration of an organized follow-up program into their treatment program.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/43949/1/10578_2005_Article_BF01433501.pd

Relationship between smoking and obesity:a cross-sectional study of 499,504 middle-aged adults in the UK general population

Background: There is a general perception that smoking protects against weight gain and this may influence commencement and continuation of smoking, especially among young women.<p></p> Methods: A cross-sectional study was conducted using baseline data from UK Biobank. Logistic regression analyses were used to explore the association between smoking and obesity; defined as body mass index (BMI) >30kg/m2. Smoking was examined in terms of smoking status, amount smoked, duration of smoking and time since quitting and we adjusted for the potential confounding effects of age, sex, socioeconomic deprivation, physical activity, alcohol consumption, hypertension and diabetes.<p></p> Results: The study comprised 499,504 adults aged 31 to 69 years. Overall, current smokers were less likely to be obese than never smokers (adjusted OR 0.83 95% CI 0.81-0.86). However, there was no significant association in the youngest sub-group (≤40 years). Former smokers were more likely to be obese than both current smokers (adjusted OR 1.33 95% CI 1.30-1.37) and never smokers (adjusted OR 1.14 95% CI 1.12-1.15). Among smokers, the risk of obesity increased with the amount smoked and former heavy smokers were more likely to be obese than former light smokers (adjusted OR 1.60, 95% 1.56-1.64, p<0.001). Risk of obesity fell with time from quitting. After 30 years, former smokers still had higher risk of obesity than current smokers but the same risk as never smokers.<p></p> Conclusion: Beliefs that smoking protects against obesity may be over-simplistic; especially among younger and heavier smokers. Quitting smoking may be associated with temporary weight gain. Therefore, smoking cessation interventions should include weight management support.<p></p&gt

A search for the decay modes B+/- to h+/- tau l

We present a search for the lepton flavor violating decay modes B+/- to h+/- tau l (h= K,pi; l= e,mu) using the BaBar data sample, which corresponds to 472 million BBbar pairs. The search uses events where one B meson is fully reconstructed in one of several hadronic final states. Using the momenta of the reconstructed B, h, and l candidates, we are able to fully determine the tau four-momentum. The resulting tau candidate mass is our main discriminant against combinatorial background. We see no evidence for B+/- to h+/- tau l decays and set a 90% confidence level upper limit on each branching fraction at the level of a few times 10^-5.Comment: 15 pages, 7 figures, submitted to Phys. Rev.

Familial aggregation of lung cancer in a high incidence area in China

To investigate whether lung cancer clusters in families in a high incidence county of China, an analysis was conducted using data on domestic fuel history and tobacco use for family members of 740 deceased lung cancer probands and 740 controls (probands' spouses). Lung cancer prevalence was compared among first-degree relatives of probands and of controls, taking into account various factors using logistic regression and generalised estimating equations. First-degree relatives of probands, compared with those of controls, showed an excess risk of lung cancer (odds ratio (OR)=2.05, 95% confidence interval (CI): 1.68–2.53). Overall, female relatives of probands had a greater risk than did their male counterparts, and the risk was 2.90-fold for parents of probands as compared with parents of spouses. Female relatives of probands had 2.67-fold greater risk than female controls. Lung cancer risk was particularly marked among mothers (OR=3.78, 95% CI: 2.03–7.12). Having two or more affected relatives was associated with a 2.69–5.40-fold risk increase. The risk elevation was also found for other cancers overall. Results confirm previous findings of a genetic predisposition to lung cancer, and also imply that lung cancer may share a genetic background with other cancers

Observation and study of baryonic B decays: B -> D(*) p pbar, D(*) p pbar pi, and D(*) p pbar pi pi

We present a study of ten B-meson decays to a D(*), a proton-antiproton pair, and a system of up to two pions using BaBar's data set of 455x10^6 BBbar pairs. Four of the modes (B0bar -> D0 p anti-p, B0bar -> D*0 p anti-p, B0bar -> D+ p anti-p pi-, B0bar -> D*+ p anti-p pi-) are studied with improved statistics compared to previous measurements; six of the modes (B- -> D0 p anti-p pi-, B- -> D*0 p anti-p pi-, B0bar -> D0 p anti-p pi- pi+, B0bar -> D*0 p anti-p pi- pi+, B- -> D+ p anti-p pi- pi-, B- -> D*+ p anti-p pi- pi-) are first observations. The branching fractions for 3- and 5-body decays are suppressed compared to 4-body decays. Kinematic distributions for 3-body decays show non-overlapping threshold enhancements in m(p anti-p) and m(D(*)0 p) in the Dalitz plots. For 4-body decays, m(p pi-) mass projections show a narrow peak with mass and full width of (1497.4 +- 3.0 +- 0.9) MeV/c2, and (47 +- 12 +- 4) MeV/c2, respectively, where the first (second) errors are statistical (systematic). For 5-body decays, mass projections are similar to phase space expectations. All results are preliminary.Comment: 28 pages, 90 postscript figures, submitted to LP0

Investigation of relative risk estimates from studies of the same population with contrasting response rates and designs

Background: There is little empirical evidence regarding the generalisability of relative risk estimates from studies which have relatively low response rates or are of limited representativeness. The aim of this study was to investigate variation in exposure-outcome relationships in studies of the same population with different response rates and designs by comparing estimates from the 45 and Up Study, a population-based cohort study (self-administered postal questionnaire, response rate 18%), and the New South Wales Population Health Survey (PHS) (computer-assisted telephone interview, response rate ~60%). Methods: Logistic regression analysis of questionnaire data from 45 and Up Study participants (n = 101,812) and 2006/ 2007 PHS participants (n = 14,796) was used to calculate prevalence estimates and odds ratios (ORs) for comparable variables, adjusting for age, sex and remoteness. ORs were compared using Wald tests modelling each study separately, with and without sampling weights. Results: Prevalence of some outcomes (smoking, private health insurance, diabetes, hypertension, asthma) varied between the two studies. For highly comparable questionnaire items, exposure-outcome relationship patterns were almost identical between the studies and ORs for eight of the ten relationships examined did not differ significantly. For questionnaire items that were only moderately comparable, the nature of the observed relationships did not differ materially between the two studies, although many ORs differed significantly. Conclusions: These findings show that for a broad range of risk factors, two studies of the same population with varying response rate, sampling frame and mode of questionnaire administration yielded consistent estimates of exposure-outcome relationships. However, ORs varied between the studies where they did not use identical questionnaire items