Fitting System Design to Work Practice:Using Observation in Evaluating A Clinical Imaging System

Designing information systems to fit work patterns is becoming a recognized design principle (Greenbaum and Kyng, 1991). Studies of individuals doing their regular work in their usual settings is a growing area of research, for example, for understanding computer system design processes (Curtis, Krasner, and Iscoe, 1988; Davies and Nielsen, 1992; Forsythe, 1992, 1993, 1994; Orlikowski, 1989; Rosson, Maass, and Kellogg, 1988; Shneiderman and Carroll, 1988; Soloway et al., 1988; Suchman, 1987). In medicine, system developers and evaluators have been investigating the interpretive nature and complexity of routine work practice. New design methodologies are based on the assumption that system design must be based on actual work routines rather than on abstract models of information processing or formal models of reasoning (Nyce and Timpka, 1993). System developers are beginning to realize that more attention should be paid to how to incorporate implicit forms of professional competence and knowledge into system design (Nyce and Graves, 1990; Nyce and Timpka, 1993). Such studies could help developers improve system acceptance by tailoring systems to users\u27 perceived needs in ways that will fit into their work practice patterns and routines (Graves and Nyce, 1992), match their values and provide them with benefits (Kaplan, 1987). One area ripe for study is the relationship between technologies of image delivery and how they may change the nature of work. Visualization technology, it is claimed, already has significantly changed work in science, engineering, and medicine (Kaufman, 1994). Changing what information is presented, its layout, and its mode of presentation, changes how professionals think about their work (Ruhleder, 1995). There are few formal studies of visualization systems and their relationship with work practices. Among them are studiesof CAD (Computer Assisted Design) systems, (Henderson, forthcoming), CT scanning (Barley 1986), visualization systems in neurology (Nyce and Graves, 1990), and a clinical imaging system (Kaplan and Lundsgaarde, 1994). Barney et al. (1990) express the needto make explicit the modes of understanding involved in accurately translating and using visual information. They advise identifying and describing the forms of reasoning and practice that are important to technological modeling of visualization so that information systems developers can construct environments that support routine practice effectively. For example, Ramey, Rowberg, and Robinson (1992) studied the task domain in diagnostic radiology for the purpose of designing radiology workstations. This paper concerns an evaluation study of a physician\u27s work and how that study led to design suggestions for a clinical imaging system. It reports on how formative evaluation can help fit system design to work practice

Preliminary Results from a Social Interactionist Evaluation of a Medical Expert System

Different models of change reflect different assumptions about information systems development, implementation, and research. These models inform the work of system development and implementation (Kaplan 1991). In one of these groups of models, Research, Development, and Diffusion (RDD) models, experts develop a system they consider beneficial, and users either adopt it or are considered resistant to it. In Problem Solving (PS) models, which derive from Lewin’ theories of group dynamics and change, experts and clients work together to develop information systems solutions to what they perceive as problems. In Social Interaction (SI) models, which are based on Rogers’s Classic Diffusion Theory, the emphasis is on how an innovation is communicated through social channels over time. New design methodologies and new evaluation approaches replace RDD models with PS or SI models. These models address social, cultural, political, or work life issues in system design and in system evaluation. methodologies and approaches

Fiber-optic endoscopic evaluation of swallowing to assess swallowing outcomes as a function of head position in a normal population

BACKGROUND: Head position practice has been shown to influence pill-swallowing ability, but the impact of head position on measures of swallowing outcomes has not yet been studied with fiber-optic endoscopic evaluation of swallowing (FEES). The primary purpose of this study was to determine whether head position impacts penetration-aspiration scale scores and/or post-swallow pharyngeal residue as assessed by FEES. Documenting the incidence of pharyngeal residue and laryngeal penetration and aspiration in a normal population was a secondary goal. METHODS: Adults without swallowing difficulties (N = 84) were taught a pill swallowing technique based on learning five head positions and were asked to practice with small, hard candies (e.g., TicTacs) for two weeks. Then they demonstrated swallowing in each of the head positions for two conditions, liquid and purée, while undergoing FEES. RESULTS: Out of 840 examined swallows, one event of aspiration and 5 events of penetration occurred. During practice >50% participants found positions they preferred over the center position for swallowing but head position was not associated with penetration-aspiration scores assessed by FEES. Significant associations and non-significant trends were found between pharyngeal residue and three variables: age, most preferred head position, and least preferred head position. CONCLUSION: Head position during swallowing (head up) and age greater than 40 years may result in increased pharyngeal residue but not laryngeal penetration or aspiration

Developing a decision support system for tobacco use counselling using primary care physicians

Background Clinical decision support systems (CDSS) have the potential to improve adherence to guidelines, but only if they are designed to work in the complex environment of ambulatory clinics as otherwise physicians may not use them. Objective To gain input from primary care physicians in designing a CDSS for smoking cessation to ensure that the design is appropriate to a clinical environment before attempts to test this CDSS in a clinical trial. This approach is of general interest to those designing similar systems. Design and approach We employed an iterative ethnographic process that used multiple evaluation methods to understand physician preferences and workflow integration. Using results from our prior survey of physicians and clinic managers, we developed a prototype CDSS, validated content and design with an expert panel, and then subjected it to usability testing by physicians, followed by iterative design changes based on their feedback. We then performed clinical testing with individual patients, and conducted field tests of the CDSS in two primary care clinics during which four physicians used it for routine patient visits. Results The CDSS prototype was substantially modified through these cycles of usability and clinical testing, including removing a potentially fatal design flaw. During field tests in primary care clinics, physicians incorporated the final CDSS prototype into their workflow, and used it to assist in smoking cessation interventions up to eight times daily. Conclusions A multi-method evaluation process utilising primary care physicians proved useful for developing a CDSS that was acceptable to physicians and patients, and feasible to use in their clinical environment

Database analysis of children and adolescents with Bipolar Disorder consuming a micronutrient formula

<p>Abstract</p> <p>Background</p> <p>Eleven previous reports have shown potential benefit of a 36-ingredient micronutrient formula (known as EMPowerplus) for the treatment of psychiatric symptoms. The current study asked whether children (7-18 years) with pediatric bipolar disorder (PBD) benefited from this same micronutrient formula; the impact of Attention-Deficit/Hyperactivity Disorder (ADHD) on their response was also evaluated.</p> <p>Methods</p> <p>Data were available from an existing database for 120 children whose parents reported a diagnosis of PBD; 79% were taking psychiatric medications that are used to treat mood disorders; 24% were also reported as ADHD. Using Last Observation Carried Forward (LOCF), data were analyzed from 3 to 6 months of micronutrient use.</p> <p>Results</p> <p>At LOCF, mean symptom severity of bipolar symptoms was 46% lower than baseline (effect size (ES) = 0.78) (<it>p </it>< 0.001). In terms of responder status, 46% experienced >50% improvement at LOCF, with 38% still taking psychiatric medication (52% drop from baseline) but at much lower levels (74% reduction in number of medications being used from baseline). The results were similar for those with both ADHD and PBD: a 43% decline in PBD symptoms (ES = 0.72) and 40% in ADHD symptoms (ES = 0.62). An alternative sample of children with just ADHD symptoms (n = 41) showed a 47% reduction in symptoms from baseline to LOCF (ES = 1.04). The duration of reductions in symptom severity suggests that benefits were not attributable to placebo/expectancy effects. Similar findings were found for younger and older children and for both sexes.</p> <p>Conclusions</p> <p>The data are limited by the open label nature of the study, the lack of a control group, and the inherent self-selection bias. While these data cannot establish efficacy, the results are consistent with a growing body of research suggesting that micronutrients appear to have therapeutic benefit for children with PBD with or without ADHD in the absence of significant side effects and may allow for a reduction in psychiatric medications while improving symptoms. The consistent reporting of positive changes across multiple sites and countries are substantial enough to warrant a call for randomized clinical trials using micronutrients.</p

Evaluating treatments in health care: The instability of a one-legged stool

<p>Abstract</p> <p>Background</p> <p>Both scientists and the public routinely refer to randomized controlled trials (RCTs) as being the 'gold standard' of scientific evidence. Although there is no question that placebo-controlled RCTs play a significant role in the evaluation of new pharmaceutical treatments, especially when it is important to rule out placebo effects, they have many inherent limitations which constrain their ability to inform medical decision making. The purpose of this paper is to raise questions about <it>over-reliance </it>on RCTs and to point out an additional perspective for evaluating healthcare evidence, as embodied in the Hill criteria. The arguments presented here are generally relevant to all areas of health care, though mental health applications provide the primary context for this essay.</p> <p>Discussion</p> <p>This article first traces the history of RCTs, and then evaluates five of their major limitations: they often lack external validity, they have the potential for increasing health risk in the general population, they are no less likely to overestimate treatment effects than many other methods, they make a relatively weak contribution to clinical practice, and they are excessively expensive (leading to several additional vulnerabilities in the quality of evidence produced). Next, the nine Hill criteria are presented and discussed as a richer approach to the evaluation of health care treatments. Reliance on these multi-faceted criteria requires more analytical thinking than simply examining RCT data, but will also enhance confidence in the evaluation of novel treatments.</p> <p>Summary</p> <p>Excessive reliance on RCTs tends to stifle funding of other types of research, and publication of other forms of evidence. We call upon our research and clinical colleagues to consider additional methods of evaluating data, such as the Hill criteria. Over-reliance on RCTs is similar to resting all of health care evidence on a one-legged stool.</p