Studying clinical reasoning, part 2: Applying social judgement theory

Part 1 of this paper (Harries and Harries 2001) examined the reasoning studies of the 1980s and 1990s and critiqued the ethnographic and informationprocessing approaches, based on stated information use. The need for an approach that acknowledged the intuitive nature of experienced thinkers’ reasoning was identified. Part 2 describes such an approach ± social judgement theory ± and presents a pilot application in occupational therapy research. The method used is judgement analysis. The issue under study is that of prioritisation policies in community mental health work. The results present the prioritisation policies of four occupational therapists in relation to managing community mental health referrals

Using Social Judgment Theory method to examine how experienced occupational therapy driver assessors use information to make fitness-to-drive recommendations

Introduction - As people with a range of disabilities strive to increase their community mobility, occupational therapy driver assessors are increasingly required to make complex recommendations regarding fitness-to-drive. However, very little is known about how therapists use information to make decisions. The aim of this study was to model how experienced occupational therapy driver assessors weight and combine information when making fitness-to-drive recommendations and establish their level of decision agreement. Method - Using Social Judgment Theory method, this study examined how 45 experienced occupational therapy driver assessors from the UK, Australia and New Zealand made fitness-to-drive recommendations for a series of 64 case scenarios. Participants completed the task on a dedicated website, and data were analysed using discriminant function analysis and an intraclass correlation coefficient. Results - Accounting for 87% of the variance, the cues central to the fitness-to-drive recommendations made by assessors are the client’s physical skills, cognitive and perceptual skills, road law craft skills, vehicle handling skills and the number of driving instructor interventions. Agreement (consensus) between fitness-to-drive recommendations was very high: intraclass correlation coefficient = .97, 95% confidence interval .96–.98). Conclusion - Findings can be used by both experienced and novice driver assessors to reflect on and strengthen the fitness-to-drive recommendations made to clients.This work was supported by the UK Occupational Therapy Research Foundation, Research Priority Grant scheme, 2012

Educating novice practitioners to detect elder financial abuse: A randomised controlled trial

© 2014 Harries et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.This article has been made available through the Brunel Open Access Publishing Fund.Background - Health and social care professionals are well positioned to identify and intervene in cases of elder financial abuse. An evidence-based educational intervention was developed to advance practitioners’ decision-making in this domain. The objective was to test the effectiveness of a decision-training educational intervention on novices’ ability to detect elder financial abuse. The research was funded by an E.S.R.C. grant reference RES-189-25-0334. Methods - A parallel-group, randomised controlled trial was conducted using a judgement analysis approach. Each participant used the World Wide Web to judge case sets at pre-test and post-test. The intervention group was provided with training after pre-test testing, whereas the control group were purely given instructions to continue with the task. 154 pre-registration health and social care practitioners were randomly allocated to intervention (n78) or control (n76). The intervention comprised of written and graphical descriptions of an expert consensus standard explaining how case information should be used to identify elder financial abuse. Participants’ ratings of certainty of abuse occurring (detection) were correlated with the experts’ ratings of the same cases at both stages of testing. Results - At pre-test, no differences were found between control and intervention on rating capacity. Comparison of mean scores for the control and intervention group at pre-test compared to immediate post-test, showed a statistically significant result. The intervention was shown to have had a positive moderate effect; at immediate post-test, the intervention group’s ratings had become more similar to those of the experts, whereas the control’s capacity did not improve. The results of this study indicate that the decision-training intervention had a positive effect on detection ability. Conclusions - This freely available, web-based decision-training aid is an effective evidence-based educational resource. Health and social care professionals can use the resource to enhance their ability to detect elder financial abuse. It has been embedded in a web resource at http://www.elderfinancialabuse.co.uk.ESR

GPs' decisions on drug treatment for patients with high cholesterol values: A think-aloud study

BACKGROUND: The purpose was to examine how General Practitioners (GPs) use clinical information and rules from guidelines in their decisions on drug treatment for high cholesterol values. METHODS: Twenty GPs were presented with six case vignettes and were instructed to think aloud while successively more information about a case was presented, and finally to decide if a drug should be prescribed or not. The statements were coded for the clinical information to which they referred and for favouring or not favouring prescription. RESULTS: The evaluation of clinical information was compatible with decision-making as a search for reasons or arguments. Lifestyle-related information like smoking and overweight seemed to be evaluated from different perspectives. A patient's smoking favoured treatment for some GPs and disfavoured treatment for others. CONCLUSIONS: The method promised to be useful for understanding why doctors differ in their decisions on the same patient descriptions and why rules from the guidelines are not followed strictly

Heart failure diagnosis in primary health care: clinical characteristics of problematic patients. A clinical judgement analysis study

BACKGROUND: Early detection of chronic heart failure has become increasingly important since the introduction of effective treatment. However, clinical diagnosis of heart failure is known to be difficult, especially in mild cases or early in the course of the disease. The purpose of this study is to analyse how patient characteristics contribute to difficulties in diagnosing systolic heart failure. METHODS: Design: A Clinical Judgement Analysis study of 40 case vignettes based on authentic patients, including relevant clinical data except echocardiography. Setting: Primary health care and two cardiology outpatient clinics in Stockholm. Subjects: 70 participants with different types of clinical experience; 27 specialists in general practice, 22 cardiologists, and 21 medical students. Main outcome measures: The assessed probability of heart failure for each case vignette, and the disagreement between the participants. The number of clinical variables (cues) indicative of heart failure in the case vignettes. RESULTS: The ten case vignettes with the least diverging assessments more often had increased relative cardiac volume and atrial fibrillation. No further specific clinical patterns could be found in subgroups of the case vignettes. The ten case vignettes with the most diverging assessments were those with an intermediate number of clinical variables. The case vignettes with the least diverging assessments more often represented patients with cardiac enlargement and atrial fibrillation. CONCLUSION: Diagnosing mild heart failure is difficult, as these patients are not easy to characterise. In our study, a larger number of positive cues resulted in more diagnostic conformity among the participants, and the most important information was cardiac enlargement. The importance of more objective diagnostic methods in diagnosing suspected cases of heart failure should be emphasised

On-going collaborative priority-setting for research activity: a method of capacity building to reduce the research-practice translational gap

Background: International policy suggests that collaborative priority setting (CPS) between researchers and end users of research should shape the research agenda, and can increase capacity to address the research-practice translational gap. There is limited research evidence to guide how this should be done to meet the needs of dynamic healthcare systems. One-off priority setting events and time-lag between decision and action prove problematic. This study illustrates the use of CPS in a UK research collaboration called Collaboration and Leadership in Applied Health Research and Care (CLAHRC). Methods: Data were collected from a north of England CLAHRC through semi-structured interviews with 28 interviewees and a workshop of key stakeholders (n = 21) including academics, NHS clinicians, and managers. Documentary analysis of internal reports and CLAHRC annual reports for the first two and half years was also undertaken. These data were thematically coded. Results: Methods of CPS linked to the developmental phase of the CLAHRC. Early methods included pre-existing historical partnerships with on-going dialogue. Later, new platforms for on-going discussions were formed. Consensus techniques with staged project development were also used. All methods demonstrated actual or potential change in practice and services. Impact was enabled through the flexibility of research and implementation work streams; ‘matched’ funding arrangements to support alignment of priorities in partner organisations; the size of the collaboration offering a resource to meet project needs; and the length of the programme providing stability and long term relationships. Difficulties included tensions between being responsive to priorities and the possibility of ‘drift’ within project work, between academics and practice, and between service providers and commissioners in the health services. Providing protected ‘matched’ time proved difficult for some NHS managers, which put increasing work pressure on them. CPS is more time consuming than traditional approaches to project development. Conclusions: CPS can produce needs-led projects that are bedded in services using a variety of methods. Contributing factors for effective CPS include flexibility in use and type of available resources, flexible work plans, and responsive leadership. The CLAHRC model provides a translational infrastructure that enables CPS that can impact on healthcare systems

Risk assessment and decision making about in-labour transfer from rural maternity care: a social judgment and signal detection analysis

Background: The importance of respecting women's wishes to give birth close to their local community is supported by policy in many developed countries. However, persistent concerns about the quality and safety of maternity care in rural communities have been expressed. Safe childbirth in rural communities depends on good risk assessment and decision making as to whether and when the transfer of a woman in labour to an obstetric led unit is required. This is a difficult decision. Wide variation in transfer rates between rural maternity units have been reported suggesting different decision making criteria may be involved; furthermore, rural midwives and family doctors report feeling isolated in making these decisions and that staff in urban centres do not understand the difficulties they face. In order to develop more evidence based decision making strategies greater understanding of the way in which maternity care providers currently make decisions is required. This study aimed to examine how midwives working in urban and rural settings and obstetricians make intrapartum transfer decisions, and describe sources of variation in decision making. Methods: The study was conducted in three stages. 1. 20 midwives and four obstetricians described factors influencing transfer decisions. 2. Vignettes depicting an intrapartum scenario were developed based on stage one data. 3. Vignettes were presented to 122 midwives and 12 obstetricians who were asked to assess the level of risk in each case and decide whether to transfer or not. Social judgment analysis was used to identify the factors and factor weights used in assessment. Signal detection analysis was used to identify participants' ability to distinguish high and low risk cases and personal decision thresholds. Results: When reviewing the same case information in vignettes midwives in different settings and obstetricians made very similar risk assessments. Despite this, a wide range of transfer decisions were still made, suggesting that the main source of variation in decision making and transfer rates is not in the assessment but the personal decision thresholds of clinicians. Conclusions: Currently health care practice focuses on supporting or improving decision making through skills training and clinical guidelines. However, these methods alone are unlikely to be effective in improving consistency of decision making

Capturing judgement strategies in risk assessments with improved quality of clinical information: How nurses' strategies differ from the ecological model

Background: Nurses' risk assessments of patients at risk of deterioration are sometimes suboptimal. Advances in clinical simulation mean higher quality information can be used as an alternative to traditional paper-based approaches as a means of improving judgement. This paper tests the hypothesis that nurses' judgement strategies and policies change as the quality of information used by nurses in simulation changes. Methods: Sixty-three student nurses and 34 experienced viewed 25 paper-case based and 25 clinically simulated scenarios, derived from real cases, and judged whether the (simulated) patient was at 'risk' of acute deterioration. Criteria of judgement "correctness" came from the same real cases. Information relative weights were calculated to examine judgement policies of individual nurses. Group comparisons of nurses and students under both paper and clinical simulation conditions were undertaken using non parametric statistical tests. Judgment policies were also compared to the ecological statistical model. Cumulative relative weights were calculated to assess how much information nurses used when making judgements. Receiver operating characteristic (ROC) curves were generated to examine predictive accuracy amongst the nurses. Results: There were significant variations between nurses' judgement policies and those optimal policies determined by the ecological model. Nurses significantly underused the cues of consciousness level, respiration rate, and systolic blood pressure than the ecological model requires. However, in clinical simulations, they tended to make appropriate use of heart rate, with non-significant difference in the relative weights of heart rate between clinical simulations and the ecological model. Experienced nurses paid substantially more attention to respiration rate in the simulated setting compared to paper cases, while students maintained a similar attentive level to this cue. This led to a non-significant difference in relative weights of respiration rate between experienced nurses and students. Conclusions: Improving the quality of information by clinical simulations significantly impacted on nurses' judgement policies of risk assessments. Nurses' judgement strategies also varied with the increased years of experience. Such variations in processing clinical information may contribute to nurses' suboptimal judgements in clinical practice. Constructing predictive models of common judgement situations, and increasing nurses' awareness of information weightings in such models may help improve judgements made by nurses

Vignette studies of medical choice and judgement to study caregivers' medical decision behaviour: systematic review

BACKGROUND: Vignette studies of medical choice and judgement have gained popularity in the medical literature. Originally developed in mathematical psychology they can be used to evaluate physicians' behaviour in the setting of diagnostic testing or treatment decisions. We provide an overview of the use, objectives and methodology of these studies in the medical field. METHODS: Systematic review. We searched in electronic databases; reference lists of included studies. We included studies that examined medical decisions of physicians, nurses or medical students using cue weightings from answers to structured vignettes. Two reviewers scrutinized abstracts and examined full text copies of potentially eligible studies. The aim of the included studies, the type of clinical decision, the number of participants, some technical aspects, and the type of statistical analysis were extracted in duplicate and discrepancies were resolved by consensus. RESULTS: 30 reports published between 1983 and 2005 fulfilled the inclusion criteria. 22 studies (73%) reported on treatment decisions and 27 (90%) explored the variation of decisions among experts. Nine studies (30%) described differences in decisions between groups of caregivers and ten studies (33%) described the decision behaviour of only one group. Only six studies (20%) compared decision behaviour against an empirical reference of a correct decision. The median number of considered attributes was 6.5 (IQR 4-9), the median number of vignettes was 27 (IQR 16-40). In 17 studies, decision makers had to rate the relative importance of a given vignette; in six studies they had to assign a probability to each vignette. Only ten studies (33%) applied a statistical procedure to account for correlated data. CONCLUSION: Various studies of medical choice and judgement have been performed to depict weightings of the value of clinical information from answers to structured vignettes of care givers. We found that the design and analysis methods used in current applications vary considerably and could be improved in a large number of cases