Evaluating the effectiveness of explanations for recommender systems : Methodological issues and empirical studies on the impact of personalization

Peer reviewedPostprin

Effective patient–clinician interaction to improve treatment outcomes for patients with psychosis: a mixed-methods design

BACKGROUND:At least 100,000 patients with schizophrenia receive care from community mental health teams (CMHTs) in England. These patients have regular meetings with clinicians, who assess them, engage them in treatment and co-ordinate care. As these routine meetings are not commonly guided by research evidence, a new intervention, DIALOG, was previously designed to structure consultations. Using a hand-held computer, clinicians asked patients to rate their satisfaction with eight life domains and three treatment aspects, and to indicate whether or not additional help was needed in each area, with responses being graphically displayed and compared with previous ratings. In a European multicentre trial, the intervention improved patients’ quality of life over a 1-year period. The current programme builds on this research by further developing DIALOG in the UK. RESEARCH QUESTIONS:(1) How can the practical procedure of the intervention be improved, including the software used and the design of the user interface? (2) How can elements of resource-oriented interventions be incorporated into a clinician manual and training programme for a new, more extensive ‘DIALOG+’ intervention? (3) How effective and cost-effective is the new DIALOG+ intervention in improving treatment outcomes for patients with schizophrenia or a related disorder? (4) What are the views of patients and clinicians regarding the new DIALOG+ intervention? METHODS:We produced new software on a tablet computer for CMHTs in the NHS, informed by analysis of videos of DIALOG sessions from the original trial and six focus groups with 18 patients with psychosis. We developed the new ‘DIALOG+’ intervention in consultation with experts, incorporating principles of solution-focused therapy when responding to patients’ ratings and specifying the procedure in a manual and training programme for clinicians. We conducted an exploratory cluster randomised controlled trial with 49 clinicians and 179 patients with psychosis in East London NHS Foundation Trust, comparing DIALOG+ with an active control. Clinicians working as care co-ordinators in CMHTs (along with their patients) were cluster randomised 1 : 1 to either DIALOG+ or treatment as usual plus an active control, to prevent contamination. Intervention and control were to be administered monthly for 6 months, with data collected at baseline and at 3, 6 and 12 months following randomisation. The primary outcome was subjective quality of life as measured on the Manchester Short Assessment of Quality of Life; secondary outcomes were also measured. We also established the cost-effectiveness of the DIALOG intervention using data from the Client Service Receipt Inventory, which records patients’ retrospective reports of using health- and social-care services, including hospital services, outpatient services and medication, in the 3 months prior to each time point. Data were supplemented by the clinical notes in patients’ medical records to improve accuracy. We conducted an exploratory thematic analysis of 16 video-recorded DIALOG+ sessions and measured adherence in these videos using a specially developed adherence scale. We conducted focus groups with patients (n = 19) and clinicians (n = 19) about their experiences of the intervention, and conducted thematic analyses. We disseminated the findings and made the application (app), manual and training freely available, as well as producing a protocol for a definitive trial. RESULTS:Patients receiving the new intervention showed more favourable quality of life in the DIALOG+ group after 3 months (effect size: Cohen’s d = 0.34), after 6 months (Cohen’s d = 0.29) and after 12 months (Cohen’s d = 0.34). An analysis of video-recorded DIALOG+ sessions showed inconsistent implementation, with adherence to the intervention being a little over half of the possible score. Patients and clinicians from the DIALOG+ arm of the trial reported many positive experiences with the intervention, including better self-expression and improved efficiency of meetings. Difficulties reported with the intervention were addressed by further refining the DIALOG+ manual and training. Cost-effectiveness analyses found a 72% likelihood that the intervention both improved outcomes and saved costs. LIMITATIONS:The research was conducted solely in urban east London, meaning that the results may not be broadly generalisable to other settings. CONCLUSIONS:(1) Although services might consider adopting DIALOG+ based on the existing evidence, a definitive trial appears warranted; (2) applying DIALOG+ to patient groups with other mental disorders may be considered, and to groups with physical health problems; (3) a more flexible use with variable intervals might help to make the intervention even more acceptable and effective; (4) more process evaluation is required to identify what mechanisms precisely are involved in the improvements seen in the intervention group in the trial; and (5) what appears to make DIALOG+ effective is that it is not a separate treatment and not a technology that is administered by a specialist; rather, it changes and utilises the existing therapeutic relationship between patients and clinicians in CMHTs to initiate positive change, helping the patients to improve their quality of life. FUTURE RESEARCH:Future studies should include a definitive trial on DIALOG+ and test the effectiveness of the intervention with other populations, such as people with depression. TRIAL REGISTRATION:Current Controlled Trials ISRCTN34757603. FUNDING:The National Institute for Health Research Programme Grants for Applied Research programme

The Effects of Mind Mapping Activities on Students\u27 Motivation

We examined how students‟ motivation differed when they participated in three different types of mind mapping activities: one activity that was completed individually outside of class time, one that was completed individually in class with the instructor available for help, and one that was completed in class with other students and the instructor available for help. Using the MUSIC Model of Academic Motivation (Jones, 2009) as a framework, we implemented a concurrent mixed methods design using identical samples whereby the quantitative component was dominant over the qualitative component. Participants included 40 undergraduate students enrolled in an educational psychology course at a U.S. university. After each of the mind mapping activities, study participants completed questionnaires that included open- and closed-ended items. Although the three activities had similar effects on students‟ motivation-related beliefs, some differences were documented in their preferences of mind mapping activities. Instructional implications are provided

Gene Technology in the eyes of the public and experts. Moral opinions, attitudes and risk perception.

Risk perceptions and attitudes to genetically modified food (GMF) were investigated in a survey study of the public (N=469) and experts (N=49). The response rate was 47 percent for the public. For the experts, response rate was 60 percent. GMF technology was rated as the worst of 18 technologies by members of the public and highly replaceable. Experts had a very different view but also saw GMF as replaceable. Models of risk perceptions and attitudes with regard to policy and consumer intentions were fitted to data. It was found that a very large share of the variance, about 70 percent, was accounted for in the latter cases, while risk perception was somewhat harder to account for (about 50 percent was explained). Traditional explanatory factors such as Dread and Novelty were weak explanatory factors as compared to new approaches, which included Interfering with Nature, Moral value of technology and Epistemic trust. Experts were throughout much more positive to GMF than were members of the public. However, their attitudes and risk perceptions still showed dynamic properties similar to those found in the data from the public. The differences between experts and the public could be well explained in terms of the models tested. In comparisons with recent Eurobarometer studies of attitudes towards GMF, risk emerged in the present study as a more important factor in attitudes, equally important as benefits. The models formulated for the present data were about twice as powerful as those in published analyses of Eurobarometer data.Gene technology; risk perception; policy attitude; consumer behavior; experts; epistemic trust; risk sensitivity

Measurement in marketing

We distinguish three senses of the concept of measurement (measurement as the selection of observable indicators of theoretical concepts, measurement as the collection of data from respondents, and measurement as the formulation of measurement models linking observable indicators to latent factors representing the theoretical concepts), and we review important issues related to measurement in each of these senses. With regard to measurement in the first sense, we distinguish the steps of construct definition and item generation, and we review scale development efforts reported in three major marketing journals since 2000 to illustrate these steps and derive practical guidelines. With regard to measurement in the second sense, we look at the survey process from the respondent's perspective and discuss the goals that may guide participants' behavior during a survey, the cognitive resources that respondents devote to answering survey questions, and the problems that may occur at the various steps of the survey process. Finally, with regard to measurement in the third sense, we cover both reflective and formative measurement models, and we explain how researchers can assess the quality of measurement in both types of measurement models and how they can ascertain the comparability of measurements across different populations of respondents or conditions of measurement. We also provide a detailed empirical example of measurement analysis for reflective measurement models

Employee Empowerment: The Key to Foundation Staff Satisfaction

Although few in numbers, foundation staff are responsible for managing hundreds of billions of dollars in charitable assets. These staff make crucial decisions about how best to allocate those resources to address some of our most pressing domestic and global challenges -- from child welfare to climate change. Given the important goals that foundation staff members are working to achieve, their performance should be a concern not just to those who supervise them, but to all of us. If we accept the argument that staff experiences are connected to performance, then foundation staff perceptions matter greatly. To better understand the experience of foundation staff, the Center for Effective Philanthropy (CEP) surveyed 1,168 staff members at 31 foundations. The surveys were conducted from 2007 to 2011 as part of CEP's Staff Perception Report (SPR) process. Through these surveys, we collect quantitative and qualitative data from respondents. Though our dataset is limited to the 31 foundations that chose to commission an SPR from CEP, it is the largest dataset that exists about foundation staff members' job satisfaction. We have sought, therefore, to analyze it to understand the answers to some basic questions: How satisfied are foundation staff in their jobs? What contributes to their satisfaction? What the data strongly indicate is that leaders set the tone. Their choices -- about a wide range of issues, including communication, delegation of authority, role definition, availability of resources, provision of feedback, recognition of contributions, and opportunities for learning and growth -- shape staff experiences. These dimensions matter far more than the issues that often are the focus of conversations about staff retention and satisfaction, such as pay levels or workload.Also included in this report are case studies of two foundations whose SPR results were particularly strong: The Commonwealth Fund and The Skillman Foundation. These two foundations' staff members rated highly on satisfaction and a host of other dimensions

A Quantitative Analysis of the Relationship Between K–12 Music Educators and Collegiate Music Education Researchers and Instructors: Is There a Disconnect?

Many researchers in a variety of fields have reported on disconnect between researcher and practitioner (Barry, Taylor, & Hair, 2001; Buysse, Sparkman, & Wesley, 2003; Fox, 1992; Fuchs et. al., 1996; Graham et. al., 2006; Hattie & Marsh, 1996; Lang, Wyer, & Haynes, 2007; Rynes, Bartunek, & Daft, 2001). In music education, this topic is frequently discussed (Brand, 1984, 2006; Byo, 1991; Flowers, Gallant, & Single, 1995; Hedden, 1979; Nelson, 2011; Paney, 2004; Radocy, 1983) but evidence is still primarily anecdotal (Nelson, 2011). The purpose of this quantitative study was to measure the relationship between K–12 music educators and collegiate music education researchers to determine to what extent disconnect exists. Research questions focused on access and utilization of scholarly publications, perception of the relationship between the researcher and practitioner, and ratings of philosophical music statements. Participants (N = 868) were solicited through the National Association for Music Education listserv, where a questionnaire was distributed via electronic link. Three types of participants emerged during analysis of descriptive data: Group 1, K–12 music educators (n = 752); Group 2, collegiate music educators (n = 86); and Group 3, music educators teaching both K–12 and collegiate level courses (n = 30). The Research to Practice Gap Analysis Instrument was developed for this study. Responses were analyzed using a variety of tests including Cronbach’s alpha test for reliability, Kruskal–Wallis One–Way Analysis of Variance followed by Mann–Whitney U post hoc with a Bonferonni correction to control for Type I errors, and a multiple regression. Findings showed Group 1 and Group 2 differed significantly on access to music research journals, the way they used and valued research findings, how they perceived their relationships with one another, and their reception of philosophical statements. Almost no instances of significance were found when comparing Group 1 or Group 2 to Group 3. While findings are not generalizable until further testing of the instrument has been conducted, this study contributes empirical data to a narrative within the field of music education that is primarily limited to anecdote