Developing a Quality Control Protocol for Evaluation of Recorded Interviews

This presentation will describe the process used at the University of Michigan Survey Research Center for evaluating interviewer performance in survey administration. Within the Survey Research Operations unit, we use an online system for evaluating the interviewer-respondent interaction using recorded interviews. We will present our framework for measuring how well interviewers adhere to General Interviewing Techniques (GIT) - the guidelines in which they were trained. The presentation will describe the question-level and session-level measurement criteria employed, in addition to the selection protocols and the integration of paradata into the selection process. The presentation will include analysis of some evaluation data, with a discussion of how the data were used to inform further development of the evaluation protocol. Although some aggregate data will be shared, the presentation will largely focus on the operational considerations related to the development and implementation of the quality control protocol across projects

Developing a Quality Control Protocol for Evaluation of Recorded Interviews

This presentation will describe the process used at the University of Michigan Survey Research Center for evaluating interviewer performance in survey administration. Within the Survey Research Operations unit, we use an online system for evaluating the interviewer-respondent interaction using recorded interviews. We will present our framework for measuring how well interviewers adhere to General Interviewing Techniques (GIT) - the guidelines in which they were trained. The presentation will describe the question-level and session-level measurement criteria employed, in addition to the selection protocols and the integration of paradata into the selection process. The presentation will include analysis of some evaluation data, with a discussion of how the data were used to inform further development of the evaluation protocol. Although some aggregate data will be shared, the presentation will largely focus on the operational considerations related to the development and implementation of the quality control protocol across projects

General Interviewing Techniques: Developing Evidence-Based Practices

This poster is a hands-on demonstration of the in-progress General Interviewer Techniques (GIT) materials described by Schaeffer, Dykema, Coombs, Schultz, Holland, and Hudson. Participants will be able to view and listen to the lesson materials, delivered via an online interface, and talk to the GIT developers

General Interviewer Techniques: Developing Evidence-Based Practices for Standardized Interviewing

The practices of standardized interviewing developed at many research sites over many years. The version of standardization that Fowler and Mangione codified in Standardized Survey Interviewing has provided researchers a core resource to use in training and supervising standardized interviewers. In recent decades, however, the accumulation of recordings and transcripts of interviews makes it possible to re-visit the practices of standardization to describe both how respondents actually answer survey questions and how interviewers actually respond. To update General Interviewer Training (GIT), we brought observations of interaction during interviews together with research about conversational practices from conversation analysis, psychology, and other sources. Using our analysis of the question-answer sequence, we identified the principal actions covered in training as reading a survey question, recognizing a codable answer, acknowledging a codable answer, and follow-up for an uncodable answer. Our analysis of each of these actions is influenced by our observations of the participants’ behavior – interviewers must be trained how to repair the reading of the question, for example -- and by how that behavior is influenced by characteristics of survey questions – follow-up differs for yes-no and selection questions. We developed a set of criteria to use in evaluating the likely impact of the choices we recommend on, for example, interviewer variance and the motivation of the respondent. Although research is not available for all (or even most) criteria, we attempted to be systematic in assessing the likely costs and benefits of our decisions. We focus on standardized interviewing, which attempts to train interviewers in behaviors that all interviewers can perform in the same way. However, the evidence supplied by studies of interviewer-respondent interaction makes clear that the impact of the question on the respondent’s answer, and the way that respondents answer questions must be taken into account in any style of interviewing

Chapter 3: General Interviewing Techniques: Developing Evidence-Based Practices for Standardized Interviewing Appendix 3

Table A3A.1 Summary of Basic Techniques of Standardized Interviewing (adapted from Fowler and Mangione 1990, pp. 35-53) Table A3A.2 Basic Question Forms (Response Formats

Magnetoresistance of high mobility HgTe quantum dot films with controlled charging

Funding: We thank Christopher Melnychuk for very useful discussions. This work was supported by the National Natural Science Foundation of China under grant number 62105022, the University of Chicago Materials Research Science and Engineering Center, which was funded by the National Science Foundation under award number DMR1420709, and by the Department of Defense (DOD) Air Force Office of Scientific Research under grant number FA9550-18-1-0099.The magnetoresistance of HgTe quantum dot films, exhibiting a well-defined 1Se state charging and a relatively high mobility (1-10 cm2 V−1 s−1), is measured as a function of temperature down to 10 K and controlled occupation of the first electronic state. There is a positive-quadratic magnetoresistance which can be several 100% at low temperature and scales like x(1 − x) where x is the filling fraction of the lowest quantum dot state in the conduction band, 1Se. This positive magnetoresistance is orders of magnitude larger than the effect estimated from mobile carriers and it is attributed to the increased confinement induced by the magnetic field. There is also a negative magnetoresistance of 1-20% from 300 K to 10 K which is rather independent of the fractional occupation, and which follows a negative exponential dependence with the magnetic field. It can be empirically fit with an effective g-factor of ∼55 and it is tentatively attributed to the reduction of barrier heights by the Zeeman splitting of the 1Se state.Peer reviewe

Conceptual acrobatics: talking about assessment standards in the transparency era

Since their introduction in the 1990s, explicit standards documents have pervaded higher education assessment – success likely linked to their compatibility with constructive alignment and quality assurance regimes. Researchers, however, criticise that such documents are based on a misconception of standards as explicit and absolute, when in fact standards have tacit and contextual qualities that make it impossible to codify them fully. This article considers how practitioners conceive of standards. It identifies the range of concepts of standards, and looks at which were dominant or marginal in 24 external examiners’ responses to interview questions about their examining practice. The article identifies a significant gap between the theoretical positions asserted in the research literature and the conceptions held by experienced academics tasked with guaranteeing national standards. It considers implications for quality assurance and reflects on whether the dominance of transparency and accountability discourses leads academics to contort the way they talk about standards