19 research outputs found
How Customization Affects Survey Interaction
One common trend in the world of survey data collection is the increasing use of new technological developments which can change the nature of the survey interview. A fairly recent trend is the use of machine-learning techniques to customize questions for respondents. This has the potential to create an individualized experience for the respondent and to improve data quality. Nevertheless, little is known so far of how customization affects the interaction in the survey interview.
We introduce a tool developed by Schierholz et al. (2018) to code respondents’ occupation categories during the survey. The tool uses supervised learning algorithms to predict occupation categories based on previously entered text. We use this example to discuss theoretical and practical implications of customization for the interaction between the interviewer and the respondent.
Preliminary results based on behavior coding of interviews will be presented that show that customization based on machine learning may lead to challenges in the standardized survey interview, particularly for the interviewer.
For future research, we propose an experimental study to investigate differences between conversational and standardized interviewing techniques when working with customized survey instruments. In particular, we will focus on whether interviewers are able to exclude obviously inadequate response options and how this effects interview duration as well as perceived burden on the respondent and the interviewer
New methods for job and occupation classification
This dissertation addresses the measurement of occupation in surveys. Many surveys ask respondents about their occupation with open-ended questions. The verbal answers are typically coded after the interview into official classifications (e.g., the 2008 International Standard Classification of Occupations or the 2010 German Classification of Occupations). This process is known to be time-consuming and prone to errors. To counter both issues, the first paper of the dissertation develops and tests a software prototype, which searches for candidate job titles at the time of the interview. A small set of relevant jobs are suggested based on the respondents’ initial verbal input, allowing respondents to select the most appropriate job on their own. A second paper compares various statistical learning algorithms to optimize the suggestions. A novel algorithm was developed employing Bayesian principles, improving the suggestions further. In a third paper, 1226 work activity descriptions were created based on close inspection of the official occupational classifications. These work activity descriptions can be used as answer options in an improved version of the prototype
Occupation coding during the interview
Currently, most surveys ask for occupation with open-ended questions. The verbatim responses are coded afterwards, which is error-prone and expensive. We describe an alternative approach that allows occupation coding during the interview. Our new technique utilizes a supervised learning algorithm to predict candidate job categories. These suggestions are presented to the respondent, who can in turn choose the most adequate occupation. 72.4% of the respondents selected an occupation when the new instrument was tested in a telephone survey, implicating potential cost savings. To aid further improvements, we identify a number of factors how to increase quality and reduce interview duration.Die Erfassung des Berufs geschieht in Umfragen üblicherweise mithilfe offener Fragen. Anschließend ist eine Kodierung der Freitextantworten notwendig, was teuer und fehleranfällig ist. Wir beschreiben einen alternativen Ansatz, bei dem die Kodierung bereits während des Interviews erfolgt. Die neue Methode verwendet Algorithmen des maschinellen Lernens um mögliche Berufskategorien automatisch vorherzusagen. Die so erzeugten Vorschläge werden dem Befragten vorgelegt, der dann sofort die am besten passende Kategorie auswählen kann. 72.4% der Teilnehmer einer Telefonbefragung haben auf diese Weise ihren Beruf direkt während des Interviews kodiert, was mögliche Kosteneinsparungen impliziert. Um weitere Verbesserungen des neuen Instruments zu ermöglichen, identifizieren wir verschiedene Faktoren, wie auch die Qualität der Kodierung erhöht und die Dauer der Interviews verkürzt werden kann
How Customization Affects Survey Interaction
One common trend in the world of survey data collection is the increasing use of new technological developments which can change the nature of the survey interview. A fairly recent trend is the use of machine-learning techniques to customize questions for respondents. This has the potential to create an individualized experience for the respondent and to improve data quality. Nevertheless, little is known so far of how customization affects the interaction in the survey interview.
We introduce a tool developed by Schierholz et al. (2018) to code respondents’ occupation categories during the survey. The tool uses supervised learning algorithms to predict occupation categories based on previously entered text. We use this example to discuss theoretical and practical implications of customization for the interaction between the interviewer and the respondent.
Preliminary results based on behavior coding of interviews will be presented that show that customization based on machine learning may lead to challenges in the standardized survey interview, particularly for the interviewer.
For future research, we propose an experimental study to investigate differences between conversational and standardized interviewing techniques when working with customized survey instruments. In particular, we will focus on whether interviewers are able to exclude obviously inadequate response options and how this effects interview duration as well as perceived burden on the respondent and the interviewer
Eine Hilfsklassifikation mit Tätigkeitsbeschreibungen für Zwecke der Berufskodierung
Berufsklassifikationen sind anhand der ausgeübten Tätigkeit gegliedert und entsprechend wird auch in Umfragen zur Erfassung des Berufs nach der „beruflichen Tätigkeit“ gefragt. Obwohl sich diese Abfrage auf die Klassifikation bezieht, wird bei der Kodierung von Antworten nur selten auf die tätigkeitsbezogenen Definitionen von Berufsklassifikationen zurückgegriffen. Stattdessen erfolgt die Kodierung meist indirekt, indem Kodierer Berufsbenennungen aus einem Kodier-Index auswählen. Da viele Berufsbenennungen aber unpräzise sind und nur unzureichend die ausgeübte berufliche Tätigkeit beschreiben, kann es dabei zu fehlerhaften Kodierungen kommen. Als alternative Vorgehensweise wird eine tätigkeitsorientierte Hilfsklassifikation zur Verwendung in computergestützten Vorschlagssystemen vorgestellt, die im Internet zum Download verfügbar ist. Dies unterstützt Kodierer, die passendste Tätigkeit ohne den Umweg über Berufsbenennungen auszuwählen. Die neue Hilfsklassifikation basiert auf der deutschen Klassifikation der Berufe 2010 und der Internationalen Standardklassifikation der Berufe 2008 und soll eine simultane Kodierung in beide Klassifikationen ermöglichen. Da zur Nutzung der Hilfsklassifikation Detailkenntnisse über die ausgeübte berufliche Tätigkeit des Befragten nötig sind, ist der größte Nutzen beim Einsatz während des Interviews zu erwarten, wenn Befragte die für sie passendste Tätigkeit selbst auswählen
Machine learning
This chapter introduces you to the value of machine learning in the social sciences, particularly focusing on the overall machine learn- ing process as well as clustering and classification methods. You will get an overview of the machine learning pipeline and methods and how those methods are applied to solve social science problems. The goal is to give an intuitive explanation for the methods and to provide practical tips on how to use them in practice
occupationMeasurement/occupationMeasurement: occupationMeasurement 0.3.1
Gracefully handle unavailability of the KldB 2010 classification
Disable multithreading in examples and tests to comply with new CRAN polic