Exploring Language Effects in Crosscultural Survey Research: Does the Language of Administration Affect Answers About Politics?

We study if the language of administration of a survey has an effect on the answers of bilingual respondents to questions measuring political dimensions. This is done in two steps. In the first we test whether the measurement instruments are equivalent for the same individual in two languages. After measurement invariance is established, we test if latent mean differences are significant across the two languages. We also test if the correlation of the same concept in two languages is equal to one or not. Results show evidence for language effects, the latent correlation is below one, although mean differences are not significant. We use data of the LISS migration panel in a within subject design, respondents answer a questionnaire twice first in Dutch and then in their (second) native language among Arabic, English, German, Papiamento and Turkish

Intergenerational perception of the utility of major transport projects

A basic question for the assessment of the fairness of a project financing formula on successive generations is to determine what is the most appropriate method to group the affected individuals over the years. The generational approach in the “Intergenerational Redistributive Effects Model” (IREM) deals with it through the ad-hoc concept of overlapping annual generations. This concept requires establishing the number of years (or timespan) included in the generation and how to incorporate in it those individuals living in years that are not the central one. To do so in a way that reflects the sensitivity of society to the effects of a major transport infrastructure project, specific surveys with a novel approach were designed. The outputs from the survey carried out in Catalonia show evidence suggesting that discount functions should tend to decline at a higher rate in the short term than in the long term. Thus, future impacts of transport investments, both future socio-economic net benefits and financial burdens, should be considered less important than immediate (present) outcomes in the decision-making process. Further, there are reasons to consider that discounting is not neutral to reflect the social interest of large-scale transport projects for the successive generations involved.Peer ReviewedPostprint (author's final draft

Measurement equivalence in multilingual comparative survey research

The present dissertation explores language effects in a comparative survey i.e. to what extent linguistic diversity affects equivalence in a comparative survey. This is done by studying three different dimensions on the challenges of designing a comparative multilingual survey: survey translation, linguistically diverse countries and bilingualism. Guidelines in survey translation do not link assessment criteria and measurement equivalence testing. I propose a systematic procedure to compare versions of a question in different languages before fieldwork which establishes that link. In linguistically diverse countries, survey instruments are translated into more than one language, equivalence is commonly assumed, not tested. I test for invariance distinguishing the response and cognitive processes to a survey question. Finally, I study measurement equivalence within an individual in two languages for political constructs (bilingualism), challenging current methodological approaches by bringing latent variable models. In each dimension, findings aim to contribute to improving comparative survey methodology.Esta tesis explora los efectos del lenguaje en una encuesta comparativa: en qué medida la diversidad lingüística afecta la equivalencia de los datos mediante el estudio de tres dimensiones: la traducción de encuestas, países lingüísticamente diversos y el bilingüismo. Las directrices actuales en la traducción de encuestas no vinculan los criterios de evaluación con un test de equivalencia. Se propone un procedimiento sistemático para comparar las versiones de una pregunta que establece dicho vínculo, en diferentes idiomas antes del trabajo de campo. En países lingüísticamente diversos, el cuestionario se traduce en más de un idioma. Se realiza un test de equivalencia que permite distinguir los procesos de respuesta de los cognitivos. Finalmente, se estudia la equivalencia de conceptos políticos en dos idiomas para un individuo (bilingüismo), proponiendo un enfoque metodológico de modelos de variables latentes. Los hallazgos tienen por objeto contribuir a mejorar la metodología de encuestas en estudios comparativos.Aquesta tesi explora els efectes del llenguatge en una enquesta comparativa: en quina mesura la diversitat lingüística afecta l'equivalència de les dades; mitjançant l'estudi de tres dimensions: la traducció d'enquestes, els països lingüísticament diversos i el bilingüisme. Les directrius actuals en la traducció d'enquestes no vinculen els criteris d'avaluació amb un test d'equivalència de mesures. Per tant, es proposa un procediment sistemàtic que estableix aquest vincle per comparar les versions d'una pregunta en diferents idiomes abans del començament del traball de camp. En països lingüísticament diversos, el qüestionari es tradueix en més d'un idioma. A la tesi, es realitza un test d'equivalència que permet diferenciar els processos de resposta dels cognitius. Finalment, s'estudia l'equivalència de conceptes polítics pel mateix individu en els seus dos idiomes (bilingüisme), mitjançant l’aplicació de models amb variables latents. Els resultats tenen per objectiu contribuir a millorar la metodologia d'enquestes en estudis comparatius

State orthodoxy versus religious freedom?: the case for religious education in Norway and Turkey

Working paper number 18 of the series of the Research and Expertise Centre for Survey Methodology (RECSM)

Evaluation of the concepts trust in institutions and trust in authorities

Working paper number 29 of the series of the Research and Expertise Centre for Survey Methodology (RECSM)

A procedure to prevent differences in translated survey items using SQP

Survey translation has developed best practice procedures to translate survey instruments aiming that the same stimuli and measurement properties should be provided. However, it is very difficult to check in a systematic way these requirements. Current procedures in translation assessment do not link the quality of the translation with a formal test of measurement equivalence. In addition, monitoring the formal structure of translated questionnaires in cross-sectional surveys is challenging. This paper presents a procedure to prevent differences in the form of translated survey instruments using Survey Quality Prediction program (SQP). SQP asks users to code a large set of properties of a survey item. Deviations in translations are detected by comparing the codes of a source questionnaire and targeted languages. The paper summarizes the findings of this procedure implemented in a set of items in the Round 5 of the European Social Survey (ESS)

Measurement equivalence in multilingual comparative survey research

The present dissertation explores language effects in a comparative survey i.e. to what extent linguistic diversity affects equivalence in a comparative survey. This is done by studying three different dimensions on the challenges of designing a comparative multilingual survey: survey translation, linguistically diverse countries and bilingualism. Guidelines in survey translation do not link assessment criteria and measurement equivalence testing. I propose a systematic procedure to compare versions of a question in different languages before fieldwork which establishes that link. In linguistically diverse countries, survey instruments are translated into more than one language, equivalence is commonly assumed, not tested. I test for invariance distinguishing the response and cognitive processes to a survey question. Finally, I study measurement equivalence within an individual in two languages for political constructs (bilingualism), challenging current methodological approaches by bringing latent variable models. In each dimension, findings aim to contribute to improving comparative survey methodology.Esta tesis explora los efectos del lenguaje en una encuesta comparativa: en qué medida la diversidad lingüística afecta la equivalencia de los datos mediante el estudio de tres dimensiones: la traducción de encuestas, países lingüísticamente diversos y el bilingüismo. Las directrices actuales en la traducción de encuestas no vinculan los criterios de evaluación con un test de equivalencia. Se propone un procedimiento sistemático para comparar las versiones de una pregunta que establece dicho vínculo, en diferentes idiomas antes del trabajo de campo. En países lingüísticamente diversos, el cuestionario se traduce en más de un idioma. Se realiza un test de equivalencia que permite distinguir los procesos de respuesta de los cognitivos. Finalmente, se estudia la equivalencia de conceptos políticos en dos idiomas para un individuo (bilingüismo), proponiendo un enfoque metodológico de modelos de variables latentes. Los hallazgos tienen por objeto contribuir a mejorar la metodología de encuestas en estudios comparativos.Aquesta tesi explora els efectes del llenguatge en una enquesta comparativa: en quina mesura la diversitat lingüística afecta l'equivalència de les dades; mitjançant l'estudi de tres dimensions: la traducció d'enquestes, els països lingüísticament diversos i el bilingüisme. Les directrius actuals en la traducció d'enquestes no vinculen els criteris d'avaluació amb un test d'equivalència de mesures. Per tant, es proposa un procediment sistemàtic que estableix aquest vincle per comparar les versions d'una pregunta en diferents idiomes abans del començament del traball de camp. En països lingüísticament diversos, el qüestionari es tradueix en més d'un idioma. A la tesi, es realitza un test d'equivalència que permet diferenciar els processos de resposta dels cognitius. Finalment, s'estudia l'equivalència de conceptes polítics pel mateix individu en els seus dos idiomes (bilingüisme), mitjançant l’aplicació de models amb variables latents. Els resultats tenen per objectiu contribuir a millorar la metodologia d'enquestes en estudis comparatius

Immigrant panel > Method Effects of Language in a Survey > Wave 1

This study is twofold. First, this study aims to investigate to what extent the language in which a questionnaire is drawn up, affects the invariance between cultural groups (within a sample). Secondly, this study investigates whether there are differences in the extent to which groups of different ethnic origin, within the Netherlands, experience feelings of distrust, dissatisfaction and a desire for political change

Immigrant panel > Method Effects of Language in a Survey > Wave 2

This study is twofold. First, this study aims to investigate to what extent the language, in which a questionnaire is drawn up, affects the invariance between cultural groups (within a sample). Secondly, this study investigates whether there are differences in the extent to which groups of different ethnic origin, within the Netherlands, experience feelings of distrust, dissatisfaction and a desire for political change

Using Machine Translation and Post-editing in the TRAPD approach: effects on the quality of translated survey texts

POQ-21-03 Diana Zavala-Rojas, Dorothée Behr, Brita Dorer, Danielly Sorato, Veronika Keck, Using Machine Translation and Post-Editing in the TRAPD Approach: Effects on the Quality of Translated Survey Texts, Public Opinion Quarterly, Volume 88, Issue 1, Spring 2024, Pages 123–148, https://doi.org/10.1093/poq/nfad06