Analyzing Survey Characteristics, Participation, and Evaluation Across 186 Surveys in an Online Opt-In Panel in Spain

"Survey designers often ask about the best length for their questionnaires and the best format for their questions. Much research has already addressed these issues. However, the answers to these questions may vary with the population of interest, the mode of data collection used, and other factors. The goal of this paper is twofold: 1. To give an overview of the present situation in opt-in online panels, in terms of survey characteristics, participation, and evaluation, by reviewing 186 surveys managed by the panel company Netquest in Spain in 2016. This will be useful to determine areas where further research needs to focus. 2. To study how key characteristics of questionnaires impact survey evaluation and levels of survey break-off. This will allow us to highlight the characteristics that best reduce break-off and improve respondents’ survey evaluation. Based on these results, we will propose practical recommendations for future survey design within the framework of opt-in online panels." (author's abstract

Willingness of Online Panelists to Perform Additional Tasks

People’s willingness to share data with researchers is the fundamental raw material for most social science research. So far, survey researchers have mainly asked respondents to share data in the form of answers to survey questions but there is a growing interest in using alternative sources of data. Less is known about people’s willingness to share these other kinds of data. In this study, we aim to: 1) provide information about the willingness of people to share different types of data; 2) explore the reasons for their acceptance or refusal, and 3) try to determine which variables affect the willingness to perform these additional tasks. We use data from a survey implemented in 2016 in Spain, in which around 1,400 panelists of the Netquest online access panel were asked about their hypothetical willingness to share different types of data: passive measurement on devices they already use; wearing special devices to passively monitor activity; providing them with measurement devices and then having them self-report the results; providing physical specimens or bodily fluids (e.g. saliva); others. Open questions were used to follow up on the reasons for acceptance or refusal in the case of the use of a tracker. Our results suggest that the acceptance level is quite low in general, but there are large differences across tasks and respondents. The main reasons justifying both acceptance and refusal are related to privacy, security and trust. Our regression models also suggest that we can identify factors associated with such willingness

Comparing the Performance of Agree/Disagree and Item-Specific Questions Across PCs and Smartphones

The use of agree/disagree (A/D) questions is a common technique to measure attitudes. For instance, this question format is employed frequently in the Eurobarometer and International Social Survey Programme (ISSP). Theoretical considerations, however, suggest that A/D questions require a complex processing. Therefore, many survey researchers have recommended the use of item-specific (IS) questions, since they seem to be less burdensome. Parallel to this methodological discussion is the discussion around the use of mobile devices for responding to surveys. However, until now, evidence has been lacking as to whether the use of mobile devices for survey response affects the performance of established question formats. In this study, implemented in the Netquest panel in Spain (N = 1,476), we investigated the cognitive effort and response quality associated with A/D and IS questions across PCs and smartphones. For this purpose, we applied a split-ballot design defined by device type and question format. Our analyses revealed longer response times for IS questions than A/D questions, irrespective of the device type and scale length. Also, the IS questions produced better response quality than their A/D counterparts. All in all, the findings indicate a more conscientious response to IS questions compared to A/D questions

Differences in measurement quality depending on recall: results for a question about trust in the parliament

A large body of studies estimate the measurement quality of survey questions using multitrait-multimethod (MTMM) experiments. In these experiments, respondents are asked (nearly) identical questions at least twice. Most commonly, this is done within the same survey by, for instance, changing the response scale characteristics. However, it has frequently been argued that the estimates from these experiments might be biased when respondents recall their first answer and base their second answer on the initial one. So far, only little is known about the impact of memory effects on the estimates of measurement quality. In this study, we conducted a MTMM experiment in the probability-based German Internet Panel (N = 4,401) to investigate whether and to what extent measurement quality estimates differ across respondent groups varying in terms of recall. For this purpose, we use a survey question on trust in the German parliament that was asked with three different response scales. The results reveal that the recall of answers and memory effects vary significantly depending on the scales used for the trust in the parliament question. We also found significant differences in measurement quality across different recall groups. However, more refined research is necessary to better understand the association between answer recall and measurement quality estimates

Completion conditions and response behavior in smartphone surveys: A prediction approach using acceleration data

Includes supplementary materials for the online appendix.This study utilizes acceleration data from smartphone sensors to predict motion conditions of smartphone respondents. Specifically, we predict whether respondents are moving or nonmoving on a survey page level to learn about distractions and the situational conditions under which respondents complete smartphone surveys. The predicted motion conditions allow us to (1) estimate the proportion of smartphone respondents who are moving during survey completion and (2) compare the response behavior of moving and nonmoving respondents. Our analytical strategy consists of two steps. First, we use data from a lab experiment that systematically varied motion conditions of smartphone respondents and train a prediction model that is able to accurately infer respondents’ motion conditions based on acceleration data. Second, we use the trained model to predict motion conditions of respondents in two cross-sectional surveys in order to compare response behavior of respondents with different motion conditions in a field setting. Our results indicate that active movement during survey completion is a relatively rare phenomenon, as only about 3%–4% of respondents were predicted as moving in both cross-sectional surveys. When comparing respondents based on their predicted motion conditions, we observe longer completion times of moving respondents. However, we observe little differences when comparing moving and nonmoving respondents with respect to indicators of superficial responding, indicating that moving during survey completion does not pose a severe threat to data quality

Cardiovascular disease, chronic kidney disease, and diabetes mortality burden of cardiometabolic risk factors from 1980 to 2010: A comparative risk assessment

Background: High blood pressure, blood glucose, serum cholesterol, and BMI are risk factors for cardiovascular diseases and some of these factors also increase the risk of chronic kidney disease and diabetes. We estimated mortality from cardiovascular diseases, chronic kidney disease, and diabetes that was attributable to these four cardiometabolic risk factors for all countries and regions from 1980 to 2010. Methods: We used data for exposure to risk factors by country, age group, and sex from pooled analyses of population-based health surveys. We obtained relative risks for the effects of risk factors on cause-specific mortality from meta-analyses of large prospective studies. We calculated the population attributable fractions for each risk factor alone, and for the combination of all risk factors, accounting for multicausality and for mediation of the effects of BMI by the other three risks. We calculated attributable deaths by multiplying the cause-specific population attributable fractions by the number of disease-specific deaths. We obtained cause-specific mortality from the Global Burden of Diseases, Injuries, and Risk Factors 2010 Study. We propagated the uncertainties of all the inputs to the final estimates. Findings: In 2010, high blood pressure was the leading risk factor for deaths due to cardiovascular diseases, chronic kidney disease, and diabetes in every region, causing more than 40% of worldwide deaths from these diseases; high BMI and glucose were each responsible for about 15% of deaths, and high cholesterol for more than 10%. After accounting for multicausality, 63% (10·8 million deaths, 95% CI 10·1-11·5) of deaths from these diseases in 2010 were attributable to the combined effect of these four metabolic risk factors, compared with 67% (7·1 million deaths, 6·6-7·6) in 1980. The mortality burden of high BMI and glucose nearly doubled from 1980 to 2010. At the country level, age-standardised death rates from these diseases attributable to the combined effects of these four risk factors surpassed 925 deaths per 100 000 for men in Belarus, Kazakhstan, and Mongolia, but were less than 130 deaths per 100 000 for women and less than 200 for men in some high-income countries including Australia, Canada, France, Japan, the Netherlands, Singapore, South Korea, and Spain. Interpretation: The salient features of the cardiometabolic disease and risk factor epidemic at the beginning of the 21st century are high blood pressure and an increasing effect of obesity and diabetes. The mortality burden of cardiometabolic risk factors has shifted from high-income to low-income and middle-income countries. Lowering cardiometabolic risks through dietary, behavioural, and pharmacological interventions should be a part of the global response to non-communicable diseases. Funding: UK Medical Research Council, US National Institutes of Health. © 2014 Elsevier Ltd

Global variation in diabetes diagnosis and prevalence based on fasting glucose and hemoglobin A1c

Fasting plasma glucose (FPG) and hemoglobin A1c (HbA1c) are both used to diagnose diabetes, but these measurements can identify different people as having diabetes. We used data from 117 population-based studies and quantified, in different world regions, the prevalence of diagnosed diabetes, and whether those who were previously undiagnosed and detected as having diabetes in survey screening, had elevated FPG, HbA1c or both. We developed prediction equations for estimating the probability that a person without previously diagnosed diabetes, and at a specific level of FPG, had elevated HbA1c, and vice versa. The age-standardized proportion of diabetes that was previously undiagnosed and detected in survey screening ranged from 30% in the high-income western region to 66% in south Asia. Among those with screen-detected diabetes with either test, the age-standardized proportion who had elevated levels of both FPG and HbA1c was 29-39% across regions; the remainder had discordant elevation of FPG or HbA1c. In most low- and middle-income regions, isolated elevated HbA1c was more common than isolated elevated FPG. In these regions, the use of FPG alone may delay diabetes diagnosis and underestimate diabetes prevalence. Our prediction equations help allocate finite resources for measuring HbA1c to reduce the global shortfall in diabetes diagnosis and surveillance

Repositioning of the global epicentre of non-optimal cholesterol

High blood cholesterol is typically considered a feature of wealthy western countries(1,2). However, dietary and behavioural determinants of blood cholesterol are changing rapidly throughout the world(3) and countries are using lipid-lowering medications at varying rates. These changes can have distinct effects on the levels of high-density lipoprotein (HDL) cholesterol and non-HDL cholesterol, which have different effects on human health(4,5). However, the trends of HDL and non-HDL cholesterol levels over time have not been previously reported in a global analysis. Here we pooled 1,127 population-based studies that measured blood lipids in 102.6 million individuals aged 18 years and older to estimate trends from 1980 to 2018 in mean total, non-HDL and HDL cholesterol levels for 200 countries. Globally, there was little change in total or non-HDL cholesterol from 1980 to 2018. This was a net effect of increases in low- and middle-income countries, especially in east and southeast Asia, and decreases in high-income western countries, especially those in northwestern Europe, and in central and eastern Europe. As a result, countries with the highest level of non-HDL cholesterol-which is a marker of cardiovascular riskchanged from those in western Europe such as Belgium, Finland, Greenland, Iceland, Norway, Sweden, Switzerland and Malta in 1980 to those in Asia and the Pacific, such as Tokelau, Malaysia, The Philippines and Thailand. In 2017, high non-HDL cholesterol was responsible for an estimated 3.9 million (95% credible interval 3.7 million-4.2 million) worldwide deaths, half of which occurred in east, southeast and south Asia. The global repositioning of lipid-related risk, with non-optimal cholesterol shifting from a distinct feature of high-income countries in northwestern Europe, north America and Australasia to one that affects countries in east and southeast Asia and Oceania should motivate the use of population-based policies and personal interventions to improve nutrition and enhance access to treatment throughout the world.Peer reviewe

Height and body-mass index trajectories of school-aged children and adolescents from 1985 to 2019 in 200 countries and territories: a pooled analysis of 2181 population-based studies with 65 million participants

Summary Background Comparable global data on health and nutrition of school-aged children and adolescents are scarce. We aimed to estimate age trajectories and time trends in mean height and mean body-mass index (BMI), which measures weight gain beyond what is expected from height gain, for school-aged children and adolescents. Methods For this pooled analysis, we used a database of cardiometabolic risk factors collated by the Non-Communicable Disease Risk Factor Collaboration. We applied a Bayesian hierarchical model to estimate trends from 1985 to 2019 in mean height and mean BMI in 1-year age groups for ages 5–19 years. The model allowed for non-linear changes over time in mean height and mean BMI and for non-linear changes with age of children and adolescents, including periods of rapid growth during adolescence. Findings We pooled data from 2181 population-based studies, with measurements of height and weight in 65 million participants in 200 countries and territories. In 2019, we estimated a difference of 20 cm or higher in mean height of 19-year-old adolescents between countries with the tallest populations (the Netherlands, Montenegro, Estonia, and Bosnia and Herzegovina for boys; and the Netherlands, Montenegro, Denmark, and Iceland for girls) and those with the shortest populations (Timor-Leste, Laos, Solomon Islands, and Papua New Guinea for boys; and Guatemala, Bangladesh, Nepal, and Timor-Leste for girls). In the same year, the difference between the highest mean BMI (in Pacific island countries, Kuwait, Bahrain, The Bahamas, Chile, the USA, and New Zealand for both boys and girls and in South Africa for girls) and lowest mean BMI (in India, Bangladesh, Timor-Leste, Ethiopia, and Chad for boys and girls; and in Japan and Romania for girls) was approximately 9–10 kg/m2. In some countries, children aged 5 years started with healthier height or BMI than the global median and, in some cases, as healthy as the best performing countries, but they became progressively less healthy compared with their comparators as they grew older by not growing as tall (eg, boys in Austria and Barbados, and girls in Belgium and Puerto Rico) or gaining too much weight for their height (eg, girls and boys in Kuwait, Bahrain, Fiji, Jamaica, and Mexico; and girls in South Africa and New Zealand). In other countries, growing children overtook the height of their comparators (eg, Latvia, Czech Republic, Morocco, and Iran) or curbed their weight gain (eg, Italy, France, and Croatia) in late childhood and adolescence. When changes in both height and BMI were considered, girls in South Korea, Vietnam, Saudi Arabia, Turkey, and some central Asian countries (eg, Armenia and Azerbaijan), and boys in central and western Europe (eg, Portugal, Denmark, Poland, and Montenegro) had the healthiest changes in anthropometric status over the past 3·5 decades because, compared with children and adolescents in other countries, they had a much larger gain in height than they did in BMI. The unhealthiest changes—gaining too little height, too much weight for their height compared with children in other countries, or both—occurred in many countries in sub-Saharan Africa, New Zealand, and the USA for boys and girls; in Malaysia and some Pacific island nations for boys; and in Mexico for girls. Interpretation The height and BMI trajectories over age and time of school-aged children and adolescents are highly variable across countries, which indicates heterogeneous nutritional quality and lifelong health advantages and risks