Efficiency of propensity score adjustment and calibration on the estimation from non-probabilistic online surveys

Peer Reviewe

Advances in estimation by the item sum technique using auxiliary information in complex surveys

To collect sensitive data, survey statisticians have designed many strategies to reduce nonresponse rates and social desirability response bias. In recent years, the item count technique (ICT) has gained considerable popularity and credibility as an alternative mode of indirect questioning survey, and several variants of this technique have been proposed as new needs and challenges arise. The item sum technique (IST), which was introduced by Chaudhuri and Christofides (2013) and Trappmann et al. (2014), is one such variant, used to estimate the mean of a sensitive quantitative variable. In this approach, sampled units are asked to respond to a two-list of items containing a sensitive question related to the study variable and various innocuous, nonsensitive, questions. To the best of our knowledge, very few theoretical and applied papers have addressed the IST. In this article, therefore, we present certain methodological advances as a contribution to appraising the use of the IST in real-world surveys. In particular, we employ a generic sampling design to examine the problem of how to improve the estimates of the sensitive mean when auxiliary information on the population under study is available and is used at the design and estimation stages. A Horvitz-Thompson type estimator and a calibration type estimator are proposed and their efficiency is evaluated by means of an extensive simulation study. Using simulation experiments, we show that estimates obtained by the IST are nearly equivalent to those obtained using “true data” and that in general they outperform the estimates provided by a competitive randomized response method. Moreover, the variance estimation may be considered satisfactory. These results open up new perspectives for academics, researchers and survey practitioners, and could justify the use of the IST as a valid alternative to traditional direct questioning survey modes.Ministerio de Economía y Competitividad of SpainMinisterio de Educacion, Cultura y Deporteproject PRIN-SURWE

Empirical likelihood confidence intervals for complex sampling designs

We define an empirical likelihood approach which gives consistent design-based confidence intervals which can be calculated without the need of variance estimates, design effects, resampling, joint inclusion probabilities and linearization, even when the point estimator is not linear. It can be used to construct confidence intervals for a large class of sampling designs and estimators which are solutions of estimating equations. It can be used for means, regressions coefficients, quantiles, totals or counts even when the population size is unknown. It can be used with large sampling fractions and naturally includes calibration constraints. It can be viewed as an extension of the empirical likelihood approach to complex survey data. This approach is computationally simpler than the pseudoempirical likelihood and the bootstrap approaches. The simulation study shows that the confidence interval proposed may give better coverages than the confidence intervals based on linearization, bootstrap and pseudoempirical likelihood. Our simulation study shows that, under complex sampling designs, standard confidence intervals based on normality may have poor coverages, because point estimators may not follow a normal sampling distribution and their variance estimators may be biased.<br/

Application of randomized response techniques for investigating cannabis use by Spanish university students

Cannabis is the most widely used illicit drug in developed countries, and has a significant impact on mental and physical health in the general population. Although the evaluation of levels of substance use is difficult, a method such as the randomized response technique (RRT), which includes both a personal component and an assurance of confidentiality, provides a combination which can achieve a considerable degree of accuracy. Various RRT surveys have been conducted to measure the prevalence of drug use, but to date no studies have been made of the effectiveness of this approach in surveys with respect to quantitative variables related to drug use. This paper describes a probabilistic, stratified sample of 1146 university students asking sensitive quantitative questions about cannabis use in Spanish universities, conducted using the RRT. On comparing the results of the direct question (DQ) survey and those of the randomized response (RR) survey, we find that the number of cannabis cigarettes consumed during the past year (DQ: 3, RR: 17 aproximately), and the number of days when consumption took place (DQ: 1, RR: 7) are much higher with RRT. The advantages of RRT, reported previously and corroborated in our study, make it a useful method for investigating cannabis use.Ministerio de Educación, Cultura y DeporteConsejería de Economía, Innovación, Ciencia y Emple

Regression estimation for finite population means in the presence of nonresponse

Nonresponse is one source of error in survey sampling. When the respondents and non-respondents are different, nonresponse introduces bias into the estimation for a population characteristic of a finite population. Regression estimation can be used to reduce the bias from nonresponse by using auxiliary variables;We investigate the consistency of regression estimators and of Horvitz-Thompson estimators with adjustment for nonresponse probabilities for stratified cluster sampling designs. The consistency of the regression estimator adjusted by the estimated response probabilities is established. Regression estimators and associated weighting procedures are applied to the data from the Survey of Income and Program Participation (SIPP) A model for the response probabilities is estimated for the SIPP

Efficiency of propensity score adjustment and calibration on the estimation from non-probabilistic online surveys

One of the main sources of inaccuracy in modern survey techniques, such as online and smartphone surveys, is the absence of an adequate sampling frame that could provide a probabilistic sampling. This kind of data collection leads to the presence of high amounts of bias in final estimates of the survey, specially if the estimated variables (also known as target variables) have some influence on the decision of the respondent to participate in the survey. Various correction techniques, such as calibration and propensity score adjustment or PSA, can be applied to remove the bias. This study attempts to analyse the efficiency of correction techniques in multiple situations, applying a combination of propensity score adjustment and calibration on both types of variables (correlated and not correlated with the missing data mechanism) and testing the use of a reference survey to get the population totals for calibration variables. The study was performed using a simulation of a fictitious population of potential voters and a real volunteer survey aimed to a population for which a complete census was available. Results showed that PSA combined with calibration results in a bias removal considerably larger when compared with calibration with no prior adjustment. Results also showed that using population totals from the estimates of a reference survey instead of the available population data does not make a difference in estimates accuracy, although it can contribute to slightly increment the variance of the estimator

Measuring inappropriate sexual behavior among university students: using the randomized response technique to enhance self-reporting

This article analyzes the efficacy of the randomized response technique (RRT) in achieving honest self-reporting about sexual behavior, compared with traditional survey techniques. A complex survey was conducted of 1,246 university students in Spain, who were asked sensitive quantitative questions about their sexual behavior, either via the RRT (n = 754) or by direct questioning (DQ) (n = 492). The RRT estimates of the number of times that the students were unable to restrain their inappropriate sexual behavior were significantly higher than the DQ estimates, among both male and female students. The results obtained suggest that the RRT method elicits higher values of self-stigmatizing reports of sexual experiences by increasing privacy in the data collection process. The RRT is shown to be a useful method for investigating sexual behavior

A Mixed-Mode Sensitive Research on Cannabis Use and Sexual Addiction: Improving Self-Reporting by Means of Indirect Questioning Techniques

In this article, we describe the methods employed and the results obtained from a mixed-mode “sensitive research” conducted in Spain to estimate certain aspects concerning patterns of cannabis consumption and sexual addiction among university students. Three different data-collection methods are considered and compared: direct questioning, randomized response technique and item sum technique. It is shown that posing direct questions to obtain sensitive data produces significantly lower estimates of the surveyed characteristics than do indirect questioning methods. From the analysis, it emerges that male students seem to be more affected by sex addiction than female students while for cannabis consumption there is no evidence of a predominant gender effect.Ministerio de Economía y CompetitividadMinisterio de Educación, Cultura y DeportePRIN-SURWE

New estimation techniques for ordinal sensitive variables

Methods to analyze multicategorical variables are extensively used in sociological, medical and educational research. Nonetheless, they have a very sparse presence in finite population sampling when sensitive topics are investigated and data are obtained by means of the randomized response technique (RRT), a survey method based on the principle that sensitive questions must not be asked directly to the respondents. The RRT is used with the aim of reducing social desirability bias, which is defined as the respondent tendency to release personal information according to what is socially acceptable. This nonstandard data-collection approach was originally developed to deal with dichotomous responses to sensitive questions. Later, the idea has been extended to multicategory responses. In this paper we consider ordinal variables with more than two response categories. In particular, we first discuss the theoretical framework for estimating the frequency of ordinal categories when data are subjected to misclassification due to the use of a particular RRT. Then, we show how it is possible to improve the efficiency of the inferential process by employing auxiliary information at the estimation stage through the calibration approach. Finally, we assess the performance of the proposed estimators in a Monte Carlo simulation study.Ministerio de Econom´ıa y Competitividad of Spai