Barth , Nelson Lerner

Ponchio, Mateus Canniatti

Zambaldi, Felipe

English

Cadernos Espinosanos (E-Journal)

ThinkBoxUsing the internet for datacollection in business researchMateus Canniatti PonchioEscola Superior de Propaganda e Marketing – ESPM, Sao Paulo, Brazil, andNelson Lerner Barth and Felipe ZambaldiSchool of Business Administration in Sao Paulo, Fundacao Getulio Vargas,Sao Paulo, BrazilIntroductionImagine a master’s degree student facing the need to collect primary data to proceed intoquantitative analysis in the business field. The tight deadline for getting the degree and theinherent difficulty in research planning and collecting data can both be frightening. Beforethe internet, the alternative would be a convenience non-probabilistic sampling, which is agenerally easier and faster way to obtain data than probabilistic sampling. Thus,considering the internet’s use to speed up data collection and reduce costs, what are theadvantages and recommendations to be accounted for by business researchers?The use of the internet to conduct surveys is attractive for business researchers due to itsbenefits in speed, access to respondents and the large amounts of data it can gather. Themain advantages of online versus offline data collection are time-saving, cost reduction,simplified data tabulation and purification processes, flexibility and format control.However, there are issues related to respondents’ attention, sample representativeness andcontrol, which can be amplified as most of the data collection for business research on theinternet occurs through convenience sampling. Also, self-selection bias can represent severejeopardy in online surveys.In this short essay, we draw on the problem of sample representativeness on Web-baseddata collection, and then we discuss it with emphasis on two common procedures to mitigateits risks and challenges: the correction of snowball sampling bias; and the use of respondent panels.The reflection is structured in four additional sections. Firstly, we introduce the issuesrelated to convenience sampling size and the problem of representativeness. We then© Mateus Canniatti Ponchio, Nelson Lerner Barth, Felipe Zambaldi, Mateus Canniatti Ponchio,Nelson Lerner Barth and Felipe Zambaldi. Published in RAUSP Management Journal. Published byEmerald Publishing Limited. This article is published under the Creative Commons Attribution (CCBY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of thisarticle (for both commercial and non-commercial purposes), subject to full attribution to the originalpublication and authors. The full terms of this licence maybe seen at http://creativecommons.org/licences/by/4.0/legalcode.RAUSP56,2254RAUSP Management JournalVol. 56 No. 2, 2021pp. 254-258EmeraldPublishingLimited2531-0488DOI 10.1108/RAUSP-04-2021-251The current issue and full text archive of this journal is available on Emerald Insight at:https://www.emerald.com/insight/2531-0488.htmdiscuss approaches to correct sampling bias and the use of respondent panels fromprofessional market research suppliers. Finally, we bring further caution-related aspects ofconducting online data surveys and present other data collection issues for businessresearch on the internet and their need for future reflections.Large convenience sampling and representativenessThere is a common misleading belief that if a sample is large (perhaps with millions ofresponses), it is automatically proper for inference about a population of interest, whichcould avoid the need to obtain a probabilistic sample. One can indeed remove the elementsthat do not belong to the sample’s target population (e.g. non-buyers, if the target populationis buyers only). However, even if the sample size has practically the same order ofmagnitude of the population, sampling biases may remain as there can be certain profilesthat are over-represented or under-represented.One can try to correct the proportions observed in the sample by weighting the actualpopulational proportion, which is called quota sampling (Rukmana, 2014). For example, ifwe admit (fictitious number) that 52% of a population comprises females, we can balancethe sample to respect this percentage. We can apply the same solution to numerous othereasily observable variables (age and type of housing). However, it is delusional to think thatthis method can eliminate the bias of over- or under-representation of specific profiles, asthere are countless other variables for which we cannot adjust the sample’s proportions(consumer behaviours, convictions, sensitivities, aversions, attractions and wishes). In otherwords, quota sampling is not an adequate substitute for probabilistic sampling when weneed statistical inferences for a targeted population.Out of the convenience sampling techniques that allow for obtaining large samples onthe internet, snowball is quite popular due to its easiness to reach respondents. In thesnowball sampling technique (Heckathorn, 2011), a group of people is invited to respond toquestionnaires, usually using social networks and/or emails. The researchers ask eachrespondent to pass on the invitation to other people while offering rewards to theparticipants (that can be a gift, cash value or merely the prestige and pleasure of belongingto the group). But even if it is possible to restrict the sample to the target audience of theresearch, this technique can generate strong bias in its results. Two of these possible effectsare there is no randomisation in the choice of the initial respondents, and there is norandomisation when respondents invite other respondents from their networks. There are,though, advantages in the use of snowball sampling and possible corrections for its bias,which we discuss next.Snowball sampling bias and correctionsDigital social networks facilitate surveys to obtain data, as we can find and recruit largesamples of respondents to apply questionnaires. Nevertheless, the use of the internet forsampling can be problematic because it may reproduce difficulties that were already thereeven before the internet. For example, textbooks have alerted us about conveniencesampling problems, pitfalls of snowball techniques and other non-probabilistic samplingapproaches (Churchill & Iacobucci, 2005; Pedhazur & Schmelkin, 1991). These are preciselythe most common data collection approaches through the internet that we see in businessresearch.However, we should note that snowball sampling makes data collection moremanageable, allowing access to typically censored groups (e.g. drug users). There are nopreviously available lists to select respondents from randomly. This fact encouraged thecreation and development of statistical techniques to correct the inherent bias of samplingBusinessresearch255methods. Such techniques belong to the category respondent-driven sampling (RDS)(Beaudry & Gile, 2020; Heckathorn, 2011). The non-probabilistic choice of respondents’ biasis corrected based on the information passed on by the respondents themselves about thecharacteristics and size of their networks of contacts, admitting that there is a large numberof successive invitations to new respondents.Tools for applying RDS include RDSAT, based on Java (Volz et al., 2012), and R’s RDSlibrary (Handcock, Fellows, & Gile, 2019). Given the need for mastering specific statisticalknowledge to use these techniques for snowball sampling bias correction – usually not theprimary specialities of business researchers – the technique requires knowledgeableresearchers in terms of these bias correction procedures or specialised professionals’ supportto run the corrections.In addition to correcting sampling bias, one can avoid them by using sampling controls.There are professional market research suppliers that are skilled and offer respondent panelservices. Next, we present the characteristics of these panels and some reflection on them.Respondent panelsMany research firms provide respondent panels (a search on Google with the words “onlineresearch panel” returns dozens of companies). Usually, these companies recruit potentialrespondents to get registered in their databases, who get rewards to completequestionnaires. The cost for researchers to have access to these online panels can varyaccording to the target population (e.g. a survey with directors of multinational companieswill be more expensive, per questionnaire, than a survey with bottled water consumers, asthe former are harder to find and recruit). The probably most well-known platform foraccessing respondents is Amazon Mechanical Turk (MTurk), a crowdsourcing service. Itsusers, such as organisations and researchers, can hire crowdworkers, remote outsourcedrespondents who perform the tasks of completing questionnaires.A variety of studies have accessed the benefits of using MTurk (Barends & Vries, 2019;Cheung, Burns, Sinclair, & Sliter, 2017; Follmer, Sperling, & Suen, 2017). For instance, dataquality assurance, as MTurk often includes attention-checking questions; nevertheless,researchers should include their own set of attention checks as well. A typical criticismregarding respondent panels is that, as crowdworkers, participants may tend to becomeprofessional respondents along time, thus losing their spontaneity in providing answers,which cannot be monitored by attention checks. On the contrary, they may becomeoverfocussed on not missing the checks and guaranteeing their rewards or even trying toprovide answers that they believe the researchers would like them to give. This behaviourmight become a serious issue, as the most common situations in primary business researchare those in which questionnaires require spontaneity from respondents.For the present discussion, it suffices to say that, regardless of the media used to gatherrespondents, there must be a focus on the respondent panel’s suitability to the researchobjectives. For example, not everyone has access to email or the internet. The globalpenetration of the internet is estimated to be roughly 60% at the end of 2020 (Statista Portal,2020), which varies significantly according to regional infrastructure and developmentvariability. Therefore, research objectives should determine the target population and, thus,the appropriateness of online data collection approaches.Further recommendations for conducting online data collectionWeb-based online data collection methods create opportunities to conduct research globally,especially among difficulties to access populations. It should be noted that, although onlinedata collection brings cost- and time-related benefits, especially for geographically dispersedRAUSP56,2256populations, the internet partly reproduces and amplifies limitations that existed priorly.Web-based research requires careful consideration of how studies will be promoted and howdata will be collected to ensure the validity of findings (Cantrell & Lupinacci, 2007; Lefever,Dal, &Matthíasdottir, 2007; Topp & Pawloski, 2002). Caution should be the rule for defininga target population and the criteria to allow the drawing of an unbiased sample.Online data collection can be carried out using different interfaces that require internetbrowsers or a smartphone application. Each approach has advantages and disadvantages,and the appropriate choice can save time when organising and analysing data. It is goodpractice to look for evidence of response rate (refusal rate) and selection bias. As people areoverwhelmed with online information, an invitation to participate in a study may be skippedor consciously ignored, bringing distinct consequences to the research’s response rate.In this short essay, we have discussed concerns about using the internet and digitalsocial networks to conduct surveys. However, other limitations persist and require newreflections, such as further issues regarding sample representativeness, computation ofresponse rates and technical difficulties. Besides that, future business research needs toconsider how to match the online environment’s characteristics with common textbookrecommendations for surveys. For example, defining study populations clearly, how tocontact respondents, designing instruments for data collection while assuring face andcontent validity, pre-testing and adjusting, respondents’ privacy and ethical aspects(Churchill & Iacobucci, 2005).Finally, non-survey Web-based techniques for data collection, such as observingsocial networks, monitoring internet browsing behaviours and applications forgathering data in mobile devices, can provide researchers with relevant information butrequire further understanding. Ethnographic studies on digital and social networks,known as digital ethnography and cyber ethnography (Atay, 2020; Lester, 2020) ornetnography (Kozinets, 2019) may imply obtaining data from digital social networks orthe Web environment. This practice, in general, leads to conducting behaviour analyseswithout asking any questions. When the object of study is the population of individualswho use the digital social network, there are no issues related to poor sampling or lackof representativeness. There are, however, privacy and ethical concerns to be observed.As non-survey techniques to collect data on the internet tend to become more frequentin business research, the knowledge on them needs to be improved and furtherdiscussed by the business research community.There is no one-size-fits-all answer to whether online data collection will be better or evensuitable for a research study. The answer needs to consider the research questions andstudy objectives. The data source’s quality cannot be defined by the data source itself, butrather by the researchers’ knowledge and consequent decisions during the stages of studydesign, data collection and analysis.ReferencesAtay, A. (2020). What is cyber or digital autoethnography? International Review of QualitativeResearch, 13(3), 267–279. doi: https://doi.org/10.1177/1940844720934373.Barends, A. J., & Vries, R. E. (2019). Noncompliant responding: Comparing exclusion criteria in MTurkpersonality research to improve data quality. Personality and Individual Differences, 143(1), 84–89. doi: https://doi.org/10.1016/j.paid.2019.02.015.Beaudry, I. S., & Gile, K. J. (2020). Correcting for differential recruitment in respondent-driven samplingdata using ego-network information. Electronic Journal of Statistics, 14(2), 2678–2713. doi:https://doi.org/10.1214/20-EJS1718.Businessresearch257Cantrell, M. A., & Lupinacci, P. (2007). Methodological issues in online data collection. Journal ofAdvanced Nursing, 60(5), 544–549. doi: https://doi.org/10.1111/j.1365-2648.2007.04448.x.Cheung, J. H., Burns, D. K., Sinclair, R. R., & Sliter, M. (2017). Amazon mechanical Turk inorganizational psychology: an evaluation and practical recommendations. Journal of Businessand Psychology, 32(4), 347–361. doi: https://doi.org/10.1007/s10869-016-9458-5.Churchill, G. A., & Iacobucci, D. (2005).Marketing research: methodological foundations, 9th ed., p. 697.Mason, OH: Thomson.Follmer, D. J., Sperling, R. A., & Suen, H. K. (2017). The role of MTurk in education research:Advantages, issues, and future directions. Educational Researcher, 46(6), 329–334. doi: https://doi.org/10.3102/0013189X17725519.Handcock, M. S. Fellows, I. E., & Gile, K. J. (2019). RDS: Respondent-Driven sampling. R packageversion 0.9-2. Retrieved from: https://CRAN.R-project.org/package=RDSHeckathorn, D. D. (2011). Snowball versus respondent-driven sampling. Sociological Methodology, 41(1),355–366. doi: https://doi.org/10.1111/j.1467-9531.2011.01244.x.Kozinets, R. V. (2019). Netnography: the essential guide to qualitative social media research, SAGEPublications Limited.Lefever, S., Dal, M., & Matthíasdottir, A. (2007). Online data collection in academic research:Advantages and limitations. British Journal of Educational Technology, 38(4), 574–582. doi:https://doi.org/10.1111/j.1467-8535.2006.00638.x.Lester, J. N. (2020). Going digital in ethnography: Navigating the ethical tensions and productivepossibilities. Cultural Studies $ Critical Methodologies, 20(5), 414–424. doi: https://doi.org/10.1177/1532708620936995.Pedhazur, E., & Schmelkin, L. (1991). Measurement, design, and analysis: an integrated approach,pp. 819. Hillsdale, NJ: Lawrence ErlbaumAssociates. pRukmana, D. (2014). Quota sampling. In A. C. Michalos, (Ed.), Encyclopedia of quality of life and Well-Being research, Dordrecht: Springer.Statista Portal. (2020). Global digital population as of October 2020. Retrieved from www.statista.com/statistics/617136/digital-population-worldwidTopp, N. W., & Pawloski, B. (2002). Online data collection. Journal of Science Education andTechnology, 11(2), 173–178. doi: https://doi.org/10.1023/A:1014669514367.Volz, E., Wejnert, C., Cameron, C., Spiller, M., Barash, V., Degani, I., & Heckathorn, D. D. (2012).Respondent-driven sampling analysis tool (RDSAT) version 7.1, Ithaca, New York, NY: CornellUniversity.For instructions on how to order reprints of this article, please visit our website:www.emeraldgrouppublishing.com/licensing/reprints.htmOr contact us for further details: permissions@emeraldinsight.comRAUSP56,2258

Using the internet for data collection in business research

https://www.revistas.usp.br/rmj/article/download/188759/174321

Using the internet for data collection in business research

Abstract

Similar works

Full text

Available Versions

Cadernos Espinosanos (E-Journal)