A Study of Ethics in Crowd Work-Based Research

Crowd work as a form of a social-technical system has become a popular setting for conducting and distributing academic research. Crowd work platforms such as Amazon Mechanical Turk (MTurk) are widely used by academic researchers. Recent scholarship has highlighted the importance of ethical issues because they could affect the long-term development and application of crowd work in various fields such as the gig economy. However, little study or deliberation has been conducted on the ethical issues associated with academic research in this context. Current sources for ethical research practice, such as the Belmont Report, have not been examined thoroughly on how they should be applied to tackle the ethical issues in crowd work-based research such as those in data collection and usage. Hence, how crowd work-based research should be conducted to make it respectful, beneficent, and just is still an open question. This dissertation research has pursued this open question by interviewing 15 academic researchers and 17 IRB directors and analysts in terms of their perceptions and reflections on ethics in research on MTurk; meanwhile, it has analyzed 15 research guidelines and consent templates for research on MTurk and 14 published papers from the interviewed scholars. Based on analyzing these different sources of data, this dissertation research has identified three dimensions of ethics in crowd work-based research, including ethical issues in payment, data, and human subjects. This dissertation research also uncovered the “original sin” of these ethical issues and discussed its impact in academia, as well as the limitations of the Belmont Report and AoIR Ethical Guidelines 3.0 for Internet Research. The findings and implications of this research can help researchers and IRBs be more conscious about ethics in crowd work-based research and also inspire academic associations such as AoIR to develop ethical guidelines that can address these ethical issues

Steps to an Ecology of Networked Knowledge and Innovation: Enabling new forms of collaboration among sciences, engineering, arts, and design

SEAD network White Papers ReportThe final White Papers (posted at http://seadnetwork.wordpress.com/white-paper- abstracts/final-white-papers/) represent a spectrum of interests in advocating for transdisciplinarity among arts, sciences, and technologies. All authors submitted plans of action and identified stakeholders they perceived as instrumental in carrying out such plans. The individual efforts led to an international scope. One of the important characteristics of this collection is that the papers do not represent a collective aim toward an explicit initiative. Rather, they offer a broad array of views on barriers faced and prospective solutions. In summary, the collected White Papers and associated Meta- analyses began as an effort to take the pulse of the SEAD community as broadly as possible. The ideas they generated provide a fruitful basis for gauging trends and challenges in facilitating the growth of the network and implementing future SEAD initiatives.National Science Foundation Grant No.1142510. Additional funding was provided by the ATEC program at the University of Texas at Dallas and the Institute for Applied Creativity at Texas A&M University

A Multi-Dimensional Approach for Framing Crowdsourcing Archetypes

All different kinds of organizations – business, public, and non-governmental alike – are becoming aware of a soaring complexity in problem solving, decision making and idea development. In a multitude of circumstances, multidisciplinary teams, high-caliber skilled resources and world-class computer suites do not suffice to cope with such a complexity: in fact, a further need concerns the sharing and ‘externalization’ of tacit knowledge already existing in the society. In this direction, participatory tendencies flourishing in the interconnected society in which we live today lead ‘collective intelligence’ to emerge as key ingredient of distributed problem solving systems going well beyond the traditional boundaries of organizations. Resulting outputs can remarkably enrich decision processes and creative processes carried out by indoor experts, allowing organizations to reap benefits in terms of opportunity, time and cost. Taking stock of the mare magnum of promising opportunities to be tapped, of the inherent diversity lying among them, and of the enormous success of some initiative launched hitherto, the thesis aspires to provide a sound basis for the clear comprehension and systematic exploitation of crowdsourcing. After a thorough literature review, the thesis explores new ways for formalizing crowdsourcing models with the aim of distilling a brand-new multi-dimensional framework to categorize various crowdsourcing archetypes. To say it in a nutshell, the proposed framework combines two dimensions (i.e., motivations to participate and organization of external solvers) in order to portray six archetypes. Among the numerous significant elements of novelty brought by this framework, the prominent one is the ‘holistic’ approach that combines both profit and non-profit, trying to put private and public sectors under a common roof in order to examine in a whole corpus the multi-faceted mechanisms for mobilizing and harnessing competence and expertise which are distributed among the crowd. Looking at how the crowd may be turned into value to be internalized by organizations, the thesis examines crowdsourcing practices in the public as well in the private sector. Regarding the former, the investigation leverages the experience into the PADGETS project through action research – drawing on theoretical studies as well as on intensive fieldwork activities – to systematize how crowdsourcing can be fruitfully incorporated into the policy lifecycle. Concerning the private realm, a cohort of real cases in the limelight is examined – having recourse to case study methodology – to formalize different ways through which crowdsourcing becomes a business model game-changer. Finally, the two perspectives (i.e., public and private) are coalesced into an integrated view acting as a backdrop for proposing next-generation governance model massively hinged on crowdsourcing. In fact, drawing on archetypes schematized, the thesis depicts a potential paradigm that government may embrace in the coming future to tap the potential of collective intelligence, thus maximizing the utilization of a resource that today seems certainly underexploited

Socialising Big Data: From concept to practice

The working paper is a report on an ESRC-fundedproject, Socialising Big Data, that sought to address problematic conceptions of Big Data in popular discourse such as the ‘data deluge’ and the tendency to reduce the term to definitions such as the oft-cited‘3 Vs’. Instead, building on how social scientists have conceived of things, methods and data as having social and cultural lives, the project sought to identify the normative, political and technical imperatives and choices that come to shape Big Data at various momentsin its social lives. Recognising that Big Data involves distributed practices across a range of fields, the project experimentedwith collaboratories as a method for bringing together and engaging with practitioners acrossthree different domains –genomics, national statistics and waste management. In this way it explored how relations between data are also simultaneously relations between people and that it is through such relations that a shared literacyand social framework for Big Datacan be forged

Users' motivation to participate in crowdsourcing: a South African case study

Growth in online communities has generated a new labour pool. Organisations are using Web 2.0 tools to tap into this online labour pool, with one approach being 'crowdsourcing'. People from different geographical destinations can now work for organisations that are thousands of kilometres from them. Organisations face a huge task of attracting a large crowd of workers that can actively contribute answers to their business problems. Knowing what motivates users and how to keep them actively participating over a long period of time is therefore crucial. This study explores how organisational, individual, technical and social factors affect users' motivation to participate in crowdsourcing projects. A single case study using a crowdsourcing company based in South Africa was used. The crowdsourcing company uses crowdsourcing for monitoring online activities on behalf of other companies for online conversations on social platforms such as Twitter, Facebook, news articles, blog posts and listings on directory sources such as Gumtree or property listings. A qualitative study on thirteen participants was conducted through semi - structured Skype interviews. A conceptual model is presented based on the research findings. Besides re - establishing a number of factors which affect motivation to participate in crowdsourcing, the study established new emergent factors which had not been common in previous studies. The factors include authenticity of the whole crowdsourcing project, mentorship of new users by seasoned users, flexibility of technological tools in meeting users' expectations and feedback. Practical lessons drawn from the study could help crowdsourcing practitioners understand users' motivation to participate in crowdsourcing and how to ensure a conducive environment for crowd participation and hence quality output. Additionally the study could inform key considerations when implementing a crowdsourcing project in an organisation

Designing for quality in real-world mobile crowdsourcing systems

PhD ThesisCrowdsourcing has emerged as a popular means to collect and analyse data on a scale for problems that require human intelligence to resolve. Its prompt response and low cost have made it attractive to businesses and academic institutions. In response, various online crowdsourcing platforms, such as Amazon MTurk, Figure Eight and Prolific have successfully emerged to facilitate the entire crowdsourcing process. However, the quality of results has been a major concern in crowdsourcing literature. Previous work has identified various key factors that contribute to issues of quality and need to be addressed in order to produce high quality results. Crowd tasks design, in particular, is a major key factor that impacts the efficiency and effectiveness of crowd workers as well as the entire crowdsourcing process. This research investigates crowdsourcing task designs to collect and analyse two distinct types of data, and examines the value of creating high-quality crowdwork activities on new crowdsource enabled systems for end-users. The main contribution of this research includes 1) a set of guidelines for designing crowdsourcing tasks that support quality collection, analysis and translation of speech and eye tracking data in real-world scenarios; and 2) Crowdsourcing applications that capture real-world data and coordinate the entire crowdsourcing process to analyse and feed quality results back. Furthermore, this research proposes a new quality control method based on workers trust and self-verification. To achieve this, the research follows the case study approach with a focus on two real-world data collection and analysis case studies. The first case study, Speeching, explores real-world speech data collection, analysis, and feedback for people with speech disorder, particularly with Parkinson’s. The second case study, CrowdEyes, examines the development and use of a hybrid system combined of crowdsourcing and low-cost DIY mobile eye trackers for real-world visual data collection, analysis, and feedback. Both case studies have established the capability of crowdsourcing to obtain high quality responses comparable to that of an expert. The Speeching app, and the provision of feedback in particular were well perceived by the participants. This opens up new opportunities in digital health and wellbeing. Besides, the proposed crowd-powered eye tracker is fully functional under real-world settings. The results showed how this approach outperforms all current state-of-the-art algorithms under all conditions, which opens up the technology for wide variety of eye tracking applications in real-world settings

Geoinformatics in Citizen Science

The book features contributions that report original research in the theoretical, technological, and social aspects of geoinformation methods, as applied to supporting citizen science. Specifically, the book focuses on the technological aspects of the field and their application toward the recruitment of volunteers and the collection, management, and analysis of geotagged information to support volunteer involvement in scientific projects. Internationally renowned research groups share research in three areas: First, the key methods of geoinformatics within citizen science initiatives to support scientists in discovering new knowledge in specific application domains or in performing relevant activities, such as reliable geodata filtering, management, analysis, synthesis, sharing, and visualization; second, the critical aspects of citizen science initiatives that call for emerging or novel approaches of geoinformatics to acquire and handle geoinformation; and third, novel geoinformatics research that could serve in support of citizen science

Human Behavior Experimentation and Participation in Scientific Activities in the Wild

La cooperació és un dels trets del comportament que defineixen els éssers humans, però, encara estem tractant de comprendre per què els humans cooperem. Els experiments conductuals estan dissenyats per donar llum sobre els mecanismes de cooperació i altres trets del comportament. Aquesta dissertació es centra en avançar en el camp de l'experimentació conductual utilitzant les pràctiques de ciència ciutadana, i es divideix en dos blocs. En el primer, presentem dues plataformes, una que permet estudiar com promoure el pensament científic i la participació científica, i l'altra que permet estudiar els trets del comportament humà amb un conjunt de jocs de comportament. Les dues plataformes estàn dissenyades per ajudar a crear experiments en el camp i per fomentar la participació en el marc de la ciència ciutadana. En el segon bloc avaluem les plataformes a través d'un conjunt d'experiments, i analitzem l'existència de patrons de comportament. Primer, vam estudiar la solidesa de la plataforma observant si sorgeixen estratègies iguals en rèpliques del mateix experiment realitzat amb diferents mostres de població. En el segon experiment, analitzem els patrons de comportament que emergeixen quan els participants s'enfronten a un conjunt de dilemes socials. Els dos últims experiments són "collective-risk dilemmas" sobre el canvi climàtic. En un, estudiem com les desigualtats de recursos generen comportaments injustos. L'altre es porta a terme dins d'un ecosistema concret per estudiar les tensions que hi ha entre els diferents actors del col·lectiu. Tenint en compte els resultats dels experiments, podem entendre com ens comportem quan enfrontem dilemes socials i, en conseqüència, avaluar els trets de comportament i l'aparició de patrons de comportament. Els dissenys, els resultats i la metodologia d'anàlisi presentats en aquest treball ajudaran a establir les bases per a futurs experiments de comportament al camp.La cooperación es uno de los rasgos de comportamiento que definen a los seres humanos, sin embargo, todavía estamos tratando de comprender por qué los humanos cooperamos. Los experimentos conductuales están diseñados para arrojar luz sobre los mecanismos de cooperación y otros rasgos de comportamiento. Esta disertación se centra en avanzar en el campo de la experimentación conductual utilizando las prácticas de ciencia ciudadana, y se divide en dos bloques. En el primero, presentamos dos plataformas, una que permite estudiar cómo promover el pensamiento científico y la participación científica, y otra para estudiar los rasgos del comportamiento humano con un conjunto de juegos de comportamiento. Ambas plataformas están diseñadas para ayudar a crear experimentos en el campo y para fomentar la participación en el marco de la ciencia ciudadana. En el segundo bloque evaluamos las plataformas a través de un conjunto de experimentos, y analizamos la existencia de patrones de comportamiento. Primero, estudiamos la solidez de la plataforma al observar si surgen estrategias iguales en réplicas del mismo experimento realizado con diferentes muestras de población. En el segundo experimento, analizamos los patrones de comportamiento que emergen cuando los participantes enfrentan un conjunto de dilemas sociales. Los dos últimos experimentos son "collective-risk dilemmas" sobre el cambio climático. En uno, estudiamos cómo las desigualdades de recursos generan comportamientos injustos. El otro se lleva a cabo dentro de un ecosistema concreto para estudiar las tensiones que existen entre los diferentes actores del colectivo. Teniendo en cuenta los resultados de los experimentos, podemos entender cómo nos comportamos cuando enfrentamos dilemas sociales y, en consecuencia, evaluar los rasgos de comportamiento y la aparición de patrones de comportamiento. Los diseños, los resultados y la metodología de análisis presentados en este trabajo ayudarán a establecer las bases para futuros experimentos de comportamiento en el campo.Cooperation is one of the behavioral traits that define human beings, however we are still trying to understand why humans cooperate. Behavioral experiments are designed to shed light into the mechanisms behind cooperation -- and other behavioral traits. This dissertation is focused on advancing the field of behavioral experimentation using experiments based on citizen science, and it is divided in two blocks. In the first, we present two platforms, one to understand how it can serve as a catalyst to promote of scientific thinking and engage in science, and another to study traits of human behavior with a suite of behavioral games. Both platforms are designed to help creating experiments in the wild and to encourage the participation within the framework of citizen science. In the second block we evaluate the platforms through a set of experiments, and we analyze the existence of behavioral patterns. First, we study the robustness of the platform by looking whether equal strategies emerge in replicas of the same experiment performed with different population samples. In the second experiment we analyze the behavioral patterns that emerge when participants face a set of social dilemmas. The last two experiments are collective-risk dilemmas framed in climate change. In one, we study how the resource inequalities generate unfair behaviors. The other is carried out within a given ecosystem to study the tensions that exist between actors of the collective. Considering the experiments' results, we can comprehend how we behave when we face social dilemmas, and consequently evaluate behavioral traits and the emergence of behavioral patterns. The designs, the results and the methodology of analysis presented in this work will help set the basis for future behavioral experiments in the field

Essays on Strategies for Increasing Repayment Rates of Digital Microloans

Access to credit can act as a highly effective tool for poverty reduction and economic growth. The ability to borrow increases the propensity of low-income people to start and maintain businesses, educate their children and withstand financial shocks. These factors, in turn, can help them to move out of poverty and lead to more sustainable economic development. However, traditional financial institutions have inherent limitations that have impeded their ability to serve the poor. Digital lenders are able to leverage the widespread adoption of mobile phones and mobile money to extend credit quickly and conveniently to more people, especially in developing countries. However, due to a lack of credit bureaus and available financial histories of borrowers, digital lenders frequently need to amass vast amounts of data in order to screen borrowers and experiment to find the appropriate loan amount by gradually increasing credit limits based on past repayment. This can lead to high user default rates and over-indebtedness. The lack of collateral during loan applications also means that digital lenders have limited mechanisms for enforcing repayment of loans. Both of these challenges threaten to limit further adoption of digital credit. Through three experimental studies conducted with an airtime lender, I explore theoretical and empirical mechanisms for reducing default rates of digital loans. In the first study, I demonstrate that limited mobile phone data contain enough signals for creating effective credit assessment methods that minimize privacy risks to borrowers. In the second study, I find that increasing credit limits negatively impacts repayments and future borrowing, and offer recommendations for increasing credit limits while minimizing the drawbacks. In the final study, I draw on theories from psychology and consumer behavior to develop vivid repayment reminders. This study found that vivid reminders had limited effectiveness for increasing loan repayment and reducing loan duration. Taken together, these three studies propose new avenues for digital lenders to reduce default rates. The hope of this dissertation is that these proposed methods would lead to a reduction in interest rates, that would ultimately benefit the borrowers