A survey of the use of crowdsourcing in software engineering

The term 'crowdsourcing' was initially introduced in 2006 to describe an emerging distributed problem-solving model by online workers. Since then it has been widely studied and practiced to support software engineering. In this paper we provide a comprehensive survey of the use of crowdsourcing in software engineering, seeking to cover all literature on this topic. We first review the definitions of crowdsourcing and derive our definition of Crowdsourcing Software Engineering together with its taxonomy. Then we summarise industrial crowdsourcing practice in software engineering and corresponding case studies. We further analyse the software engineering domains, tasks and applications for crowdsourcing and the platforms and stakeholders involved in realising Crowdsourced Software Engineering solutions. We conclude by exposing trends, open issues and opportunities for future research on Crowdsourced Software Engineering

Quality Control in Crowdsourcing: A Survey of Quality Attributes, Assessment Techniques and Assurance Actions

Crowdsourcing enables one to leverage on the intelligence and wisdom of potentially large groups of individuals toward solving problems. Common problems approached with crowdsourcing are labeling images, translating or transcribing text, providing opinions or ideas, and similar - all tasks that computers are not good at or where they may even fail altogether. The introduction of humans into computations and/or everyday work, however, also poses critical, novel challenges in terms of quality control, as the crowd is typically composed of people with unknown and very diverse abilities, skills, interests, personal objectives and technological resources. This survey studies quality in the context of crowdsourcing along several dimensions, so as to define and characterize it and to understand the current state of the art. Specifically, this survey derives a quality model for crowdsourcing tasks, identifies the methods and techniques that can be used to assess the attributes of the model, and the actions and strategies that help prevent and mitigate quality problems. An analysis of how these features are supported by the state of the art further identifies open issues and informs an outlook on hot future research directions.Comment: 40 pages main paper, 5 pages appendi

機械学習を用いたモバイルWeb通信品質の測定評価に関する研究

電気通信大学202

Assessing the effectiveness of crowdsourced geographic information for solid waste management in Timor-Leste : a thesis presented in partial fulfilment of the requirements for the degree of Master of Information Sciences (Information Technology) at Massey University, Albany, New Zealand

Dili, the capital city of Timor-Leste has been faced with serious solid waste problems in recent years. Responding to this issue, the government has adopted various policies including setting up solid waste collection sites in community areas and outsourcing collection to the private sector to collect waste directly from homes in several areas. Despite, these efforts, waste is still found scattered on the roads and disposed of in rivers and open lands. A proper solid waste management strategy is necessary to transform the city into a clean city. In order to develop an effective solid waste management strategy, reliable data and public participation are required. This study, therefore, investigated whether crowdsourcing, in particular, Volunteered Geographic Information (VGI) can effectively be used to collect data about solid waste disposal and collection practices in Dili and raise awareness of the impact of waste disposal practices among the public. The study result demonstrated that crowdsourcing is a viable method for collecting solid waste data. Challenges such as collecting accurate location-specific data still remain, hence, the crowdsourced dataset may not entirely substitute for the usual traditional dataset. At this stage, however, the collected data can still be utilized as a supplementary data source. In the future, by improving data collection methodologies, such as using smaller rewards or providing necessary facilities, a crowdsourcing-based data collection method could be utilized as an adequate substitute for traditional data source because of its ability to collect data in real- time with lower operational costs. This approach is feasible for a developing country such as Timor-Leste where critical area such as waste management has less priority for funding

Multi-objective Search-based Mobile Testing

Despite the tremendous popularity of mobile applications, mobile testing still relies heavily on manual testing. This thesis presents mobile test automation approaches based on multi-objective search. We introduce three approaches: Sapienz (for native Android app testing), Octopuz (for hybrid/web JavaScript app testing) and Polariz (for using crowdsourcing to support search-based mobile testing). These three approaches represent the primary scientific and technical contributions of the thesis. Since crowdsourcing is, itself, an emerging research area, and less well understood than search-based software engineering, the thesis also provides the first comprehensive survey on the use of crowdsourcing in software testing (in particular) and in software engineering (more generally). This survey represents a secondary contribution. Sapienz is an approach to Android testing that uses multi-objective search-based testing to automatically explore and optimise test sequences, minimising their length, while simultaneously maximising their coverage and fault revelation. The results of empirical studies demonstrate that Sapienz significantly outperforms both the state-of-the-art technique Dynodroid and the widely-used tool, Android Monkey, on all three objectives. When applied to the top 1,000 Google Play apps, Sapienz found 558 unique, previously unknown crashes. Octopuz reuses the Sapienz multi-objective search approach for automated JavaScript testing, aiming to investigate whether it replicates the Sapienz’ success on JavaScript testing. Experimental results on 10 real-world JavaScript apps provide evidence that Octopuz significantly outperforms the state of the art (and current state of practice) in automated JavaScript testing. Polariz is an approach that combines human (crowd) intelligence with machine (computational search) intelligence for mobile testing. It uses a platform that enables crowdsourced mobile testing from any source of app, via any terminal client, and by any crowd of workers. It generates replicable test scripts based on manual test traces produced by the crowd workforce, and automatically extracts from these test traces, motif events that can be used to improve search-based mobile testing approaches such as Sapienz