Leveraging the Power of Crowds: Automated Test Report Processing for The Maintenance of Mobile Applications

Crowdsourcing is an emerging distributed problem-solving model combining human and machine computation. It collects intelligence and knowledge from a large and diverse workforce to complete complex tasks. In the software engineering domain, crowdsourced techniques have been adopted to facilitate various tasks, such as design, testing, debugging, development, and so on. Specifically, in crowdsourced testing, crowdsourced workers are given testing tasks to perform and submit their feedback in the form of test reports. One of the key advantages of crowdsourced testing is that it is capable of providing engineers software engineers with domain knowledge and feedback from a large number of real users. Based on diverse software and hardware settings of these users, engineers can bugs that are not caught by traditional quality assurance techniques. Such benefits are particularly ideal for mobile application testing, which needs rapid development-and-deployment iterations and support diverse execution environments. However, crowdsourced testing naturally generates an overwhelming number of crowdsourced test reports, and inspecting such a large number of reports becomes a time-consuming yet inevitable task. This dissertation presents a series of techniques, tools and experiments to assist in crowdsourced report processing. These techniques are designed for improving this task in multiple aspects: 1. prioritizing crowdsourced report to assist engineers in finding as many unique bugs as possible, and as quickly as possible; 2. grouping crowdsourced report to assist engineers in identifying the representative ones in a short time; 3. summarizing the duplicate reports to provide engineers with a concise and accurate understanding of a group of reports; In the first step, I present a text-analysis-based technique to prioritize test reports for manual inspection. This technique leverages two key strategies: (1) a diversity strategy to help developers inspect a wide variety of test reports and to avoid duplicates and wasted effort on falsely classified faulty behavior, and (2) a risk-assessment strategy to help developers identify test reports that may be more likely to be fault-revealing based on past observations.Together, these two strategies form our technique to prioritize test reports in crowdsourced testing. Moreover, in the mobile testing domain, test reports often consist of more screenshots and shorter descriptive text, and thus text-analysis-based techniques may be ineffective or inapplicable. The shortage and ambiguity of natural-language text information and the well-defined screenshots of activity views within mobile applications motivate me to propose a novel technique based on using image understanding for multi-objective test-report prioritization. This technique employs the Spatial Pyramid Matching (SPM) technique to measure the similarity of the screenshots, and apply the natural-language processing technique to measure the distance between the text of test reports. Next, I design and implement CTRAS: a novel approach to leveraging duplicates to enrich the content of bug descriptions and improve the efficiency of inspecting these reports. CTRAS is capable of automatically aggregating duplicates based on both textual information and screenshots, and further summarizes the duplicate test reports into a comprehensive and comprehensible report.I validate all of these techniques on industrial data by collaborating with several companies. The results show my techniques can improve both the efficiency and effectiveness of crowdsourced test report processing. Also, I suggest settings for different usage scenarios and discuss future research directions

Taming Android Fragmentation through Lightweight Crowdsourced Testing

Android fragmentation refers to the overwhelming diversity of Android devices and OS versions. These lead to the impossibility of testing an app on every supported device, leaving a number of compatibility bugs scattered in the community and thereby resulting in poor user experiences. To mitigate this, our fellow researchers have designed various works to automatically detect such compatibility issues. However, the current state-of-the-art tools can only be used to detect specific kinds of compatibility issues (i.e., compatibility issues caused by API signature evolution), i.e., many other essential types of compatibility issues are still unrevealed. For example, customized OS versions on real devices and semantic changes of OS could lead to serious compatibility issues, which are non-trivial to be detected statically. To this end, we propose a novel, lightweight, crowdsourced testing approach, LAZYCOW, to fill this research gap and enable the possibility of taming Android fragmentation through crowdsourced efforts. Specifically, crowdsourced testing is an emerging alternative to conventional mobile testing mechanisms that allow developers to test their products on real devices to pinpoint platform-specific issues. Experimental results on thousands of test cases on real-world Android devices show that LAZYCOW is effective in automatically identifying and verifying API-induced compatibility issues. Also, after investigating the user experience through qualitative metrics, users' satisfaction provides strong evidence that LAZYCOW is useful and welcome in practice

Mapping Crisis: Participation, Datafication, and Humanitarianism in the Age of Digital Mapping

This book brings together critical perspectives on the role that mapping people, knowledges and data now plays in humanitarian work, both in cartographic terms and through data visualisations. Since the rise of Google Earth in 2005, there has been an explosion in the use of mapping tools to quantify and assess the needs of the poor, including those affected by climate change and the wider neo-liberal agenda. Yet, while there has been a huge upsurge in the data produced around these issues, the representation of people remains questionable. Some have argued that representation has diminished in humanitarian crises as people are increasingly reduced to data points. In turn, this data becomes ever more difficult to analyse without vast computing power, leading to a dependency on the old colonial powers to refine the data of the poor, before selling it back to them. These issues are not entirely new, and questions around representation, participation and humanitarianism can be traced back beyond the speeches of Truman, but the digital age throws these issues back to the fore, as machine learning, algorithms and big data centres take over the process of mapping the subjugated and subaltern. This book questions whether, as we map crises, it is the map itself that is in crisis

An investigation into the role of crowdsourcing in generating information for flood risk management

Flooding is a major global hazard whose management relies on an accurate understanding of its risks. Crowdsourcing represents a major opportunity for supporting flood risk management as members of the public are highly capable of producing useful flood information. This thesis explores a wide range of issues related to flood crowdsourcing using an interdisciplinary approach. Through an examination of 31 different projects a flood crowdsourcing typology was developed. This identified five key types of flood crowdsourcing: i) Incident Reporting, ii) Media Engagement, iii) Collaborative Mapping, iv) Online Volunteering and v) Passive VGI. These represent a wide range of initiatives with radically different aims, objectives, datasets and relationships with volunteers. Online Volunteering was explored in greater detail using Tomnod as a case study. This is a micro-tasking platform in which volunteers analyse satellite imagery to support disaster response. Volunteer motivations for participating on Tomnod were found to be largely altruistic. Demographics of participants were significant, with retirement, disability or long-term health problems identified as major drivers for participation. Many participants emphasised that effective communication between volunteers and the site owner is strongly linked to their appreciation of the platform. In addition, the feedback on the quality and impact of their contributions was found to be crucial in maintaining interest. Through an examination of their contributions, volunteers were found to be able to ascertain with a higher degree of accuracy, many features in satellite imagery which supervised image classification struggled to identify. This was more pronounced in poorer quality imagery where image classification had a very low accuracy. However, supervised classification was found to be far more systematic and succeeded in identifying impacts in many regions which were missed by volunteers. The efficacy of using crowdsourcing for flood risk management was explored further through the iterative development of a Collaborative Mapping web-platform called Floodcrowd. Through interviews and focus groups, stakeholders from the public and private sector expressed an interest in crowdsourcing as a tool for supporting flood risk management. Types of data which stakeholders are particularly interested in with regards to crowdsourcing differ between organisations. Yet, they typically include flood depths, photos, timeframes of events and historical background information. Through engagement activities, many citizens were found to be able and motivated to share such observations. Yet, motivations were strongly affected by the level of attention their contributions receive from authorities. This presents many opportunities as well as challenges for ensuring that the future of flood crowdsourcing improves flood risk management and does not damage stakeholder relationships with participants

Mapping Crisis

The digital age has thrown questions of representation, participation and humanitarianism back to the fore, as machine learning, algorithms and big data centres take over the process of mapping the subjugated and subaltern. Since the rise of Google Earth in 2005, there has been an explosion in the use of mapping tools to quantify and assess the needs of those in crisis, including those affected by climate change and the wider neo-liberal agenda. Yet, while there has been a huge upsurge in the data produced around these issues, the representation of people remains questionable. Some have argued that representation has diminished in humanitarian crises as people are increasingly reduced to data points. In turn, this data has become ever more difficult to analyse without vast computing power, leading to a dependency on the old colonial powers to refine the data collected from people in crisis, before selling it back to them. This book brings together critical perspectives on the role that mapping people, knowledges and data now plays in humanitarian work, both in cartographic terms and through data visualisations, and questions whether, as we map crises, it is the map itself that is in crisis

Thesis title: Crowdsourced Testing Approach For Mobile Compatibility Testing

The frequent release of mobile devices and operating system versions bring several compatibility issues to mobile applications. This thesis addresses fragmentation-induced compatibility issues. The thesis comprises three main phases. The first of these involves an in-depth review of relevant literature that identifies the main challenges of existing compatibility testing approaches. The second phase reflects on the conduction of an in-depth exploratory study on Android/iOS developers in academia and industry to gain further insight into their actual needs in testing environments whilst gauging their willingness to work with public testers with varied experience. The third phase relates to implementing a new manual crowdtesting approach that supports large-scale distribution of tests and execution by public testers and real users on a larger number of devices in a short time. The approach is designed based on a direct crowdtesting workflow to bridge the communication gap between developers and testers. The approach supports performing the three dimensions of compatibility testing. This approach helps explore different behaviours of the app and the users of the app to identify all compatibility issues. Two empirical evaluation studies were conducted on iOS/Android developers and testers to gauge developers' and testers' perspectives regarding the benefits, satisfaction, and effectiveness of the proposed approach. Our findings show that the approach is effective and improves on current state-of-the-art approaches. The findings also show that the approach met the several unmet needs of different groups of developers and testers. The evaluation proved that the different groups of developers and testers were satisfied with the approach. Importantly, the level of satisfaction was especially high in small and medium-sized enterprises that have limited access to traditional testing infrastructures, which are instead present in large enterprises. This is the first research that provides insights for future research into the actual needs of each group of developers and testers

Microwork: Theory, Models and Mechanics for enabling impact through aggregate action

This major research project will focus on the primary investigation area of microwork. Several sub-areas of inquiry will be visited in order to explore potential new directions, determine and suggest factors potentially maximizing impact via microwork projects, including historical examples of analog micro-tasks and their possible correlations to both existing and future digital microwork; the mechanized design elements for executing microwork projects, including drivers, challenges and opportunities, and ultimately the potential for future impacts via microwork, on individual and collective levels, with focus on increasing social impact, and volumes of action. In turn, this combined understanding will suggest the formation of a new microwork model, as well as a business model canvas for evaluation, by helping to suggest the theoretical and physical components required for success, such as new socially-based drivers, tools, mechanics, success metrics, and processes

Comunicação mutimédia de inovação de ciência e tecnologia em contexto empresarial

The need to communicate science and technology is increasing. In Research and Development (R&D) projects, which encompass the creation of novel technologies and paradigms, the adequate dissemination of representative content plays a pivotal role to establish such technologies within their target audiences, including the scientific community and project partners. Within the scope of this dissertation, multimedia artifacts were produced for the communication of complex scientific and technological content. This was made in the context of the MOG Technologies company, and its need to communicate with partners, clients, and other stakeholders, the characteristics of their innovation-based services and products. The produced multimedia artifacts were videos and broadcast events. A social media dissemination plan was also created and implemented. The challenge of these concrete outputs was supported by the knowledge and understanding of the MOG projects, products, and target audiences, and by the theoretical framework of science and technology communication using multimedia. The artifacts were implemented and used, obtaining good feedback.A necessidade de comunicar ciência e tecnologia é cada vez maior. Em projetos de Investigação e Desenvolvimento, que incluem a criação de novas tecnologias e paradigmas, a disseminação adequada de conteúdos representativos desempenha um papel fundamental para estabelecer essas tecnologias junto dos seus públicos-alvo, incluindo a comunidade científica e parceiros de projeto. No âmbito desta dissertação, foram produzidos artefactos multimédia para a comunicação de conteúdos científicos e tecnológicos complexos. Isto foi feito no contexto da empresa MOG Technologies, e partindo da sua necessidade de comunicar com parceiros, clientes e outros intervenientes, as características dos seus serviços e produtos baseados em inovação. Os artefactos multimédia produzidos foram vídeos e eventos de broadcasting. Foi também criado e implementado um plano de disseminação nas redes sociais. O desafio de obter estes resultados concretos é apoiado pelo conhecimento e compreensão dos projetos, produtos e públicos-alvo da MOG e pelo enquadramento teórico de comunicação de ciência e tecnologia usando multimédia. Os artefactos foram implementados e utilizados, obtendo-se um bom feedback.Mestrado em Comunicação Multimédi