Engineering Crowdsourced Stream Processing Systems

A crowdsourced stream processing system (CSP) is a system that incorporates crowdsourced tasks in the processing of a data stream. This can be seen as enabling crowdsourcing work to be applied on a sample of large-scale data at high speed, or equivalently, enabling stream processing to employ human intelligence. It also leads to a substantial expansion of the capabilities of data processing systems. Engineering a CSP system requires the combination of human and machine computation elements. From a general systems theory perspective, this means taking into account inherited as well as emerging properties from both these elements. In this paper, we position CSP systems within a broader taxonomy, outline a series of design principles and evaluation metrics, present an extensible framework for their design, and describe several design patterns. We showcase the capabilities of CSP systems by performing a case study that applies our proposed framework to the design and analysis of a real system (AIDR) that classifies social media messages during time-critical crisis events. Results show that compared to a pure stream processing system, AIDR can achieve a higher data classification accuracy, while compared to a pure crowdsourcing solution, the system makes better use of human workers by requiring much less manual work effort

Build an app and they will come? Lessons learnt from trialling the GetThereBus app in rural communities

Acknowledgements The research described here was supported by the award made by the RCUK Digital Economy programme to the dot.rural Digital Economy Hub; award reference: EP/G066051/1.Peer reviewedPostprin

Given Enough Eyeballs, all Bugs are Shallow - A Literature Review for the Use of Crowdsourcing in Software Testing

Over the last years, the use of crowdsourcing has gained a lot of attention in the domain of software engineering. One key aspect of software development is the testing of software. Literature suggests that crowdsourced software testing (CST) is a reliable and feasible tool for manifold kinds of testing. Research in CST made great strides; however, it is mostly unstructured and not linked to traditional software testing practice and terminology. By conducting a literature review of traditional and crowdsourced software testing literature, this paper delivers two major contributions. First, it synthesizes the fields of crowdsourcing research and traditional software testing. Second, the paper gives a comprehensive overview over findings in CST-research and provides a classification into different software testing types

Quality of Information in Mobile Crowdsensing: Survey and Research Challenges

Smartphones have become the most pervasive devices in people's lives, and are clearly transforming the way we live and perceive technology. Today's smartphones benefit from almost ubiquitous Internet connectivity and come equipped with a plethora of inexpensive yet powerful embedded sensors, such as accelerometer, gyroscope, microphone, and camera. This unique combination has enabled revolutionary applications based on the mobile crowdsensing paradigm, such as real-time road traffic monitoring, air and noise pollution, crime control, and wildlife monitoring, just to name a few. Differently from prior sensing paradigms, humans are now the primary actors of the sensing process, since they become fundamental in retrieving reliable and up-to-date information about the event being monitored. As humans may behave unreliably or maliciously, assessing and guaranteeing Quality of Information (QoI) becomes more important than ever. In this paper, we provide a new framework for defining and enforcing the QoI in mobile crowdsensing, and analyze in depth the current state-of-the-art on the topic. We also outline novel research challenges, along with possible directions of future work.Comment: To appear in ACM Transactions on Sensor Networks (TOSN

Translating Video Recordings of Mobile App Usages into Replayable Scenarios

Screen recordings of mobile applications are easy to obtain and capture a wealth of information pertinent to software developers (e.g., bugs or feature requests), making them a popular mechanism for crowdsourced app feedback. Thus, these videos are becoming a common artifact that developers must manage. In light of unique mobile development constraints, including swift release cycles and rapidly evolving platforms, automated techniques for analyzing all types of rich software artifacts provide benefit to mobile developers. Unfortunately, automatically analyzing screen recordings presents serious challenges, due to their graphical nature, compared to other types of (textual) artifacts. To address these challenges, this paper introduces V2S, a lightweight, automated approach for translating video recordings of Android app usages into replayable scenarios. V2S is based primarily on computer vision techniques and adapts recent solutions for object detection and image classification to detect and classify user actions captured in a video, and convert these into a replayable test scenario. We performed an extensive evaluation of V2S involving 175 videos depicting 3,534 GUI-based actions collected from users exercising features and reproducing bugs from over 80 popular Android apps. Our results illustrate that V2S can accurately replay scenarios from screen recordings, and is capable of reproducing $\approx$ 89% of our collected videos with minimal overhead. A case study with three industrial partners illustrates the potential usefulness of V2S from the viewpoint of developers.Comment: In proceedings of the 42nd International Conference on Software Engineering (ICSE'20), 13 page

Crowdsourcing traffic data for travel time estimation

Travel time estimation is a fundamental measure used in routing and navigation applications, in particular in emerging intelligent transportation systems (ITS). For example, many users may prefer the fastest route to their destination and would rely on real-time predicted travel times. It also helps real-time traffic management and traffic light control. Accurate estimation of travel time requires collecting a lot of real-time data from road networks. This data can be collected using a wide variety of sources like inductive loop detectors, video cameras, radio frequency identification (RFID) transponders etc. But these systems include deployment of infrastructure which has some limitations and drawbacks. The main drawbacks in these modes are the high cost and the high probability of error caused by prevalence of equipment malfunctions and in the case of sensor based methods, the problem of spatial coverage.;As an alternative to traditional way of collecting data using expensive equipment, development of cellular & mobile technology allows for leveraging embedded GPS sensors in smartphones carried by millions of road users. Crowd-sourcing GPS data will allow building traffic monitoring systems that utilize this opportunity for the purpose of accurate and real-time prediction of traffic measures. However, the effectiveness of these systems have not yet been proven or shown in real applications. In this thesis, we study some of the current available data sets and identify the requirements for accurate prediction. In our work, we propose the design for a crowd-sourcing traffic application, including an android-based mobile client and a server architecture. We also develop map-matching method. More importantly, we present prediction methods using machine learning techniques such as support vector regression.;Machine learning provides an alternative to traditional statistical method such as using averaged historic data for estimation of travel time. Machine Learning techniques played a key role in estimation in the last two decades. They are proved by providing better accuracy in estimation and in classification. However, employing a machine learning technique in any application requires creative modeling of the system and its sensory data. In this thesis, we model the road network as a graph and train different models for different links on the road. Modeling a road network as graph with nodes and links enables the learner to capture patterns occurring on each segment of road, thereby providing better accuracy. To evaluate the prediction models, we use three sets of data out of which two sets are collected using mobile probing and one set is generated using VISSIM traffic simulator. The results show that crowdsourcing is only more accurate than traditional statistical methods if the input values for input data are very close to the actual values. In particular, when speed of vehicles on a link are concerned, we need to provide the machine learning model with data that is only few minutes old; using average speed of vehicles, for example from the past half hour, as is usually seen in many web based traffic information sources may not allow for better performance