Automated annotation of landmark images using community contributed datasets and web resources

A novel solution to the challenge of automatic image annotation is described. Given an image with GPS data of its location of capture, our system returns a semantically-rich annotation comprising tags which both identify the landmark in the image, and provide an interesting fact about it, e.g. "A view of the Eiffel Tower, which was built in 1889 for an international exhibition in Paris". This exploits visual and textual web mining in combination with content-based image analysis and natural language processing. In the first stage, an input image is matched to a set of community contributed images (with keyword tags) on the basis of its GPS information and image classification techniques. The depicted landmark is inferred from the keyword tags for the matched set. The system then takes advantage of the information written about landmarks available on the web at large to extract a fact about the landmark in the image. We report component evaluation results from an implementation of our solution on a mobile device. Image localisation and matching oers 93.6% classication accuracy; the selection of appropriate tags for use in annotation performs well (F1M of 0.59), and it subsequently automatically identies a correct toponym for use in captioning and fact extraction in 69.0% of the tested cases; finally the fact extraction returns an interesting caption in 78% of cases

A Cost-Quality Beneficial Cell Selection Approach for Sparse Mobile Crowdsensing with Diverse Sensing Costs

The Internet of Things (IoT) and mobile techniques enable real-time sensing for urban computing systems. By recruiting only a small number of users to sense data from selected subareas (namely cells), Sparse Mobile Crowdsensing (MCS) emerges as an effective paradigm to reduce sensing costs for monitoring the overall status of a large-scale area. The current Sparse MCS solutions reduce the sensing subareas (by selecting the most informative cells) based on the assumption that each sample has the same cost, which is not always realistic in real-world, as the cost of sensing in a subarea can be diverse due to many factors, e.g. condition of the device, location, and routing distance. To address this issue, we proposed a new cell selection approach consisting of three steps (information modeling, cost estimation, and cost-quality beneficial cell selection) to further reduce the total costs and improve the task quality. Specifically, we discussed the properties of the optimization goals and modeled the cell selection problem as a solvable bi-objective optimization problem under certain assumptions and approximation. Then, we presented two selection strategies, i.e. Pareto Optimization Selection (POS) and Generalized Cost-Benefit Greedy (GCB-GREEDY) Selection along with our proposed cell selection algorithm. Finally, the superiority of our cell selection approach is assessed through four real-life urban monitoring datasets (Parking, Flow, Traffic, and Humidity) and three cost maps (i.i.d with dynamic cost map, monotonic with dynamic cost map and spatial correlated cost map). Results show that our proposed selection strategies POS and GCB-GREEDY can save up to 15.2% and 15.02% sample costs and reduce the inference errors to a maximum of 16.8% (15.5%) compared to the baseline-Query by Committee (QBC) in a sensing cycle. The findings show important implications in Sparse Mobile Crowdsensing for urban context properties

Supporting Device Discovery and Spontaneous Interaction with Spatial References

The RELATE interaction model is designed to support spontaneous interaction of mobile users with devices and services in their environment. The model is based on spatial references that capture the spatial relationship of a user’s device with other co-located devices. Spatial references are obtained by relative position sensing and integrated in the mobile user interface to spatially visualize the arrangement of discovered devices, and to provide direct access for interaction across devices. In this paper we discuss two prototype systems demonstrating the utility of the model in collaborative and mobile settings, and present a study on usability of spatial list and map representations for device selection

Nomadic input on mobile devices: the influence of touch input technique and walking speed on performance and offset modeling

In everyday life people use their mobile phones on-the-go with different walking speeds and with different touch input techniques. Unfortunately, much of the published research in mobile interaction does not quantify the influence of these variables. In this paper, we analyze the influence of walking speed, gait pattern and input techniques on commonly used performance parameters like error rate, accuracy and tapping speed, and we compare the results to the static condition. We examine the influence of these factors on the machine learned offset model used to correct user input and we make design recommendations. The results show that all performance parameters degraded when the subject started to move, for all input techniques. Index finger pointing techniques demonstrated overall better performance compared to thumb-pointing techniques. The influence of gait phase on tap event likelihood and accuracy was demonstrated for all input techniques and all walking speeds. Finally, it was shown that the offset model built on static data did not perform as well as models inferred from dynamic data, which indicates the speed-specific nature of the models. Also, models identified using specific input techniques did not perform well when tested in other conditions, demonstrating the limited validity of offset models to a particular input technique. The model was therefore calibrated using data recorded with the appropriate input technique, at 75% of preferred walking speed, which is the speed to which users spontaneously slow down when they use a mobile device and which presents a tradeoff between accuracy and usability. This led to an increase in accuracy compared to models built on static data. The error rate was reduced between 0.05% and 5.3% for landscape-based methods and between 5.3% and 11.9% for portrait-based methods

Sensing and visualizing spatial relations of mobile devices

Location information can be used to enhance interaction with mobile devices. While many location systems require instrumentation of the environment, we present a system that allows devices to measure their spatial relations in a true peer-to-peer fashion. The system is based on custom sensor hardware implemented as USB dongle, and computes spatial relations in real-time. In extension of this system we propose a set of spatialized widgets for incorporation of spatial relations in the user interface. The use of these widgets is illustrated in a number of applications, showing how spatial relations can be employed to support and streamline interaction with mobile devices

ReCon: Revealing and Controlling PII Leaks in Mobile Network Traffic

It is well known that apps running on mobile devices extensively track and leak users' personally identifiable information (PII); however, these users have little visibility into PII leaked through the network traffic generated by their devices, and have poor control over how, when and where that traffic is sent and handled by third parties. In this paper, we present the design, implementation, and evaluation of ReCon: a cross-platform system that reveals PII leaks and gives users control over them without requiring any special privileges or custom OSes. ReCon leverages machine learning to reveal potential PII leaks by inspecting network traffic, and provides a visualization tool to empower users with the ability to control these leaks via blocking or substitution of PII. We evaluate ReCon's effectiveness with measurements from controlled experiments using leaks from the 100 most popular iOS, Android, and Windows Phone apps, and via an IRB-approved user study with 92 participants. We show that ReCon is accurate, efficient, and identifies a wider range of PII than previous approaches.Comment: Please use MobiSys version when referencing this work: http://dl.acm.org/citation.cfm?id=2906392. 18 pages, recon.meddle.mob