Active User Authentication for Smartphones: A Challenge Data Set and Benchmark Results

In this paper, automated user verification techniques for smartphones are investigated. A unique non-commercial dataset, the University of Maryland Active Authentication Dataset 02 (UMDAA-02) for multi-modal user authentication research is introduced. This paper focuses on three sensors - front camera, touch sensor and location service while providing a general description for other modalities. Benchmark results for face detection, face verification, touch-based user identification and location-based next-place prediction are presented, which indicate that more robust methods fine-tuned to the mobile platform are needed to achieve satisfactory verification accuracy. The dataset will be made available to the research community for promoting additional research.Comment: 8 pages, 12 figures, 6 tables. Best poster award at BTAS 201

Challenges in context-aware mobile language learning: the MASELTOV approach

Smartphones, as highly portable networked computing devices with embedded sensors including GPS receivers, are ideal platforms to support context-aware language learning. They can enable learning when the user is en-gaged in everyday activities while out and about, complementing formal language classes. A significant challenge, however, has been the practical implementation of services that can accurately identify and make use of context, particularly location, to offer meaningful language learning recommendations to users. In this paper we review a range of approaches to identifying context to support mobile language learning. We consider how dynamically changing aspects of context may influence the quality of recommendations presented to a user. We introduce the MASELTOV project’s use of context awareness combined with a rules-based recommendation engine to present suitable learning content to recent immigrants in urban areas; a group that may benefit from contextual support and can use the city as a learning environment

Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely; “sensing”, “analysis”, and “application”. Sensing investigates the different sensing modalities that are used in existing real-time affective applications, Analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and Application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing

Smart scientific instruments based on smartphones: a brief review

Smartphone has gone beyond a communication hub to be a measurement device itself, thanks to various built-in sensors. This article reviewed achievements in transforming ubiquitous smartphones into cost-effective scientific instruments for educational laboratories, environmental studies, point-of-care diagnostics, home-based health monitoring, and rehabilitation. Magnetic fields were precisely measured by built-in magnetometers, leading to demonstrations for engineering and medical applications. The smartphone-based joint-angle measurement was a viable alternative to traditional goniometers. Characterizations of optical signals captured by cameras led to portable spectrophotometers and colorimeters for both educational and practical uses. Interestingly, smartphones became a platform for high-resolution microscopes and fluorescence microscopes were developed with add-on components. These smart instruments become even more attractive options in the pandemic period with limited facility and laboratory access

On the security of mobile sensors

PhD ThesisThe age of sensor technology is upon us. Sensor-rich mobile devices are ubiquitous. Smart-phones, tablets, and wearables are increasingly equipped with sensors such as GPS, accelerometer, Near Field Communication (NFC), and ambient sensors. Data provided by such sensors, combined with the fast-growing computational capabilities on mobile platforms, offer richer and more personalised apps. However, these sensors introduce new security challenges to the users, and make sensor management more complicated. In this PhD thesis, we contribute to the field of mobile sensor security by investigating a wide spectrum of open problems in this field covering attacks and defences, standardisation and industrial approaches, and human dimensions. We study the problems in detail and propose solutions. First, we propose “Tap-Tap and Pay” (TTP), a sensor-based protocol to prevent the Mafia attack in NFC payment. The Mafia attack is a special type of Man-In-The-Middle attack which charges the user for something more expensive than what she intends to pay by relaying transactions to a remote payment terminal. In TTP, a user initiates the payment by physically tapping her mobile phone against the reader. We observe that this tapping causes transient vibrations at both devices which are measurable by the embedded accelerometers. Our observations indicate that these sensor measurements are closely correlated within the same tapping, and different if obtained from different tapping events. By comparing the similarity between the two measurements, the bank can distinguish the Mafia fraud apart from a legitimate NFC transaction. The experimental results and the user feedback suggest the practical feasibility of TTP. As compared with previous sensor-based solutions, ours is the only one that works even when the attacker and the user are in nearby locations or share similar ambient environments. Second, we demonstrate an in-app attack based on a real world problem in contactless payment known as the card collision or card clash. A card collision happens when more than one card (or NFC-enabled device) are presented to the payment terminal’s field, and the terminal does not know which card to choose. By performing experiments, we observe that the implementation of contactless terminals in practice matches neither EMV nor ISO standards (the two primary standards for smart card payment) on card collision. Based on this inconsistency, we propose “NFC Payment Spy”, a malicious app that tracks the user’s contactless payment transactions. This app, running on a smart phone, simulates a card which requests the payment information (amount, time, etc.) from the terminal. When the phone and the card are both presented to a contactless terminal (given that many people use mobile case wallets to travel light and keep wallet essentials close to hand), our app can effectively win the race condition over the card. This attack is the first privacy attack on contactless payments based on the problem of card collision. By showing the feasibility of this attack, we raise awareness of privacy and security issues in contactless payment protocols and implementation, specifically in the presence of new technologies for payment such as mobile platforms. Third, we show that, apart from attacking mobile devices by having access to the sensors through native apps, we can also perform sensor-based attacks via mobile browsers. We examine multiple browsers on Android and iOS platforms and study their policies in granting permissions to JavaScript code with respect to access to motion and orientation sensor data. Based on our observations, we identify multiple vulnerabilities, and propose “TouchSignatures” and “PINLogger.js”, two novel attacks in which malicious JavaScript code listens to such sensor data measurements. We demonstrate that, despite the much lower sampling rate (comparing to a native app), a remote attacker is able to learn sensitive user information such as physical activities, phone call timing, touch actions (tap, scroll, hold, zoom), and PINs based on these sensor data. This is the first report of such a JavaScript-based attack. We disclosed the above vulnerability to the community and major mobile browser vendors classified the problem as high-risk and fixed it accordingly. Finally, we investigate human dimensions in the problem of sensor management. Although different types of attacks via sensors have been known for many years, the problem of data leakage caused by sensors has remained unsolved. While working with W3C and browser vendors to fix the identified problem, we came to appreciate the complexity of this problem in practice and the challenge of balancing security, usability, and functionality. We believe a major reason for this is that users are not fully aware of these sensors and the associated risks to their privacy and security. Therefore, we study user understanding of mobile sensors, specifically their risk perceptions. This is the only research to date that studies risk perceptions for a comprehensive list of mobile sensors (25 in total). We interview multiple participants from a range of backgrounds by providing them with multiple self-declared questionnaires. The results indicate that people in general do not have a good understanding of the complexities of these sensors; hence making security judgements about these sensors is not easy for them. We discuss how this observation, along with other factors, renders many academic and industry solutions ineffective. This makes the security and privacy issues of mobile sensors and other sensorenabled technologies an important topic to be investigated further

Anticipatory Mobile Computing: A Survey of the State of the Art and Research Challenges

Today's mobile phones are far from mere communication devices they were ten years ago. Equipped with sophisticated sensors and advanced computing hardware, phones can be used to infer users' location, activity, social setting and more. As devices become increasingly intelligent, their capabilities evolve beyond inferring context to predicting it, and then reasoning and acting upon the predicted context. This article provides an overview of the current state of the art in mobile sensing and context prediction paving the way for full-fledged anticipatory mobile computing. We present a survey of phenomena that mobile phones can infer and predict, and offer a description of machine learning techniques used for such predictions. We then discuss proactive decision making and decision delivery via the user-device feedback loop. Finally, we discuss the challenges and opportunities of anticipatory mobile computing.Comment: 29 pages, 5 figure

myStress: Unobtrusive Smartphone-Based Stress Detection

Life is becoming increasingly stressful in many aspects, e.g., due to technology-induced stress and stress in organizational context. The assessment of stress experienced by individuals enables stress management and prevention with the long-term aim to avoid psychological and physiological harm from excessive stress. Commonly this assessment is performed through questionnaires on perceived stress or physiological measurements evaluating body reactions to stress. We explore a third assessment method: Our design science approach aims to unobtrusively assess perceived stress based on smartphone data while waiving additional devices and explicit user input. The presented design artefact, myStress, reads 36 hardware and software sensors to infer users’ perceived stress levels. A prototypical instantiation of myStress for the Android platform is distributed to test users. For evaluation purposes, the stress level additionally is determined by a questionnaire consisting of the Perceived Stress Scale. By analyzing data from test users, we gain first insights into the feasibility of unobtrusive, continuous stress assessment considering exclusively data from smartphone sensors. We find that several sensors seem to correlate with perceived stress, e.g., the frequency of switching the display on/off. For future research, behavioral and situational prevention measures can build on this method of unobtrusive stress assessment

Detecting Social Interactions in Working Environments Through Sensing Technologies

The knowledge about social ties among humans is important to optimize several aspects concerning networking in mobile social networks. Generally, ties among people are detected on the base of proximity of people. We discuss here how ties concerning colleagues in an office can be detected by leveraging on a number of sociological markers like co-activity, proximity, speech activity and similarity of locations visited. We present the results from two data gathering campaigns located in Italy and Spain.Ministerio de Economía y Competitividad TIN2013-46801-C4-1-RJunta de Andalucía TIC-805