Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely; “sensing”, “analysis”, and “application”. Sensing investigates the different sensing modalities that are used in existing real-time affective applications, Analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and Application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing

PowerForecaster: Predicting Smartphone Power Impact of Continuous Sensing Applications at Pre-installation Time

Today's smartphone application (hereinafter 'app') markets miss a key piece of information, power consumption of apps. This causes a severe problem for continuous sensing apps as they consume significant power without users' awareness. Users have no choice but to repeatedly install one app after another and experience their power use. To break such an exhaustive cycle, we propose PowerForecaster, a system that provides users with power use of sensing apps at pre-installation time. Such advanced power estimation is extremely challenging since the power cost of a sensing app largely varies with users' physical activities and phone use patterns. We observe that the time for active sensing and processing of an app can vary up to three times with 27 people's sensor traces collected over three weeks. PowerForecaster adopts a novel power emulator that emulates the power use of a sensing app while reproducing users' physical activities and phone use patterns, achieving accurate, personalized power estimation. Our experiments with three commercial apps and two research prototypes show that PowerForecaster achieves 93.4% accuracy under 20 use cases. Also, we optimize the system to accelerate emulation speed and reduce overheads, and show the effectiveness of such optimization techniques.

Design and implementation of an indoor modeling method through crowdsensing

While automatic modeling and mapping of outdoor environments is well-established, the indoor equivalent of automated generation of building floor plans poses a challenge. In fact, outdoor localization is commonly available and inexpensive through the existing satellite positioning systems, such as GPS and Galileo. However, these technologies are not applicable in indoor environments, since a direct line of sight to the satellites, orbiting the globes, is required. As a substitution, the technical literature comprises several proposals for the development of simultaneous indoor localization and mapping (SLAM). In these approaches, the authors mostly exploit indoor resources such as the WiFi access points and the mobile smart devices carried by individuals in the indoor environment. Collecting data from several mobile devices is referred to as crowdsensing. To enable the generation of two-dimensional (2D) as well as three-dimensional (3D) maps, we propose crowdsensing of point clouds, which are 3D data structures of points in space. For localization, we integrate two features of a recently developed mobile device, called Project Tango. Specifically, the Tango platform provides two main technologies for reliable localization, namely motion tracking and area learning. Moreover, Tango-powered devices provide us with the ability to collect point clouds though a third technology, called depth perception. In the past few years, spatial data obtained from range imaging was used to generate indoor maps. Nevertheless, range images are expensive and not always available. The required equipment, e.g. laser range scanners, are both expensive in procurement and require trained personnel for proper setup and operation. In this thesis, we aim for obtaining spatial point clouds via crowdsensing. The main idea is to use sensor data which can be scanned by volunteering individuals using easy to handle mobile devices. Specifically, we depend on depth perception capabilities as provided by Google Tango-powered tablet computers. A crowdsensing infrastructure assigns scanning tasks to individuals carrying a Tango device. Execution of such a task consists of taking scans of e.g. offices in a public building. The scanning results contain both spatial information about the room layout and its position. Energy consumption on the mobile device is reduced by applying Octree compression to the scanned point clouds, which results in a significant reduction of the amount of data, which has to be transferred to a back-end server. Afterwards, the back-end is responsible for assembling the received scans and the extraction of an indoors model. The modeling process - developed in this thesis - comprises two-phases. First, we extract a basic model from the obtained point clouds, which may contain outliers, inaccuracies and gaps. In the second phase, we refine the model by exploiting formal grammars. It is worth to mention here that we are the first to exploit formal grammars as a model fitting tool. We feed the information obtained in the first phase to an indoors grammar, which has been developed in the ComNSense project, University of Stuttgart. The resultant model both contains much less deviations from the ground truth and provides improved robustness against aberrations with respect to localization during the scanning process. Thus, instead of scanning multiple point clouds per room, we need only one scan to be able to construct an indoor map. During evaluation of this process, using scans of offices of our department, we were able to reproduce a model which is very close to the ground truth

Improving Energy Efficiency and Security for Pervasive Computing Systems

Pervasive computing systems are comprised of various personal mobile devices connected by the wireless networks. Pervasive computing systems have gained soaring popularity because of the rapid proliferation of the personal mobile devices. The number of personal mobile devices increased steeply over years and will surpass world population by 2016.;However, the fast development of pervasive computing systems is facing two critical issues, energy efficiency and security assurance. Power consumption of personal mobile devices keeps increasing while the battery capacity has been hardly improved over years. at the same time, a lot of private information is stored on and transmitted from personal mobile devices, which are operating in very risky environment. as such, these devices became favorite targets of malicious attacks. Without proper solutions to address these two challenging problems, concerns will keep rising and slow down the advancement of pervasive computing systems.;We select smartphones as the representative devices in our energy study because they are popular in pervasive computing systems and their energy problem concerns users the most in comparison with other devices. We start with the analysis of the power usage pattern of internal system activities, and then identify energy bugs for improving energy efficiency. We also investigate into the external communication methods employed on smartphones, such as cellular networks and wireless LANs, to reduce energy overhead on transmissions.;As to security, we focus on implantable medical devices (IMDs) that are specialized for medical purposes. Malicious attacks on IMDs may lead to serious damages both in the cyber and physical worlds. Unlike smartphones, simply borrowing existing security solutions does not work on IMDs because of their limited resources and high requirement of accessibility. Thus, we introduce an external device to serve as the security proxy for IMDs and ensure that IMDs remain accessible to save patients\u27 lives in certain emergency situations when security credentials are not available

A Framework for Generic and Energy Efficient Context Recognition for Personal Mobile Devices

The advancements in the field of mobile computing over the last decade have enabled the scientific community to expedite the theoretical and experimental work to achieve the vision of ubiquitous computing. As ubiquitous computing aims to provide seamless and distraction free task support to its users, one of the essential pieces of information required by the ubiquitous computing systems to do so is the context of its users. Context of a user can be defined as the information that describes the task the user is performing and the environment in which the user is currently present. Among various platforms that are commonly used to determine user's context, the personal mobile devices like smart phones stand out as one of the most widely used and widely evaluated ones. However, despite numerous advantages that are provided by modern day personal mobile devices, such as high computational and communication capabilities, variety of on-board sensors to capture raw data related to user's motion and environment, high resolution displays to enable interaction with other services and systems, these devices suffer from limited battery resources. In contrast to the advancements in other domains, the advancements in the battery domain have not been up to the mark. Consequently, the context recognition applications developed for these devices suffer from the trade-off between achieving accuracy and longevity of other device's basic operations. As a result, most of the existing context recognition applications for these devices are fine tuned for specific context types and thereby lacks generality. The situation gets worse when a number of context recognition applications are executed simultaneously, thus competing for limited resources and consuming the device's battery additively. To address the aforementioned issues, this thesis provides a generic and energy efficient context recognition framework for personal mobile devices. The main contribution consists of a generic framework to support development of context recognition applications supported by algorithms to achieve their energy efficient execution. The proposed framework consists of two systems namely the component system and the activation system. The component system allows developers to create context recognition applications using a component abstraction. This enables runtime analysis of applications' structures to adopt our novel energy efficiency mechanism. The activation system uses a state machine abstraction to allow context dependent activation of context recognition configurations pertaining to relevant user's tasks such that only needed configurations are executed to determine only the relevant context characteristics, thereby enabling energy efficiency. The activation system also provides generic applicability of four different energy efficiency techniques, already used in different existing systems but mostly for specific context characteristics. To aid rapid prototyping, both systems are equipped with off-line development tools. The tools include graphical editors and a component tool-kit. The graphical editors allow developers to create component configurations used by the component system and state machines used by the activation system. These editors enable developers to create component configurations and state machines by simply dragging, dropping and connecting different models used in our component and state machine abstractions. These tools also provide validation and code generation utilities. In addition to the graphical editor, the framework provides a component tool-kit which consists of a number of already implemented sensing, preprocessing and classification components which can be re-used in new applications. In order to provide the energy efficient execution of context recognition applications, the thesis introduces a novel energy efficiency technique called configuration folding. Configuration folding analyses structures of simultaneously executing context recognition applications to identify redundant functionalities between them and as an output produces a single redundancy free context recognition configuration which holds the structural integrity of all applications. Consequently, the overall energy expenditure is reduced compared to the original expenditure when redundant functionalities are not removed. The experimental evaluation of configuration folding on test applications shows energy savings between 13 and 48 %. The thesis also investigates optimization possibilities in configuration folding in case the redundant functionalities between the applications differ in parametrization. Towards this end, the thesis identifies commonly used parameters in context recognition applications and defines relations between them. Finally, an extended version of configuration folding is introduced to handle the differences in parametrization. The evaluation of the extended version of configuration folding on test scenarios shows energy saving of up to 45%. The contributions in this thesis have been evaluated extensively. The framework has been used in number of European Commission (EC) projects and in student projects and theses at the University of Duisburg-Essen, Germany. Using the component system and the activation system, a number of applications have been developed in those projects. Some of these applications include crowd density estimation in buses, bus ride detections, navigation application for buses in Madrid, user movement detection, user localization, fall detection application etc. Moreover, the component system, the activation system and the configuration folding technique have been published in different prestigious conferences and workshops