A mobile sensing solution for indoor and outdoor state detection

Abstract. One important research challenge in ubiquitous computing is determining a device’s indoor/outdoor environmental state. Particularly with modern smartphones, environmental information is important for enabling of new types of services and optimizing already existing functionalities. This thesis presents a tool for Android-powered smartphones called ContextIO for detecting the device’s indoor/outdoor state by combining different onboard sensors of the device itself. To develop ContextIO, we developed a plugin to AWARE mobile sensing framework. Together the plugin and its separate controller component collect rich environmental sensor data. The data analysis and ContextIO’s design considers collected data particularly about magnetometer, ambient light and GSM cellular signal strength. We manually derive thresholds in the data that can be used in combination to infer whether a device is indoor or outdoor. ContextIO uses the same thresholds to infer the state in real time. This thesis contributes an Android tool for inferring the device’s indoor/outdoor status, an open dataset that other researchers can use in their work and an analysis of the collected sensor data for environmental indoor/outdoor state detection.Tiivistelmä. Yksi jokapaikan tietotekniikan tutkimuskysymyksistä keskittyy selvittämään onko laitteen sijainti sisä- vai ulkotilassa. Etenkin uudet älypuhelimet pystyvät hyödyntämään tätä tietoa uudenlaisten palveluiden ja sovellusten kehittämisessä sekä vanhojen toiminnallisuuksien optimoinnissa. Tämä diplomityö esittelee Android-käyttöjärjestelmällä toimiville puhelimille suunnatun työkalun nimeltään ContextIO. Työkalu yhdistelee älypuhelimen sensorien tuottamaa tietoa ja havaitsee laitteen siirtymisen eri sijaintiin sisä- ja ulkotilojen suhteen. ContextIO:n suunnittelu ja kehitystyö perustuvat data-analyysiin, jonka data kerättiin AWARE-sensorialustan liitännäisellä sekä erillisellä nimeämistyökalulla. Data-analyysi keskittyy magnetometrin, valosensorin sekä GSM-kentän voimakkuuden hyödyntämiseen paikantamisessa. Kerätystä datasta määriteltiin raja-arvot, joita yhdistelemällä voidaan varsin luotettavasti todeta laitteen sijainti sisä- ja ulkotilojen suhteen. Nämä raja-arvot luovat perustan ContextIO:n reaaliaikaiselle laitteen sijainnin määrittämiselle. Tämän diplomityön pääasialliset tulokset ovat työkalu Android-pohjaisten älypuhelinten sijainnin määrittämiseen sisä- ja ulkotilojen suhteen, avoin datasetti, jota muut tutkijat voivat käyttää sekä sijainnin määrittämiseen keskittyvä data-analyysi

The Challenge of Continuous Mobile Context Sensing

National Research Foundation (NRF) Singapore under IDM Futures Funding Initiativ

Energy-Efficient Time-Stampless Adaptive Nonuniform Sampling

Nowadays, since more and more battery-operated devices are involved in applications with continuous sensing, development of an efficient sampling mechanisms is an important issue for these applications. In this paper, we investigate power efficiency aspects of a recently proposed adaptive nonuniform sampling. This sampling scheme minimizes the energy consumption of the sampling process, which is approximately proportional to sampling rate. The main characteristics of our method are that, first, sampling times do not need to be transmitted, since the receiver can compute them by using a function of previously taken samples, and second, only innovative samples are taken from the signal of interest, reducing the sampling rate and therefore the energy consumption. We call this scheme Time-Stampless Adaptive Nonuniform Sampling (TANS). TANS can be used in several scenarios, showing promising results in terms of energy savings, and can potentially enable the development of new applications that require continuous signals sensing, such as applications related to health monitoring, location tracking and entertainment

Mobile Sensing Systems

[EN] Rich-sensor smart phones have made possible the recent birth of the mobile sensing research area as part of ubiquitous sensing which integrates other areas such as wireless sensor networks and web sensing. There are several types of mobile sensing: individual, participatory, opportunistic, crowd, social, etc. The object of sensing can be people-centered or environment-centered. The sensing domain can be home, urban, vehicular Currently there are barriers that limit the social acceptance of mobile sensing systems. Examples of social barriers are privacy concerns, restrictive laws in some countries and the absence of economic incentives that might encourage people to participate in a sensing campaign. Several technical barriers are phone energy savings and the variety of sensors and software for their management. Some existing surveys partially tackle the topic of mobile sensing systems. Published papers theoretically or partially solve the above barriers. We complete the above surveys with new works, review the barriers of mobile sensing systems and propose some ideas for efficiently implementing sensing, fusion, learning, security, privacy and energy saving for any type of mobile sensing system, and propose several realistic research challenges. The main objective is to reduce the learning curve in mobile sensing systems where the complexity is very high.This work has been partially supported by the "Ministerio de Ciencia e Innovacion", through the "Plan Nacional de I+D+i 2008-2011" in the "Subprograma de Proyectos de Investigacion Fundamental", project TEC2011-27516, and by the Polytechnic University of Valencia, through the PAID-05-12 multidisciplinary projects.Macias Lopez, EM.; Suarez Sarmiento, A.; Lloret, J. (2013). Mobile Sensing Systems. Sensors. 13(12):17292-17321. https://doi.org/10.3390/s131217292S1729217321131

Context-awareness for mobile sensing: a survey and future directions

The evolution of smartphones together with increasing computational power have empowered developers to create innovative context-aware applications for recognizing user related social and cognitive activities in any situation and at any location. The existence and awareness of the context provides the capability of being conscious of physical environments or situations around mobile device users. This allows network services to respond proactively and intelligently based on such awareness. The key idea behind context-aware applications is to encourage users to collect, analyze and share local sensory knowledge in the purpose for a large scale community use by creating a smart network. The desired network is capable of making autonomous logical decisions to actuate environmental objects, and also assist individuals. However, many open challenges remain, which are mostly arisen due to the middleware services provided in mobile devices have limited resources in terms of power, memory and bandwidth. Thus, it becomes critically important to study how the drawbacks can be elaborated and resolved, and at the same time better understand the opportunities for the research community to contribute to the context-awareness. To this end, this paper surveys the literature over the period of 1991-2014 from the emerging concepts to applications of context-awareness in mobile platforms by providing up-to-date research and future research directions. Moreover, it points out the challenges faced in this regard and enlighten them by proposing possible solutions

DSP.Ear: Leveraging co-processor support for continuous audio sensing on smartphones

The rapidly growing adoption of sensor-enabled smartphones has greatly fueled the proliferation of applications that use phone sensors to monitor user behavior. A central sensor among these is the microphone which enables, for instance, the detection of valence in speech, or the identification of speakers. Deploying multiple of these applications on a mobile device to continuously monitor the audio environment allows for the acquisition of a diverse range of sound-related contextual inferences. However, the cumulative processing burden critically impacts the phone battery. To address this problem, we propose DSP.Ear - an integrated sensing system that takes advantage of the latest low-power DSP co-processor technology in commodity mobile devices to enable the continuous and simultaneous operation of multiple established algorithms that perform complex audio inferences. The system extracts emotions from voice, estimates the number of people in a room, identifies the speakers, and detects commonly found ambient sounds, while critically incurring little overhead to the device battery. This is achieved through a series of pipeline optimizations that allow the computation to remain largely on the DSP. Through detailed evaluation of our prototype implementation we show that, by exploiting a smartphone's co-processor, DSP.Ear achieves a 3 to 7 times increase in the battery lifetime compared to a solution that uses only the phone's main processor. In addition, DSP.Ear is 2 to 3 times more power efficient than a naive DSP solution without optimizations. We further analyze a large-scale dataset from 1320 Android users to show that in about 80-90% of the daily usage instances DSP.Ear is able to sustain a full day of operation (even in the presence of other smartphone workloads) with a single battery charge.This work was supported by Microsoft Research through its PhD Scholarship Program.This is the author's accepted manuscript. The final version is available from ACM in the proceedings of the ACM Conference on Embedded Networked Sensor Systems: http://dl.acm.org/citation.cfm?id=2668349

From data acquisition to data fusion : a comprehensive review and a roadmap for the identification of activities of daily living using mobile devices

This paper focuses on the research on the state of the art for sensor fusion techniques, applied to the sensors embedded in mobile devices, as a means to help identify the mobile device user’s daily activities. Sensor data fusion techniques are used to consolidate the data collected from several sensors, increasing the reliability of the algorithms for the identification of the different activities. However, mobile devices have several constraints, e.g., low memory, low battery life and low processing power, and some data fusion techniques are not suited to this scenario. The main purpose of this paper is to present an overview of the state of the art to identify examples of sensor data fusion techniques that can be applied to the sensors available in mobile devices aiming to identify activities of daily living (ADLs)

Compressed Sensing in Resource-Constrained Environments: From Sensing Mechanism Design to Recovery Algorithms

Compressed Sensing (CS) is an emerging field based on the revelation that a small collection of linear projections of a sparse signal contains enough information for reconstruction. It is promising that CS can be utilized in environments where the signal acquisition process is extremely difficult or costly, e.g., a resource-constrained environment like the smartphone platform, or a band-limited environment like visual sensor network (VSNs). There are several challenges to perform sensing due to the characteristic of these platforms, including, for example, needing active user involvement, computational and storage limitations and lower transmission capabilities. This dissertation focuses on the study of CS in resource-constrained environments. First, we try to solve the problem on how to design sensing mechanisms that could better adapt to the resource-limited smartphone platform. We propose the compressed phone sensing (CPS) framework where two challenging issues are studied, the energy drainage issue due to continuous sensing which may impede the normal functionality of the smartphones and the requirement of active user inputs for data collection that may place a high burden on the user. Second, we propose a CS reconstruction algorithm to be used in VSNs for recovery of frames/images. An efficient algorithm, NonLocal Douglas-Rachford (NLDR), is developed. NLDR takes advantage of self-similarity in images using nonlocal means (NL) filtering. We further formulate the nonlocal estimation as the low-rank matrix approximation problem and solve the constrained optimization problem using Douglas-Rachford splitting method. Third, we extend the NLDR algorithm to surveillance video processing in VSNs and propose recursive Low-rank and Sparse estimation through Douglas-Rachford splitting (rLSDR) method for recovery of the video frame into a low-rank background component and sparse component that corresponds to the moving object. The spatial and temporal low-rank features of the video frame, e.g., the nonlocal similar patches within the single video frame and the low-rank background component residing in multiple frames, are successfully exploited

DeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices

© 2016 IEEE. Breakthroughs from the field of deep learning are radically changing how sensor data are interpreted to extract the high-level information needed by mobile apps. It is critical that the gains in inference accuracy that deep models afford become embedded in future generations of mobile apps. In this work, we present the design and implementation of DeepX, a software accelerator for deep learning execution. DeepX signif- icantly lowers the device resources (viz. memory, computation, energy) required by deep learning that currently act as a severe bottleneck to mobile adoption. The foundation of DeepX is a pair of resource control algorithms, designed for the inference stage of deep learning, that: (1) decompose monolithic deep model network architectures into unit- blocks of various types, that are then more efficiently executed by heterogeneous local device processors (e.g., GPUs, CPUs); and (2), perform principled resource scaling that adjusts the architecture of deep models to shape the overhead each unit-blocks introduces. Experiments show, DeepX can allow even large-scale deep learning models to execute efficently on modern mobile processors and significantly outperform existing solutions, such as cloud-based offloading