Context Aware Computing for The Internet of Things: A Survey

As we are moving towards the Internet of Things (IoT), the number of sensors deployed around the world is growing at a rapid pace. Market research has shown a significant growth of sensor deployments over the past decade and has predicted a significant increment of the growth rate in the future. These sensors continuously generate enormous amounts of data. However, in order to add value to raw sensor data we need to understand it. Collection, modelling, reasoning, and distribution of context in relation to sensor data plays critical role in this challenge. Context-aware computing has proven to be successful in understanding sensor data. In this paper, we survey context awareness from an IoT perspective. We present the necessary background by introducing the IoT paradigm and context-aware fundamentals at the beginning. Then we provide an in-depth analysis of context life cycle. We evaluate a subset of projects (50) which represent the majority of research and commercial solutions proposed in the field of context-aware computing conducted over the last decade (2001-2011) based on our own taxonomy. Finally, based on our evaluation, we highlight the lessons to be learnt from the past and some possible directions for future research. The survey addresses a broad range of techniques, methods, models, functionalities, systems, applications, and middleware solutions related to context awareness and IoT. Our goal is not only to analyse, compare and consolidate past research work but also to appreciate their findings and discuss their applicability towards the IoT.Comment: IEEE Communications Surveys & Tutorials Journal, 201

Media aesthetics based multimedia storytelling.

Since the earliest of times, humans have been interested in recording their life experiences, for future reference and for storytelling purposes. This task of recording experiences --i.e., both image and video capture-- has never before in history been as easy as it is today. This is creating a digital information overload that is becoming a great concern for the people that are trying to preserve their life experiences. As high-resolution digital still and video cameras become increasingly pervasive, unprecedented amounts of multimedia, are being downloaded to personal hard drives, and also uploaded to online social networks on a daily basis. The work presented in this dissertation is a contribution in the area of multimedia organization, as well as automatic selection of media for storytelling purposes, which eases the human task of summarizing a collection of images or videos in order to be shared with other people. As opposed to some prior art in this area, we have taken an approach in which neither user generated tags nor comments --that describe the photographs, either in their local or on-line repositories-- are taken into account, and also no user interaction with the algorithms is expected. We take an image analysis approach where both the context images --e.g. images from online social networks to which the image stories are going to be uploaded--, and the collection images --i.e., the collection of images or videos that needs to be summarized into a story--, are analyzed using image processing algorithms. This allows us to extract relevant metadata that can be used in the summarization process. Multimedia-storytellers usually follow three main steps when preparing their stories: first they choose the main story characters, the main events to describe, and finally from these media sub-groups, they choose the media based on their relevance to the story as well as based on their aesthetic value. Therefore, one of the main contributions of our work has been the design of computational models --both regression based, as well as classification based-- that correlate well with human perception of the aesthetic value of images and videos. These computational aesthetics models have been integrated into automatic selection algorithms for multimedia storytelling, which are another important contribution of our work. A human centric approach has been used in all experiments where it was feasible, and also in order to assess the final summarization results, i.e., humans are always the final judges of our algorithms, either by inspecting the aesthetic quality of the media, or by inspecting the final story generated by our algorithms. We are aware that a perfect automatically generated story summary is very hard to obtain, given the many subjective factors that play a role in such a creative process; rather, the presented approach should be seen as a first step in the storytelling creative process which removes some of the ground work that would be tedious and time consuming for the user. Overall, the main contributions of this work can be capitalized in three: (1) new media aesthetics models for both images and videos that correlate with human perception, (2) new scalable multimedia collection structures that ease the process of media summarization, and finally, (3) new media selection algorithms that are optimized for multimedia storytelling purposes.Postprint (published version

Augmenting users' task performance through workspace narrative exploration

In a fast-paced office setting, information workers inevitably experience expected and unexpected interruptions daily. As the volume and the diversity of information and application types grow, the impact of frequent interruptions on their task performance gets more severe. To manage the negative effects of interruptions on work performance, workers often engage in task management activities to ensure they are better prepared to resume suspended task less stressfully. However, managing tasks causes additional cognitive burden and a time cost to users who already are experiencing the tight attention and time economies. This dissertation presents an approach to augmenting users' task performance by allowing them to manage and retrieve desired work contexts with ease. The Context Browser, the implementation of the proposed approach, is designed to help the users to explore narratives of their workspace manner and restore their previous work contexts. The goals of implementing the Context Browser are to 1) unload the users? burden of taking care of their task-related or task status information promptly and thus help them focus solely on executing a given task, 2) allow them to browse their previous workspace intuitively, and 3) enhance continuity of their tasks by supporting them to retrieve desired work context more quickly and easily. In order to validate the proposed approach, a user study comparing task performances of the group with the Context Browser to the one without the Context Browser was conducted. The study produced both quantitative and qualitative results. The study confirmed that with the Context Browser subjects expressed better quantitative numbers than the ones without. Subjects using the Context Browser were able to restore and retrieve their desired work setting and task-related information more quickly and correctly. Qualitative results showed that the subjects using the Context Browser found that various contextual cues and the interfaces responsible for providing the cues offered effective artifacts to help them recover both cognitive and work contexts, while the other subjects experienced a difficult time in restoring the desired contexts that were necessary to perform their assigned tasks. In addition, we re-invited 6 subjects from the group without the Context Browser 6 weeks after the study. We asked them to perform the same tasks as the ones they did 6 weeks before with the Context Browser. It showed that with the Context Browser they outperformed their previous performance even after a lengthy period