Contextual awareness, messaging and communication in nomadic audio environments

Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1998.Includes bibliographical references (p. 119-122).Nitin Sawhney.M.S

Interruptibility prediction for ubiquitous systems: conventions and new directions from a growing field

When should a machine attempt to communicate with a user? This is a historical problem that has been studied since the rise of personal computing. More recently, the emergence of pervasive technologies such as the smartphone have extended the problem to be ever-present in our daily lives, opening up new opportunities for context awareness through data collection and reasoning. Complementary to this there has been increasing interest in techniques to intelligently synchronise interruptions with human behaviour and cognition. However, it is increasingly challenging to categorise new developments, which are often scenario specific or scope a problem with particular unique features. In this paper we present a meta-analysis of this area, decomposing and comparing historical and recent works that seek to understand and predict how users will perceive and respond to interruptions. In doing so we identify research gaps, questions and opportunities that characterise this important emerging field for pervasive technology

A model for adaptive multimodal mobile notification

Information is useless unless it is used whilst still applicable. Having a system that notifies the user of important messages using the most appropriate medium and device will benefit users that rely on time critical information. There are several existing systems and models for mobile notification as well as for adaptive mobile notification using context awareness. Current models and systems are typically designed for a specific set of mobile devices, modes and services. Communication however, can take place in many different modes, across many different devices and may originate from many different sources. The aim of this research was to develop a model for adaptive mobile notification using context awareness. An extensive literature study was performed into existing models for adaptive mobile notification systems using context awareness. The literature study identified several potential models but no way to evaluate and compare the models. A set of requirements to evaluate these models was developed and the models were evaluated against these criteria. The model satisfying the most requirements was adapted so as to satisfy the remaining criteria. The proposed model is extensible in terms of the modes, devices and notification sources supported. The proposed model determines the importance of a message, the appropriate device and mode (or modes) of communication based on the user‘s context, and alerts the user of the message using these modes. A prototype was developed as a proof-of-concept of the proposed model and evaluated by conducting an extensive field study. The field study highlighted the fact that most users did not choose the most suitable mode for the context during their initial subscription to the service. The field study also showed that more research needs to be done on an appropriate filtering mechanism for notifications. Users found that the notifications became intrusive and less useful the longer they used them

Reachable but not receptive: enhancing smartphone interruptibility prediction by modelling the extent of user engagement with notifications

Smartphone notifications frequently interrupt our daily lives, often at inopportune moments. We propose the decision-on-information-gain model, which extends the existing data collection convention to capture a range of interruptibility behaviour implicitly. Through a six-month in-the-wild study of 11,346 notifications, we find that this approach captures up to 125% more interruptibility cases. Secondly, we find different correlating contextual features for different behaviour using the approach and find that predictive models can be built with >80% precision for most users. However we note discrepancies in performance across labelling, training, and evaluation methods, creating design considerations for future systems

A Design Rationale for Pervasive Computing - User Experience, Contextual Change, and Technical Requirements

The vision of pervasive computing promises a shift from information technology per se to what can be accomplished by using it, thereby fundamentally changing the relationship between people and information technology. In order to realize this vision, a large number of issues concerning user experience, contextual change, and technical requirements should be addressed. We provide a design rationale for pervasive computing that encompasses these issues, in which we argue that a prominent aspect of user experience is to provide user control, primarily founded in human values. As one of the more significant aspects of the user experience, we provide an extended discussion about privacy. With contextual change, we address the fundamental change in previously established relationships between the practices of individuals, social institutions, and physical environments that pervasive computing entails. Finally, issues of technical requirements refer to technology neutrality and openness--factors that we argue are fundamental for realizing pervasive computing. We describe a number of empirical and technical studies, the results of which have helped to verify aspects of the design rationale as well as shaping new aspects of it. The empirical studies include an ethnographic-inspired study focusing on information technology support for everyday activities, a study based on structured interviews concerning relationships between contexts of use and everyday planning activities, and a focus group study of laypeople’s interpretations of the concept of privacy in relation to information technology. The first technical study concerns the model of personal service environments as a means for addressing a number of challenges concerning user experience, contextual change, and technical requirements. Two other technical studies relate to a model for device-independent service development and the wearable server as a means to address issues of continuous usage experience and technology neutrality respectively

Designing a Wise Home: Leveraging Lightweight Dialogue, Proactive Coaching, Guided Experimentation and Mutual-learning to Support Mixed-initiative Homes --Comfort-aware Thermostats as a Case

Science fiction writers have been dreaming of homes that can understand our preferences, assist our daily chores and teach us to be healthier, more sustainable and more knowledgeable. While we are still far from achieving this dream, the recent development of mobile devices, wearable interfaces, smart home appliances, and machine learning offer unprecedented opportunities for homes to better understand our goals, preferences and contexts, as well as to facilitate our everyday tasks and decision making. These recent advancement open a new possibility to create homes beyond simple automation: they enable creation of homes to coach us to achieve our better selves. That is, homes that are not just smart, but wise as well. However, to develop a wise home, there are still two key questions: How can a wise home coach its occupants while considering their different goals and needs? How can a home integrates emerging sensors, devices and interfaces to better understand their goals, preferences and contexts in order to support coaching? To answer these two questions, in this dissertation I use residential heating and cooling control as a lens to advance the development of wiser homes. Based on the three studies conducted, this thesis provides three contributions. First, I show that we can integrate a diverse class of emerging devices, including mobile phones, smartwatches, in-home sensors and home appliances to capture important user contexts, such as individual preferences for thermal comfort. The integration of these emerging devices enables a home to better coach its occupants and potentially better support automation. Secondly, I show that mixed-initiative interaction is an effective approach in the design of a wise home, and further propose four design strategies to support mixed-initiative homes, namely, lightweight dialogue, proactive coaching, guided experimentation and mutual-learning. Finally, I demonstrate a novel system that integrates the above-mentioned strategies to support the development of wise homes, facilitating home occupants to identify actions to achieve a better balance between their comfort and savings goals.PHDInformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/153457/1/chuanche_1.pd

行動認識機械学習データセット収集のためのクラウドソーシングの研究

In this thesis, we propose novel methods to explore and improve crowdsourced data labeling for mobile activity recognition. This thesis concerns itself with the quality (i.e., the performance of a classification model), quantity (i.e., the number of data collected), and motivation (i.e., the process that initiates and maintains goal-oriented behaviors) of participant contributions in mobile activity data collection studies. We focus on achieving high-quality and consistent ground-truth labeling and, particularly, on user feedback’s impact under different conditions. Although prior works have used several techniques to improve activity recognition performance, differences to our approach exist in terms of the end goals, proposed method, and implementation. Many researchers commonly investigate post-data collection to increase activity recognition accuracy, such as implementing advanced machine learning algorithms to improve data quality or exploring several preprocessing ways to increase data quantity. However, utilizing post-data collection results is very difficult and time-consuming due to dirty data challenges for most real-world situations. Unlike those commonly used in other literature, in this thesis, we aim to motivate and sustain user engagement during their on-going-self-labeling task to optimize activity recognition accuracy. The outline of the thesis is as follows: In chapter 1 and 2, we briefly introduce the thesis work and literature review. In Chapter 3, we introduce novel gamified active learning and inaccuracy detection for crowdsourced data labeling for an activity recognition system (CrowdAct) using mobile sensing. We exploited active learning to address the lack of accurate information. We presented the integration of gamification into active learning to overcome the lack of motivation and sustained engagement. We introduced an inaccuracy detection algorithm to minimize inaccurate data. In Chapter 4, we introduce a novel method to exploit on-device deep learning inference using a long short-term memory (LSTM)-based approach to alleviate the labeling effort and ground truth data collection in activity recognition systems using smartphone sensors. The novel idea behind this is that estimated activities are used as feedback for motivating users to collect accurate activity labels. In Chapter 5, we introduce a novel on-device personalization for data labeling for an activity recognition system using mobile sensing. The key idea behind this system is that estimated activities personalized for a specific individual user can be used as feedback to motivate user contribution and improve data labeling quality. We exploited finetuning using a Deep Recurrent Neural Network (RNN) to address the lack of sufficient training data and minimize the need for training deep learning on mobile devices from scratch. We utilized a model pruning technique to reduce the computation cost of on-device personalization without affecting the accuracy. Finally, we built a robust activity data labeling system by integrating the two techniques outlined above, allowing the mobile application to create a personalized experience for the user. To demonstrate the proposed methods’ capability and feasibility in realistic settings, we developed and deployed the systems to real-world settings such as crowdsourcing. For the process of data labeling, we challenged online and self-labeling scenarios using inertial smartphone sensors, such as accelerometers. We recruited diverse participants and con- ducted the experiments both in a laboratory setting and in a semi-natural setting. We also applied both manual labeling and the assistance of semi-automated labeling. Addition- ally, we gathered massive labeled training data in activity recognition using smartphone sensors and other information such as user demographics and engagement. Chapter 6 offers a brief discussion of the thesis. In Chapter 7, we conclude the thesis with conclusion and some future work issues. We empirically evaluated these methods across various study goals such as machine learning and descriptive and inferential statistics. Our results indicated that this study enabled us to effectively collect crowdsourced activity data. Our work revealed clear opportunities and challenges in combining human and mobile phone-based sensing techniques for researchers interested in studying human behavior in situ. Researchers and practitioners can apply our findings to improve recognition accuracy and reduce unreliable labels by human users, increase the total number of collected responses, as well as enhance participant motivation for activity data collection.九州工業大学博士学位論文 学位記番号：工博甲第526号 学位授与年月日：令和3年6月28日1 Introduction|2 Related work|3 Achieving High-Quality Crowdsourced Datasets in Mobile Activity Recognition|4 On-Device Deep Learning Inference for Activity Data Collection|5 On-Device Deep Personalization for Activity Data Collection|6 Discussion|7 Conclusion九州工業大学令和3年

Contextual Social Networking

The thesis centers around the multi-faceted research question of how contexts may be detected and derived that can be used for new context aware Social Networking services and for improving the usefulness of existing Social Networking services, giving rise to the notion of Contextual Social Networking. In a first foundational part, we characterize the closely related fields of Contextual-, Mobile-, and Decentralized Social Networking using different methods and focusing on different detailed aspects. A second part focuses on the question of how short-term and long-term social contexts as especially interesting forms of context for Social Networking may be derived. We focus on NLP based methods for the characterization of social relations as a typical form of long-term social contexts and on Mobile Social Signal Processing methods for deriving short-term social contexts on the basis of geometry of interaction and audio. We furthermore investigate, how personal social agents may combine such social context elements on various levels of abstraction. The third part discusses new and improved context aware Social Networking service concepts. We investigate special forms of awareness services, new forms of social information retrieval, social recommender systems, context aware privacy concepts and services and platforms supporting Open Innovation and creative processes. This version of the thesis does not contain the included publications because of copyrights of the journals etc. Contact in terms of the version with all included publications: Georg Groh, [email protected] zentrale Gegenstand der vorliegenden Arbeit ist die vielschichtige Frage, wie Kontexte detektiert und abgeleitet werden können, die dazu dienen können, neuartige kontextbewusste Social Networking Dienste zu schaffen und bestehende Dienste in ihrem Nutzwert zu verbessern. Die (noch nicht abgeschlossene) erfolgreiche Umsetzung dieses Programmes führt auf ein Konzept, das man als Contextual Social Networking bezeichnen kann. In einem grundlegenden ersten Teil werden die eng zusammenhängenden Gebiete Contextual Social Networking, Mobile Social Networking und Decentralized Social Networking mit verschiedenen Methoden und unter Fokussierung auf verschiedene Detail-Aspekte näher beleuchtet und in Zusammenhang gesetzt. Ein zweiter Teil behandelt die Frage, wie soziale Kurzzeit- und Langzeit-Kontexte als für das Social Networking besonders interessante Formen von Kontext gemessen und abgeleitet werden können. Ein Fokus liegt hierbei auf NLP Methoden zur Charakterisierung sozialer Beziehungen als einer typischen Form von sozialem Langzeit-Kontext. Ein weiterer Schwerpunkt liegt auf Methoden aus dem Mobile Social Signal Processing zur Ableitung sinnvoller sozialer Kurzzeit-Kontexte auf der Basis von Interaktionsgeometrien und Audio-Daten. Es wird ferner untersucht, wie persönliche soziale Agenten Kontext-Elemente verschiedener Abstraktionsgrade miteinander kombinieren können. Der dritte Teil behandelt neuartige und verbesserte Konzepte für kontextbewusste Social Networking Dienste. Es werden spezielle Formen von Awareness Diensten, neue Formen von sozialem Information Retrieval, Konzepte für kontextbewusstes Privacy Management und Dienste und Plattformen zur Unterstützung von Open Innovation und Kreativität untersucht und vorgestellt. Diese Version der Habilitationsschrift enthält die inkludierten Publikationen zurVermeidung von Copyright-Verletzungen auf Seiten der Journals u.a. nicht. Kontakt in Bezug auf die Version mit allen inkludierten Publikationen: Georg Groh, [email protected]