User Modeling for a Personal Assistant

We present a user modeling system that serves as the foun-dation of a personal assistant. The system ingests web search history for signed-in users, and identifies coherent contexts that correspond to tasks, interests, and habits. Un-like past work which focused on either in-session tasks or tasks over a few days, we look at several months of his-tory in order to identify not just short-term tasks, but also long-term interests and habits. The features we use for iden-tifying coherent contexts yield substantially higher precision and recall than past work. We also present an algorithm for identifying contexts that is 8 to 30 times faster than previous algorithms. The user modeling system has been deployed in production. It runs over hundreds of millions of users, and updates the models with a 10-minute latency. The contexts identified by the system serve as the foundation for gener-ating recommendations in Google Now. 1

Anticipating Information Needs Based on Check-in Activity

In this work we address the development of a smart personal assistant that is capable of anticipating a user's information needs based on a novel type of context: the person's activity inferred from her check-in records on a location-based social network. Our main contribution is a method that translates a check-in activity into an information need, which is in turn addressed with an appropriate information card. This task is challenging because of the large number of possible activities and related information needs, which need to be addressed in a mobile dashboard that is limited in size. Our approach considers each possible activity that might follow after the last (and already finished) activity, and selects the top information cards such that they maximize the likelihood of satisfying the user's information needs for all possible future scenarios. The proposed models also incorporate knowledge about the temporal dynamics of information needs. Using a combination of historical check-in data and manual assessments collected via crowdsourcing, we show experimentally the effectiveness of our approach.Comment: Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM '17), 201

Proactive Information Retrieval via Screen Surveillance

We demonstrate proactive information retrieval via screen surveillance. A user's digital activities are continuously monitored by capturing all content on a user's screen using optical character recognition. This includes all applications and services being exploited and relies on each individual user's computer usage, such as their Web browsing, emails, instant messaging, and word processing. Topic modeling is then applied to detect the user's topical activity context to retrieve information. We demonstrate a system that proactively retrieves information from a user's activity history being observed on the screen when the user is performing unseen activities on a personal computer. We report an evaluation with ten participants that shows high user satisfaction and retrieval effectiveness. Our demonstration and experimental results show that surveillance of a user's screen can be used to build an extremely rich model of a user's digital activities across application boundaries and enable effective proactive information retrieval.Peer reviewe

A Natural Language Query Interface for Searching Personal Information on Smartwatches

Currently, personal assistant systems, run on smartphones and use natural language interfaces. However, these systems rely mostly on the web for finding information. Mobile and wearable devices can collect an enormous amount of contextual personal data such as sleep and physical activities. These information objects and their applications are known as quantified-self, mobile health or personal informatics, and they can be used to provide a deeper insight into our behavior. To our knowledge, existing personal assistant systems do not support all types of quantified-self queries. In response to this, we have undertaken a user study to analyze a set of “textual questions/queries” that users have used to search their quantified-self or mobile health data. Through analyzing these questions, we have constructed a light-weight natural language based query interface - including a text parser algorithm and a user interface - to process the users’ queries that have been used for searching quantified-self information. This query interface has been designed to operate on small devices, i.e. smartwatches, as well as augmenting the personal assistant systems by allowing them to process end users’ natural language queries about their quantified-self data

Watching inside the Screen: Digital Activity Monitoring for Task Recognition and Proactive Information Retrieval

We investigate to what extent it is possible to infer a user’s work tasks by digital activity monitoring and use the task models for proactive information retrieval. Ten participants volunteered for the study, in which their computer screen was monitored and related logs were recorded for 14 days. Corresponding diary entries were collected to provide ground truth to the task detection method. We report two experiments using this data. The unsupervised task detection experiment was conducted to detect tasks using unsupervised topic modeling. The results show an average task detection accuracy of more than 70% by using rich screen monitoring data. The single-trial task detection and retrieval experiment utilized unseen user inputs in order to detect related work tasks and retrieve task-relevant information on-line. We report an average task detection accuracy of 95%, and the corresponding model-based document retrieval with Normalized Discounted Cumulative Gain of 98%. We discuss and provide insights regarding the types of digital tasks occurring in the data, the accuracy of task detection on different task types, and the role of using different data input such as application names, extracted keywords, and bag-of-words representations in the task detection process. We also discuss the implications of our results for ubiquitous user modeling and privacy.Peer reviewe

Human Behavior in Domestic Environments: Prediction and Applications

A longstanding goal of human behavior science is to model and predict how humans interact with each other or with other systems. Such models are beneficial and have many applications, including designing and implementing assistive technologies, improving users\u27 experiences and quality of life and making better decisions to create public policies. Behavior is highly complex due to uncertainties and a lack of scientific tools to measure it. Hence prediction of human behavior cannot be 100% accurate. However, prediction is also not hopeless because the biological needs, as well as cultural conventions (for instance, regarding meal times) set the general patterns of the humans\u27 daily behavior. Furthermore, while individual humans might adjust these patterns according to their own preferences, they also show some degree of consistency in their daily routine. In this dissertation, we focus on interrelated challenges of improving the prediction models for human daily activities and developing techniques through which intelligent applications can benefit from this improved prediction. We describe techniques for creating predictive models that can help humans in their daily life using deep learning-based models. One of the challenges of learning based approaches in this setting is the scarcity of data. If we are collecting information about a given human in a home, our database will increase with exactly one sample a day – this is insufficient for deep learning algorithms that are often trained on datasets with millions of samples. We investigate three directions through which the paucity of samples can be overcome. First, we discuss techniques through which, starting from a small number of representative samples, we can generate much larger synthetic datasets that capture the statistical properties of the real world data, and can be used in training. We consider an application where we apply human behavior prediction to the practical problem of improving the quality of experience. By learning to predict the experience requested by the user, we are able to perform intelligent pre-caching, and achieve higher average quality of experience for a given available network bandwidth. Another direction we investigate is the collection of data from multiple users. This creates multiple challenges. First, users would prefer to minimize the shared personal data. This requires us to investigate techniques that learn predictive models from multiple user experiences without requiring the users to upload their data to a common repository. We adapt the technique of federated learning, which requires the users to only share the training gradients on a model that had been sent by a central server, but not raw data. We investigate procedures that allow the user to obtain the best possible model for her own prediction while minimizing the amount of data disclosed. The second challenge is that not all the users benefit to the same degree from creating a central learning model; by investigating how much the user can benefit, we can stop the learning process and implicit privacy loss earlier. Finally, we developed predictive models for the spread of pandemics and techniques that use these predictions to recommend Non-Pharmaceutical Interventions (NPIs) to local stakeholders. We find that the prediction of pandemics is also conditioned on the behavior of individual humans and the actions taken by the governments and, especially in the early phases of the pandemic, suffers from a lack of data. We used a combination of a deep learning-based predictive model with a compartmental model, which is trained on the months elapsed from the pandemic and predicts infection rates for the next months. We used cultural and geographical attributes as constant features along with the history of cases and deaths as context features and NPIs as action features to train a single predictive model that can predict both the infection rate and the stringency of the NPIs deployed by policymakers for all countries / regions. We found that the stringency is not always aligned with the number of cases but also depends on political, economic and cultural factors

Inferring user interests in microblogging social networks: a survey

With the growing popularity of microblogging services such as Twitter in recent years, an increasing number of users are using these services in their daily lives. The huge volume of information generated by users raises new opportunities in various applications and areas. Inferring user interests plays a significant role in providing personalized recommendations on microblogging services, and also on third-party applications providing social logins via these services, especially in cold-start situations. In this survey, we review user modeling strategies with respect to inferring user interests from previous studies. To this end, we focus on four dimensions of inferring user interest profiles: (1) data collection, (2) representation of user interest profiles, (3) construction and enhancement of user interest profiles, and (4) the evaluation of the constructed profiles. Through this survey, we aim to provide an overview of state-of-the-art user modeling strategies for inferring user interest profiles on microblogging social networks with respect to the four dimensions. For each dimension, we review and summarize previous studies based on specified criteria. Finally, we discuss some challenges and opportunities for future work in this research domain

Just-in-time information retrieval and summarization for personal assistance

With the rapid development of means for producing user-generated data opportunities for collecting such data over a time-line and utilizing it for various human-aid applications are more than ever. Wearable and mobile data capture devices as well as many online data channels such as search engines are all examples of means of user data collection. Such user data could be utilized to model user behavior, identify relevant information to a user and retrieve it in a timely fashion for personal assistance. User data can include recordings of one's conversations, images, biophysical data, health-related data captured by wearable devices, interactions with smartphones and computers, and more. In order to utilize such data for personal assistance, summaries of previously recorded events can be presented to a user in order to augment the user's memory, send notifications about important events to the user, predict the user's near-future information needs and retrieve relevant content even before the user asks. In this PhD dissertation, we design a personal assistant with a focus on two main aspects: The first aspect is that a personal assistant should be able to summarize user data and present it to a user. To achieve this goal, we build a Social Interactions Log Analysis System (SILAS) that summarizes a person's conversations into event snippets consisting of spoken topics paired with images and other modalities of data captured by the person's wearable devices. Furthermore, we design a novel discrete Dynamic Topic Model (dDTM) capable of tracking the evolution of the intermittent spoken topics over time. Additionally, we present the first neural Customizable Abstractive Topic-based Summarization (CATS) model that produces summaries of textual documents including meeting transcripts in the form of natural language. The second aspect that a personal assistant should be capable of, is proactively addressing the user's information needs. For this purpose, we propose a family of just-in-time information retrieval models such as an evolutionary model named Kalman combination of Recency and Establishment (K2RE) that can anticipate a user's near-future information needs. Such information needs can include information for preparing a future meeting or near-future search queries of a user

Grundlagen der Informationswissenschaft

Die 7. Ausgabe der "Grundlagen der praktischen Information und Dokumentation" (Erstausgabe 1972) heißt jetzt: „Grundlagen der Informationswissenschaft". Der Bezug zur Praxis und zur Ausbildung bleibt erhalten, aber der neue Titel trägt dem Rechnung, dass die wissenschaftliche theoretische Absicherung für alle Bereiche von Wissen und Information, nicht nur in der Fachinformation, sondern auch in den Informationsdiensten des Internet immer wichtiger wird. Für die Grundlagen sind 73 Artikel in 6 Hauptkapiteln vorgesehen. Viele Themen werden zum ersten Mal behandelt, z.B. Information und Emotion, Informationelle Selbstbestimmung, Informationspathologien. Alle Beiträge sind neu verfasst