From Hyperlinks to Hypercues : Entity-Based Affordances for Fluid Information Exploration

Peer reviewe

Naturalistic digital task modeling for personal information assistance via continuous screen monitoring

Peer reviewe

Proactive Information Retrieval via Screen Surveillance

We demonstrate proactive information retrieval via screen surveillance. A user's digital activities are continuously monitored by capturing all content on a user's screen using optical character recognition. This includes all applications and services being exploited and relies on each individual user's computer usage, such as their Web browsing, emails, instant messaging, and word processing. Topic modeling is then applied to detect the user's topical activity context to retrieve information. We demonstrate a system that proactively retrieves information from a user's activity history being observed on the screen when the user is performing unseen activities on a personal computer. We report an evaluation with ten participants that shows high user satisfaction and retrieval effectiveness. Our demonstration and experimental results show that surveillance of a user's screen can be used to build an extremely rich model of a user's digital activities across application boundaries and enable effective proactive information retrieval.Peer reviewe

Interactive faceted query suggestion for exploratory search : Whole-session effectiveness and interaction engagement

Abstract The outcome of exploratory information retrieval is not only dependent on the effectiveness of individual responses to a set of queries, but also on relevant information retrieved during the entire exploratory search session. We study the effect of search assistance, operationalized as an interactive faceted query suggestion, for both whole-session effectiveness and engagement through interactive faceted query suggestion. A user experiment is reported, where users performed exploratory search tasks, comparing interactive faceted query suggestion and a control condition with only conventional typed-query interaction. Data comprised of interaction and search logs show that the availability of interactive faceted query suggestion substantially improves whole-session effectiveness by increasing recall without sacrificing precision. The increased engagement with interactive faceted query suggestion is targeted to direct situated navigation around the initial query scope, but is not found to improve individual queries on average. The results imply that research in exploratory search should focus on measuring and designing tools that engage users with directed situated navigation support for improving whole-session performance.Peer reviewe

Watching inside the Screen: Digital Activity Monitoring for Task Recognition and Proactive Information Retrieval

We investigate to what extent it is possible to infer a user’s work tasks by digital activity monitoring and use the task models for proactive information retrieval. Ten participants volunteered for the study, in which their computer screen was monitored and related logs were recorded for 14 days. Corresponding diary entries were collected to provide ground truth to the task detection method. We report two experiments using this data. The unsupervised task detection experiment was conducted to detect tasks using unsupervised topic modeling. The results show an average task detection accuracy of more than 70% by using rich screen monitoring data. The single-trial task detection and retrieval experiment utilized unseen user inputs in order to detect related work tasks and retrieve task-relevant information on-line. We report an average task detection accuracy of 95%, and the corresponding model-based document retrieval with Normalized Discounted Cumulative Gain of 98%. We discuss and provide insights regarding the types of digital tasks occurring in the data, the accuracy of task detection on different task types, and the role of using different data input such as application names, extracted keywords, and bag-of-words representations in the task detection process. We also discuss the implications of our results for ubiquitous user modeling and privacy.Peer reviewe

Why Do Users Issue Good Queries? : Neural Correlates of Term Specificity

JUFO2Despite advances in the past few decades in studying what kind of queries users input to search engines and how to suggest queries for the users, the fundamental question of what makes human cognition able to estimate goodness of query terms is largely unanswered. For example, a person searching information about "cats" is able to choose query terms, such as "housecat", "feline", or "animal" and avoid terms like "similar", "variety", and "distinguish". We investigated the association between the specificity of terms occurring in documents and human brain activity measured via electroencephalography (EEG). We analyzed the brain activity data of fifteen participants, recorded in response to reading terms from Wikipedia documents. Term specificity was shown to be associated with the amplitude of evoked brain responses. The results indicate that by being able to determine which terms carry maximal information about, and can best discriminate between, documents, people have the capability to enter good query terms. Moreover, our results suggest that the effective query term selection process, often observed in practical search behavior studies, has a neural basis. We believe our findings constitute an important step in revealing the cognitive processing behind query formulation and evaluating informativeness of language in general.Peer reviewe

T4Tags 2.0: a tool to support the serendipitous use of domestic technologies

The authors present an iterative design exploration to support serendipitous uses of technology: quick reconfigurations of the domestic environment to address inhabitants' current needs, whether they are transient and ephemeral or more habitual. As a result of this exploration, the authors developed T4Tags 2.0, an open-ended toolkit for programming Web-connected and versatile physical tokens embedded with different sensing technologies (including near field communication, physical buttons, and motion and environmental sensors) and can be easily integrated with existing artifacts. The design of the toolkit was informed by fieldwork that provided design drivers for domestic technologies that can be repurposed or appropriated through features such as end-user programming of device behavior and crowd-fueled appropriation by sharing "recipes" of programmed tokens. A user study with three families provides insights into system usefulness and the recipe-sharing functionality. The authors also discuss opportunities and challenges, reflecting on the tradeoffs of an open system in terms of user engagement, creative input, and real-world deployment. This article is part of a special issue on domestic pervasive computing.This research has been partially funded by the 2015 UC3M Post-Doc Mobility Scholarship and by the Academy of Finland (286440, Evidence

Augmenting the Performer-Audience Live Participation in Professional Event Productions

Event productions, such as corporate workshops, night galas, or networking events, can reach higher levels of participant experience and productiveness if performer-audience interactions are augmented with possibilities for live participation from mobile terminals. However, it easily happens that polls, backchannels, chat screens and other methods remain as mere activation tricks that fall short from a successful integration to events' goals and content. Based on a 10-year process of developing live participation technologies, deploying them successfully in collaboration with event producers in over 100 professionally organised event productions ranging from 10 to 400 participants, we analyse techniques that increase events' value for the audience and the organisers. Building on our experiences and event studies literature, we describe how positive audience participation can be achieved by supporting cognitive (informational), affective (experiential) and conative (behavioural) elements of event participation, thus helping the audience members notice how the event supports their needs.Peer reviewe

Augmenting objects at home through programmable sensor tokens: A design journey

End-user development for the home has been gaining momentum in research. Previous works demonstrate feasibility and potential but there is a lack of analysis of the extent of technology needed and its impact on the diversity of activities that can be supported. We present a design exploration with a tangible end-user toolkit for programming smart tokens embedding different sensing technologies. Our system allows to augment physical objects with smart tags and use trigger-action programming with multiple triggers to define smart behaviors. We contribute through a field-oriented study that provided insights on (i) household's activities as emerging from people's lived experience in terms of high-level goals, their ephemerality or recurrence, and the types of triggers, actions and interactions with augmented objects, and (ii) the programmability needed for supporting desired behaviors. We conclude that, while trigger action covers most scenarios, more advanced programming and direct interaction with physical objects spur novel uses.This work was supported by the 2015 UC3M Mobility Grant, the Spanish Ministry of Economy and Competitivity (TIN2014-56534-R, CREAx) and by the Academy of Finland (286440, Evidence)