Understanding Mobile Search Task Relevance and User Behaviour in Context

Improvements in mobile technologies have led to a dramatic change in how and when people access and use information, and is having a profound impact on how users address their daily information needs. Smart phones are rapidly becoming our main method of accessing information and are frequently used to perform `on-the-go' search tasks. As research into information retrieval continues to evolve, evaluating search behaviour in context is relatively new. Previous research has studied the effects of context through either self-reported diary studies or quantitative log analysis; however, neither approach is able to accurately capture context of use at the time of searching. In this study, we aim to gain a better understanding of task relevance and search behaviour via a task-based user study (n=31) employing a bespoke Android app. The app allowed us to accurately capture the user's context when completing tasks at different times of the day over the period of a week. Through analysis of the collected data, we gain a better understanding of how using smart phones on the go impacts search behaviour, search performance and task relevance and whether or not the actual context is an important factor.Comment: To appear in CHIIR 2019 in Glasgow, U

Modeling user information needs on mobile devices: from recommendation to conversation

Recent advances in the development of mobile devices, equipped with multiple sensors, together with the availability of millions of applications have made these devices more pervasive in our lives than ever. The availability of the diverse set of sensors, as well as high computational power, enable information retrieval (IR) systems to sense a user’s context and personalize their results accordingly. Relevant studies show that people use their mobile devices to access information in a wide range of topics in various contextual situations, highlighting the fact that modeling user information need on mobile devices involves studying several means of information access. In this thesis, we study three major aspects of information access on mobile devices. First, we focus on proactive approaches to modeling users for venue suggestion. We investigate three methods of user modeling, namely, content-based, collaborative, and hybrid, focusing on personalization and context-awareness. We propose a two-phase collaborative ranking algorithm for leveraging users’ implicit feedback while incorporating temporal and geographical information into the model. We then extend our collaborative model to include multiple cross-venue similarity scores and combine it with our content-based approach to produce a hybrid recommendation. Second, we introduce and investigate a new task on mobile search, that is, unified mobile search. We take the first step in defining, studying, and modeling this task by collecting two datasets and conducting experiments on one of the main components of unified mobile search frameworks, that is target apps selection. To this end, we propose two neural approaches. Finally, we address the conversational aspect of mobile search where we propose an offline evaluation protocol and build a dataset for asking clarifying questions for conversational search. Also, we propose a retrieval framework consisting of three main components: question retrieval, question selection, and document retrieval. The experiments and analyses indicate that asking clarifying questions should be an essential part of a conversational system, resulting in high performance gain

MAPLE: Mobile App Prediction Leveraging Large Language model Embeddings

Despite the rapid advancement of mobile applications, predicting app usage remains a formidable challenge due to intricate user behaviours and ever-evolving contexts. To address these issues, this paper introduces the Mobile App Prediction Leveraging Large Language Model Embeddings (MAPLE) model. This innovative approach utilizes Large Language Models (LLMs) to predict app usage accurately. Rigorous testing on two public datasets highlights MAPLE's capability to decipher intricate patterns and comprehend user contexts. These robust results confirm MAPLE's versatility and resilience across various scenarios. While its primary design caters to app prediction, the outcomes also emphasize the broader applicability of LLMs in different domains. Through this research, we emphasize the potential of LLMs in app usage prediction and suggest their transformative capacity in modelling human behaviours across diverse fields

Proceedings of the International Workshop on EuroPLOT Persuasive Technology for Learning, Education and Teaching (IWEPLET 2013)

"This book contains the proceedings of the International Workshop on EuroPLOT Persuasive Technology for Learning, Education and Teaching (IWEPLET) 2013 which was held on 16.-17.September 2013 in Paphos (Cyprus) in conjunction with the EC-TEL conference. The workshop and hence the proceedings are divided in two parts: on Day 1 the EuroPLOT project and its results are introduced, with papers about the specific case studies and their evaluation. On Day 2, peer-reviewed papers are presented which address specific topics and issues going beyond the EuroPLOT scope. This workshop is one of the deliverables (D 2.6) of the EuroPLOT project, which has been funded from November 2010 – October 2013 by the Education, Audiovisual and Culture Executive Agency (EACEA) of the European Commission through the Lifelong Learning Programme (LLL) by grant #511633. The purpose of this project was to develop and evaluate Persuasive Learning Objects and Technologies (PLOTS), based on ideas of BJ Fogg. The purpose of this workshop is to summarize the findings obtained during this project and disseminate them to an interested audience. Furthermore, it shall foster discussions about the future of persuasive technology and design in the context of learning, education and teaching. The international community working in this area of research is relatively small. Nevertheless, we have received a number of high-quality submissions which went through a peer-review process before being selected for presentation and publication. We hope that the information found in this book is useful to the reader and that more interest in this novel approach of persuasive design for teaching/education/learning is stimulated. We are very grateful to the organisers of EC-TEL 2013 for allowing to host IWEPLET 2013 within their organisational facilities which helped us a lot in preparing this event. I am also very grateful to everyone in the EuroPLOT team for collaborating so effectively in these three years towards creating excellent outputs, and for being such a nice group with a very positive spirit also beyond work. And finally I would like to thank the EACEA for providing the financial resources for the EuroPLOT project and for being very helpful when needed. This funding made it possible to organise the IWEPLET workshop without charging a fee from the participants.

Supporting lay users in privacy decisions when sharing sensitive data

The first part of the thesis focuses on assisting users in choosing their privacy settings, by using machine learning to derive the optimal set of privacy settings for the user. In contrast to other work, our approach uses context factors as well as individual factors to provide a personalized set of privacy settings. The second part consists of a set of intelligent user interfaces to assist the users throughout the complete privacy journey, from defining friend groups that allow targeted information sharing; through user interfaces for selecting information recipients, to find possible errors or unusual settings, and to refine them; up to mechanisms to gather in-situ feedback on privacy incidents, and investigating how to use these to improve a user’s privacy in the future. Our studies have shown that including tailoring the privacy settings significantly increases the correctness of the predicted privacy settings; whereas the user interfaces have been shown to significantly decrease the amount of unwanted disclosures.Insbesondere nach den jüngsten Datenschutzskandalen in sozialen Netzwerken wird der Datenschutz für Benutzer immer wichtiger. Obwohl die meisten Benutzer behaupten Wert auf Datenschutz zu legen, verhalten sie sich online allerdings völlig anders: Sie lassen die meisten Datenschutzeinstellungen der online genutzten Dienste, wie z. B. von sozialen Netzwerken oder Diensten zur Standortfreigabe, unberührt und passen sie nicht an ihre Datenschutzanforderungen an. In dieser Arbeit werde ich einen Ansatz zur Lösung dieses Problems vorstellen, der auf zwei verschiedenen Säulen basiert. Der erste Teil konzentriert sich darauf, Benutzer bei der Auswahl ihrer Datenschutzeinstellungen zu unterstützen, indem maschinelles Lernen verwendet wird, um die optimalen Datenschutzeinstellungen für den Benutzer abzuleiten. Im Gegensatz zu anderen Arbeiten verwendet unser Ansatz Kontextfaktoren sowie individuelle Faktoren, um personalisierte Datenschutzeinstellungen zu generieren. Der zweite Teil besteht aus einer Reihe intelligenter Benutzeroberflächen, die die Benutzer in verschiedene Datenschutzszenarien unterstützen. Dies beginnt bei einer Oberfläche zur Definition von Freundesgruppen, die im Anschluss genutzt werden können um einen gezielten Informationsaustausch zu ermöglichen, bspw. in sozialen Netzwerken; über Benutzeroberflächen um die Empfänger von privaten Daten auszuwählen oder mögliche Fehler oder ungewöhnliche Datenschutzeinstellungen zu finden und zu verfeinern; bis hin zu Mechanismen, um In-Situ- Feedback zu Datenschutzverletzungen zum Zeitpunkt ihrer Entstehung zu sammeln und zu untersuchen, wie diese verwendet werden können, um die Privatsphäreeinstellungen eines Benutzers anzupassen. Unsere Studien haben gezeigt, dass die Verwendung von individuellen Faktoren die Korrektheit der vorhergesagten Datenschutzeinstellungen erheblich erhöht. Es hat sich gezeigt, dass die Benutzeroberflächen die Anzahl der Fehler, insbesondere versehentliches Teilen von Daten, erheblich verringern

A Survey of Prediction and Classification Techniques in Multicore Processor Systems

In multicore processor systems, being able to accurately predict the future provides new optimization opportunities, which otherwise could not be exploited. For example, an oracle able to predict a certain application\u27s behavior running on a smart phone could direct the power manager to switch to appropriate dynamic voltage and frequency scaling modes that would guarantee minimum levels of desired performance while saving energy consumption and thereby prolonging battery life. Using predictions enables systems to become proactive rather than continue to operate in a reactive manner. This prediction-based proactive approach has become increasingly popular in the design and optimization of integrated circuits and of multicore processor systems. Prediction transforms from simple forecasting to sophisticated machine learning based prediction and classification that learns from existing data, employs data mining, and predicts future behavior. This can be exploited by novel optimization techniques that can span across all layers of the computing stack. In this survey paper, we present a discussion of the most popular techniques on prediction and classification in the general context of computing systems with emphasis on multicore processors. The paper is far from comprehensive, but, it will help the reader interested in employing prediction in optimization of multicore processor systems

Digitalization of the individual : consequences, design, and behavior

In the past decades, digitalization has increasingly influenced our daily lives and habits in almost all areas and has even become indispensable for individuals, organizations, and society. The interactions between individuals and organizations have changed significantly as digitalization extends the boundaries of organizations to the point at which it affects individuals. Consequently, new research efforts and better understanding are essential to understand how the behavior of individuals is affected by the use of digital technologies, how customers demands change, and how the purchasing process of organizations needs to be adapted. Currently, the literature on digital transformation is mainly treating the organizational perspective. Nevertheless, organizations should not neglect the individual perspective as it is essential to understand customer needs and their consequences affected by digital technologies. Matt et al. (2019)1 present a holistic research framework with three research perspectives for the digitalization of the individual. This framework encompasses the behavior of individuals, the design of information systems, and the consequences that digitalization entails. Additionally, this research framework suggests that a digitized individual can take on different roles. The dissertation uses this framework of Matt et al. (2019)1 to structure and classify the covered contents and research objectives. The aim of this dissertation is to contribute to a comprehensive overview for organizations to understand their customers concerns regarding digital technologies, which design options they have to address these concerns, and how it influences their behavior to realize the potential of the technologies or reduce their harms. Therefore, this work applies pluralistic methodological approaches (qualitative methods, e.g., semi-structured interviews and qualitative content analysis, and quantitative methods, e.g., quantitative decision models and data collection from online questionnaires). With that, the dissertation provides novel insights for organizations to better implement digital technologies by regarding the consequences for individuals and the behavior of individuals. First, to contribute to an understanding of the negative consequences digitalization can bring along for individuals, part A of this dissertation presents two research articles that focus on the concerns of individuals. The research papers P1 and P2 show in two different domains what individuals are concerned about when using digital technologies and what prevents individuals from using them. Therefore, this dissertation presents knowledge about the fears and concerns of the individuals have and offers starting points to develop responsible and transparent digital technologies that address the concerns of the individuals. Second, to contribute to design approaches for information systems that enable organizations to increase customer satisfaction with digital products and services, part B presents design approaches that organizations can use to address individuals perceived consequences and change their behavior using digital technologies. Both research papers in part B present quantitative decision models as decision support for organizations. This dissertation offers two design approaches that provide organizations with information on designing technologies to serve digitized individuals and foster them better to make well-founded decisions when introducing digital technologies. Third, to contribute to the understanding of why and how individuals behave in certain ways and how this behavior can be influenced, Part C examines the behavior of individuals when using digital technologies. Research paper P5 develops a metric to better explore the privacy paradox. With that, this dissertation offers a basis, especially to researchers and individuals, to prevent unwanted behavior when using digital technologies. To sum up, this dissertation contributes to scientific knowledge in research on the digitalization of the individual and thus addresses a subject of fundamental importance in this digital age. The models and approaches developed in this dissertation explore ways to improve conditions for the digitized individual at all three research perspectives with equal regard for the individual as itself and the individual as a customer.In den vergangenen Jahrzehnten hat die Digitalisierung zunehmend unseren Alltag und unsere Gewohnheiten in nahezu Bereichen des Lebens beeinflusst und ist damit für Individuen, Organisationen und die Gesellschaft unverzichtbar geworden. So hat sich die Beziehung zwischen Individuen und Organisationen erheblich verändert, da die Digitalisierung die Organisationsgrenzen aufweicht und ihre Kund:innen mehr integriert. Folglich sind neue Forschungsanstrengungen und ein besseres Verständnis erforderlich, um nachvollziehen zu können, wie das Verhalten von Individuen durch den Einsatz digitaler Technologien beeinflusst wird, wie sich die Anforderungen von Kund:innen ändern und wie der Kaufprozess von Organisationen angepasst werden muss. Derzeit wird in der Literatur zum Themengebiet der digitalen Transformation hauptsächlich die organisationale Perspektive behandelt. Nichtsdestotrotz sollten Organisationen die individuelle Perspektive nicht vernachlässigen. Sie ist grundlegend, um die Kund:innenbedürfnisse, die durch digitale Technologien beeinflusst werden, und deren Folgen zu verstehen. Matt et al. (2019) stellen einen ganzheitlichen Forschungsrahmen mit drei Forschungsperspektiven für die Digitalisierung des Individuums vor. Dieser umfasst das Verhalten der Individuen, die Gestaltung von Informationssystemen und die Konsequenzen, die die Digitalisierung für Individuen mit sich bringen kann. Zusätzlich zeigt dieser, dass ein digitalisiertes Individuum verschiedene Rollen einnehmen kann. Die Dissertation nutzt das Framework von Matt et al. (2019), um die Inhalte und Forschungsziele zu strukturieren und einzuordnen. Ziel dieser Dissertation ist es, einen Beitrag zu einem umfassenden Überblick für Organisationen zu leisten, um die Individuen im Zuge der Digitalisierung zu verstehen. Dabei wird untersucht, welche Bedenken ihre Kund:innen in Bezug auf digitale Technologien haben, welche Gestaltungsmöglichkeiten sie haben, um diese Bedenken zu adressieren, und wie es das Verhalten von Kund:innen beeinflusst. Dadurch können sie das Potenzial dieser Technologien realisieren oder ihre Schäden reduzieren. Diese Arbeit wendet eine Vielzahl an methodischen Ansätzen an (qualitative Methoden, z.B. halbstrukturierte Interviews und qualitative Inhaltsanalyse, und quantitative Methoden, z.B. quantitative Entscheidungsmodelle und Datenerhebung aus Online-Fragebögen). Damit liefert die Dissertation neue Erkenntnisse für Organisationen, um digitale Technologien besser zu implementieren, indem sie die Konsequenzen für Individuen und das Verhalten von Individuen betrachtet. Um erstens einen Beitrag zum besseren Verständnis der negativen Folgen, die die Digitalisierung für den Einzelnen mit sich bringen kann, zu leisten, umfasst Teil A dieser Dissertation zwei Forschungsartikel, die sich mit den Bedenken des Einzelnen beschäftigen. Die Forschungsartikel P1 und P2 zeigen in zwei unterschiedlichen Bereichen, welche Bedenken Individuen bei der Nutzung digitaler Technologien haben und was Individuen davon abhält, diese zu nutzen. Daher präsentiert diese Dissertation Wissen über die Ängste und Bedenken der Individuen und bietet Ansatzpunkte, um verantwortungsvolle und transparente digitale Technologien zu entwickeln. Um zweitens einen Beitrag zu Gestaltungsansätzen für Informationssysteme zu leisten, werden in Teil B Gestaltungsansätze vorgestellt, mit denen Organisationen die wahrgenommenen Konsequenzen für Individuen adressieren und das Verhalten im Umgang mit digitalen Technologien ändern können. Diese ermöglichen es Organisationen die Kund:innenzufriedenheit bei der Nutzung von digitalen Produkten und Dienstleistungen zu erhöhen. Beide Forschungsarbeiten in Teil B stellen quantitative Entscheidungsmodelle als Entscheidungshilfe für Organisationen vor. Diese Dissertation bietet zwei Gestaltungsansätze, die Organisationen Informationen zur Gestaltung von Informationssystemen liefern und sie dabei unterstützen, fundierte Entscheidungen bei der Einführung digitaler Technologien zu treffen. Drittens, um zum Verständnis beizutragen, warum und wie sich Individuen auf bestimmte Weise verhalten und wie dieses Verhalten beeinflusst werden kann, wird in Teil C das Verhalten von Individuen bei der Nutzung digitaler Technologien untersucht. P5 entwickelt eine Metrik, um das Privacy-Paradoxon besser zu erforschen. Damit bietet diese Dissertation eine Grundlage, insbesondere für Forscherinnen und Forscher sowie Individuen, um unerwünschtes Verhalten bei der Nutzung digitaler Technologien zu verhindern. Zusammenfassend lässt sich sagen, dass diese Dissertation wissenschaftliche Erkenntnisse zur Erforschung der Digitalisierung des Individuums leistet und damit ein Thema von grundlegender Bedeutung im digitalen Zeitalter behandelt. Die in dieser Dissertation entwickelten Modelle und Ansätze zeigen Wege auf, wie die Bedingungen für das digitalisierte Individuum auf allen drei Forschungsperspektiven verbessert werden können

Recordings of digital media life: Advancing (qualitative) media diaries as a method

In times of digitalization, analyzing the highly complex media practices and mediated life worlds of individuals has become highly challenging, both in theoretical and methodological terms. From an empirical point of view, diary methods, and particularly qualitative (media) diaries, bear a great potential to gain access to these media practices and analyze them within the contexts of people’s everyday lives. In this article, we propose that it is fruitful to apply the characteristics of real diaries to research settings and consider them when designing diary studies as a researcher. Doing so can help to collect more “genuine” data and get a more holistic and adequate picture of digital media life. These characteristics comprise: (1) authenticity and naturalness, (2) autonomy in design, (3) multimodality and materiality, (4) intrinsic motivation, (5) functionalities of diary keeping, (6) continuity and periodicity, as well as (7) inferences about cultural and social conditions. We provide suggestions for implementing these characteristics in qualitative diary studies, and discuss the empirical challenges accompanying this approach

Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding

Visually-situated language is ubiquitous -- sources range from textbooks with diagrams to web pages with images and tables, to mobile apps with buttons and forms. Perhaps due to this diversity, previous work has typically relied on domain-specific recipes with limited sharing of the underlying data, model architectures, and objectives. We present Pix2Struct, a pretrained image-to-text model for purely visual language understanding, which can be finetuned on tasks containing visually-situated language. Pix2Struct is pretrained by learning to parse masked screenshots of web pages into simplified HTML. The web, with its richness of visual elements cleanly reflected in the HTML structure, provides a large source of pretraining data well suited to the diversity of downstream tasks. Intuitively, this objective subsumes common pretraining signals such as OCR, language modeling, image captioning. In addition to the novel pretraining strategy, we introduce a variable-resolution input representation and a more flexible integration of language and vision inputs, where language prompts such as questions are rendered directly on top of the input image. For the first time, we show that a single pretrained model can achieve state-of-the-art results in six out of nine tasks across four domains: documents, illustrations, user interfaces, and natural images