Understanding Mobile Search Task Relevance and User Behaviour in Context

Improvements in mobile technologies have led to a dramatic change in how and when people access and use information, and is having a profound impact on how users address their daily information needs. Smart phones are rapidly becoming our main method of accessing information and are frequently used to perform `on-the-go' search tasks. As research into information retrieval continues to evolve, evaluating search behaviour in context is relatively new. Previous research has studied the effects of context through either self-reported diary studies or quantitative log analysis; however, neither approach is able to accurately capture context of use at the time of searching. In this study, we aim to gain a better understanding of task relevance and search behaviour via a task-based user study (n=31) employing a bespoke Android app. The app allowed us to accurately capture the user's context when completing tasks at different times of the day over the period of a week. Through analysis of the collected data, we gain a better understanding of how using smart phones on the go impacts search behaviour, search performance and task relevance and whether or not the actual context is an important factor.Comment: To appear in CHIIR 2019 in Glasgow, U

Study of Relevance and Effort across Devices

Relevance judgements are essential for designing information retrieval systems. Traditionally, judgements have been judgements have been gathered via desktop interfaces. However, with the rise in popularity of smaller devices for information access, it has become imperative to investigate whether desktop based judgements are different from judgements gathered using mobiles. Recently, user effort and document usefulness have also emerged as important dimensions to optimize and evaluate information retrieval systems. Since existing work is limited to desktops, it remains to be seen how these judgements are affected by user’s search device. In this paper, we address these shortcomings by collecting and analyzing relevance, usefulness and effort judgements on mobiles and desktops. Analysis of these judgements indicates that high agreement rate between desktop and mobile judges for relevance, followed by usefulness and findability. We also found that desktop judges are likely to spend more time and examine documents in greater depth on non-relevant/notuseful/difficult documents compared to mobile judges. Based on our findings, we suggest that relevance judgements should be gathered via desktops and effort judgements should be collected on each device independently

Target Apps Selection: Towards a Unified Search Framework for Mobile Devices

With the recent growth of conversational systems and intelligent assistants such as Apple Siri and Google Assistant, mobile devices are becoming even more pervasive in our lives. As a consequence, users are getting engaged with the mobile apps and frequently search for an information need in their apps. However, users cannot search within their apps through their intelligent assistants. This requires a unified mobile search framework that identifies the target app(s) for the user's query, submits the query to the app(s), and presents the results to the user. In this paper, we take the first step forward towards developing unified mobile search. In more detail, we introduce and study the task of target apps selection, which has various potential real-world applications. To this aim, we analyze attributes of search queries as well as user behaviors, while searching with different mobile apps. The analyses are done based on thousands of queries that we collected through crowdsourcing. We finally study the performance of state-of-the-art retrieval models for this task and propose two simple yet effective neural models that significantly outperform the baselines. Our neural approaches are based on learning high-dimensional representations for mobile apps. Our analyses and experiments suggest specific future directions in this research area.Comment: To appear at SIGIR 201

Predictive Analytics of E-Commerce Search Behavior for Conversion

This study explores online customer search behavior on a large e-commerce website—Walmart.com. In order to more accurately predict customer purchase conversion based on their search behavior, we adopt a modern machine-learning technique, random forest, as well as logistic regression to develop two computational models. We also integrate information retrieval literature to propose metrics to quantify online consumers’ search behavior. Results show that the random forest model performs better with a very high accuracy rate (76%) in predicting customers who will purchase the item they browsed. Among all the predictors, page and session dwell time, user type, click entropy, and click position are the strongest influential factors for the conversion behavior. The findings suggest that, with the enhanced metrics and modeling approaches, search behavior could offer strong cues about customers’ purchasing decision. Additionally, the findings also suggest operational implications about how to accommodate and induce the desired search behavior with the e-commerce website

Supporting cross-device web search with social navigation-based mobile touch interactions

The wide adoption of smartphones eliminates the time and location barriers for people’s daily information access, but also limits users’ information exploration activities due to the small mobile screen size. Thus, cross-device web search, where people initialize information needs on one device but complete them on another device, is frequently observed in modern search engines, especially for exploratory information needs. This paper aims to support the cross-device web search, on top of the commonly used context-sensitive retrieval framework, for exploratory tasks. To better model users’ search context, our method not only utilizes the search history (query history and click-through) but also employs the mobile touch interactions (MTI) on mobile devices. To be more specific, we combine MTI’s ability of locating relevant subdocument content [10] with the idea of social navigation that aggregates MTIs from other users who visit the same page. To demonstrate the effectiveness of our proposed approach, we designed a user study to collect cross-device web search logs on three different types of tasks from 24 participants and then compared our approach with two baselines: a traditional full text based relevance feedback approach and a self-MTI based subdocument relevance feedback approach. Our results show that the social navigation-based MTIs outperformed both baselines. A further analysis shows that the performance improvements are related to several factors, including the quality and quantity of click-through documents, task types and users’ search conditions