Predicting re-finding activity and difficulty

In this study, we address the problem of identifying if users are attempting to re-find information and estimating the level of difficulty of the re- finding task. We propose to consider the task information (e.g. multiple queries and click information) rather than only queries. Our resultant prediction models are shown to be significantly more accurate (by 2%) than the current state of the art. While past research assumes that previous search history of the user is available to the prediction model, we examine if re-finding detection is possible without access to this information. Our evaluation indicates that such detection is possible, but more challenging. We further describe the first predictive model in detecting re-finding difficulty, showing it to be significantly better than existing approaches for detecting general search difficulty

Impact of Covid-19 Pandemic on User Search Behavior: A Case Study of Postgraduate Students

Relevance Feedback (RF) is crucial for building a user profile which is a fundamental element of different intelligent systems such as information retrieval, information filtering, and personalization. RF is affected by a number of contextual factors such as mood, stress level, and sentimental state of the user. Covid-19 pandemic imposed dramatic changes to the user environment as well as the search context. This paper investigates user’s search behaviour to identify the differences in the behavior between the contexts before and during the Covid-19 pandemic. This can be practically translated into identifying the differences in the relationship between the implicit feedback and the explicit relevance level between the two contexts. For this purpose, we conducted three user studies (i) Pre-COVID-19,  (ii) Mid COVID-19 and (iii) after Covid-19. A user study was conducted on the same group of users on the three user studies. The Pre-COVID-19 user study took place before the pandemic started and the Mid-COVID-19 user study took place three months after the beginning of the pandemic. After Covid-19 stage took place after 18 months of the pandemic. A linear regression model was developed for each user study using IBM-SPSS. The analysis showed a significant variation in the user behavior between the two studies due to the COVID-19 context and its impact on user search behaviour. Also, two new RF parameters in Mid-COVID-19 were shown to have a significant relationship with the explicit user interest which were Mouse Clicks and Page/Down strikes. Furthermore, the comparison between the two models showed that the second regression model achieved a higher accuracy level that is attributed to the common behavioral change imposed by the pandemic

Fuzzy rule based profiling approach for enterprise information seeking and retrieval

With the exponential growth of information available on the Internet and various organisational intranets there is a need for profile based information seeking and retrieval (IS&R) systems. These systems should be able to support users with their context-aware information needs. This paper presents a new approach for enterprise IS&R systems using fuzzy logic to develop task, user and document profiles to model user information seeking behaviour. Relevance feedback was captured from real users engaged in IS&R tasks. The feedback was used to develop a linear regression model for predicting document relevancy based on implicit relevance indicators. Fuzzy relevance profiles were created using Term Frequency and Inverse Document Frequency (TF/IDF) analysis for the successful user queries. Fuzzy rule based summarisation was used to integrate the three profiles into a unified index reflecting the semantic weight of the query terms related to the task, user and document. The unified index was used to select the most relevant documents and experts related to the query topic. The overall performance of the system was evaluated based on standard precision and recall metrics which show significant improvements in retrieving relevant documents in response to user queries

Utilizing Re-finding for Personalized Information Retrieval

Individuals often use search engines to return to web pages they have previously visited. This behaviour, called refinding, accounts for about 38 % of all queries. While researchers have shown how re-finding differs from traditionally studied new-findings, research on how to predict and utilize re-finding is limited. In this paper we explore refinding for personalized search. We compared three machine learning algorithms (decision trees, Bayesian multinomial regression and support vector machines) to identify refindings. We then propose several re-ranking methods to utilize the prediction, including promoting predicted re-finding URLs and combining re-finding prediction with relevance estimation. The experimental results demonstrate that using re-finding predictions can improve retrieval performance for personalized search

Identification of re-finding tasks and search difficulty

We address the problem of identifying if users are attempting to re-find information and estimating the level of difficulty of the re-finding task. Identifying re-finding tasks and detecting search difficulties will enable search engines to respond dynamically to the search task being undertaken. To this aim, we conduct user studies and query log analysis to make a better understanding of re-finding tasks and search difficulties. Computing features particularly gathered in our user studies, we generate training sets from query log data, which is used for constructing automatic identification (prediction) models. Using machine learning techniques, our built re-finding identification model, which is the first model at the task level, could significantly outperform the existing query-based identifications. While past research assumes that previous search history of the user is available to the prediction model, we examine if re-finding detection is possible without access to this information. Our evaluation indicates that such detection is possible, but more challenging. We further describe the first predictive model in detecting re-finding difficulty, showing it to be significantly better than existing approaches for detecting general search difficulty. We also analyze important features for both identifications of re-finding and difficulties. Next, we investigate detailed identification of re-finding tasks and difficulties in terms of the type of the vertical document to be re-found. The accuracy of constructed predictive models indicates that re-finding tasks are indeed distinguishable across verticals and in comparison to general search tasks. This illustrates the requirement of adapting existing general search techniques for the re-finding context in terms of presenting vertical-specific results. Despite the overall reduction of accuracy in predictions independent of the original search of the user, it appears that identifying “image re-finding” is less dependent on such past information. Investigating the real-time prediction effectiveness of the models show that predicting ``image'' document re-finding obtains the highest accuracy early in the search. Early predictions would benefit search engines with adaptation of search results during re-finding activities. Furthermore, we study the difficulties in re-finding across verticals given some of the established indications of difficulties in the general web search context. In terms of user effort, re-finding “image” vertical appears to take more effort in terms of number of queries and clicks than other investigated verticals, while re-finding “reference” documents seems to be more time consuming when there is a longer time gap between the re-finding and corresponding original search. Exploring other features suggests that there could be particular difficulty indications for the re-finding context and specific to each vertical. To sum up, this research investigates the issue of effectively supporting users with re-finding search tasks. To this end, we have identified features that allow for more accurate distinction between re-finding and general tasks. This will enable search engines to better adapt search results for the re-finding context and improve the search experience of the users. Moreover, features indicative of similar/different and easy/difficult re-finding tasks can be employed for building balanced test environments, which could address one of the main gaps in the re-finding context