Using the information seeker to elicit construct models for search engine evaluation

Users’ internal representations of their interactions with systems are often termed ‘mental models’, and for successful system use, the users’ mental models and system designers’ conceptual models of the tools should be congruent. This study explores a method for non-biased determination of the user’s subconscious view of Internet search engines, in order to derive a mental model comprising those aspects of the systems of importance to the users. The investigation utilises a repertory grid approach in combination with laddering technique, the latter being based on the cause and effect style of mental model development. The detailed qualitative analysis of the data determined through use of laddering interviews is presented here in the development of a mental model comprising three strata. The main hierarchical stratum of the model conveys the interrelations between basic system description, evaluative description, and the key evaluations of ease, efficiency, effort and effectiveness. Two additional strata relating to the perceived process and the experience of emotion are also discussed. The conjunction of the procedural elements with the key evaluations is of particular significance, and further research proposes the extension of this to provide a framework for search engine evaluation

Identifying Unclear Questions in Community Question Answering Websites

Thousands of complex natural language questions are submitted to community question answering websites on a daily basis, rendering them as one of the most important information sources these days. However, oftentimes submitted questions are unclear and cannot be answered without further clarification questions by expert community members. This study is the first to investigate the complex task of classifying a question as clear or unclear, i.e., if it requires further clarification. We construct a novel dataset and propose a classification approach that is based on the notion of similar questions. This approach is compared to state-of-the-art text classification baselines. Our main finding is that the similar questions approach is a viable alternative that can be used as a stepping stone towards the development of supportive user interfaces for question formulation.Comment: Proceedings of the 41th European Conference on Information Retrieval (ECIR '19), 201

Searching for Authoritative Documents in Knowledge-Base Communities

Knowledge-based communities are popular Web-based tools that allow members to share and seek knowledge globally. However, research on how to search effectively within such knowledge repositories is scant. In this paper we study the problem of finding authoritative documents for user queries within a knowledge-based community. Unlike prior research on the ranking function design which considers only content or hyperlink information, we leverage the social network information embedded in the rich social media, in addition to content, to design novel ranking strategies. Using the Knowledge Adoption Model as the guiding theoretical framework, we design features that gauge the two major factors affecting users’ knowledge adoption decisions: argument quality (AQ) and source credibility (SC). We design two ranking strategies that blend these two sources of evidence with the content-based relevance judgment. A preliminary study using a real world knowledge-based community showed that both AQ and SC features improved search effectiveness

Personalized Memory Transfer for Conversational Recommendation Systems

Dialogue systems are becoming an increasingly common part of many users\u27 daily routines. Natural language serves as a convenient interface to express our preferences with the underlying systems. In this work, we implement a full-fledged Conversational Recommendation System, mainly focusing on learning user preferences through online conversations. Compared to the traditional collaborative filtering setting where feedback is provided quantitatively, conversational users may only indicate their preferences at a high level with inexact item mentions in the form of natural language chit-chat. This makes it harder for the system to correctly interpret user intent and in turn provide useful recommendations to the user. To tackle the ambiguities in natural language conversations, we propose Personalized Memory Transfer (PMT) which learns a personalized model in an online manner by leveraging a key-value memory structure to distill user feedback directly from conversations. This memory structure enables the integration of prior knowledge to transfer existing item representations/preferences and natural language representations. We also implement a retrieval based response generation module, where the system in addition to recommending items to the user, also responds to the user, either to elicit more information regarding the user intent or just for a casual chit-chat. The experiments were conducted on two public datasets and the results demonstrate the effectiveness of the proposed approach

The impact of work seeker support platforms on the development of South Africa's unemployed youth

Youth unemployment remains an enduring and significant challenge in South Africa, with 43.2 % of people aged 15-34 and 59% aged 15-24 remaining unemployed, respectively. Similarly, economic discouragement among young people is on the rise. Micro-level barriers contribute significantly to the inability to access employment opportunities. These include the low skills levels of many young South Africans, the high costs of job-seeking, a lack of social capital, a lack of access to relevant job-seeker information, as well as the adverse mental health impacts of alienation, poverty and unemployment. With the rise of the Fourth Industrial Revolution and the rise of ICT for Development (ICT4D) interventions, several digital solutions have been developed in South Africa. These attempt to provide low-cost, scalable solutions to youth unemployment by addressing some of the barriers that young people experience. Despite the increased prevalence of such digital interventions, the degree to which they are capable of engaging and transforming the lives of the unemployed youth they target remains unclear. With increasing investments into 4IR interventions to address youth unemployment, closer examination is required. Accordingly, this study appraises one digital work seeker support platform in South Africa that provides skills matching and development opportunities to unemployed youth. The study focuses specifically on their experience of the platform. It uses post phenomenological constructs to analyse how young unemployed South Africans interpret the digital intervention and examines how these interpretations promote or inhibit their sense of agency and wellbeing. The findings suggest that digital youth employment interventions can inadvertently exacerbate some of the existing barriers, while also providing insight into how ICT4D interventions may be reimagined to address some of the factors that drive economic discouragement among young people

Understanding and conceptualising the document triage process through information seekers' visual and navigational attention

Information, is a valuable commodity and its effective use is a vital part of everyday life. With the advancements of the internet and the increasing accessibility to it, the location of information is no longer the primary concern of information seekers. Digitisation techniques have made a wide variety of documents available on-line, and more and more publications are being published in electronic form simultaneously to their physical counterpart. The largest challenge currently facing information seekers is that of locating the correct information from the abundance available to them. Whenever a search query is made, the user is inundated with multiple options of documents to choose from. These documents are all deemed to have some relevance to the query produced by using an information retrieval algorithm. Thus far, automatic support has only been provided until the document retrieval level. The user is then left to search through the result set, mostly unaided, by the system he is using. In order to facilitate support for the users, a solid understanding of the information seeker's behaviours during this triage process is vital. Thus far, research into the behaviour of information seekers during the specific triage behaviour is limited. Even more limited however, is the evidence reporting the visual attention of the users. Since the triage process is highly visual, this important element need to be thoroughly evidence before accurately conceptualising the entire process. For this reason, this thesis aims to investigate the visual attention of information seekers during the document triage process. This will inform the modelling and conceptualisation of information seekers' behaviour during triage. In turn, this can be used to inform the design of supportive software. The thesis contains a review of related research and identifies the gaps that needs further investigation. From these, a series of user studies are then conducted on document triage. These in turn, facilitate the formulation and discussion of 2 document triage models and measurements to record the effectiveness of document triage. We study the visual attention of information seekers in four lab based studies, eliciting their exact gaze and focus details. We expand current research in the information seeking domain by reporting on findings from users' triage activities on small screen devices and when under time constraints. Furthermore, a high level diary study, gives us richer data on participants' triage activities over a larger period of time in their natural surroundings. All the studies are brought together to elicit requirements and measurements to understand system and user efficiency during each stage of the triage process