Report on the Second International Workshop on the Evaluation on Collaborative Information Seeking and Retrieval (ECol'2017 @ CHIIR)

The 2nd workshop on the evaluation of collaborative information retrieval and seeking (ECol) was held in conjunction with the ACM SIGIR Conference on Human Information Interaction & Retrieval (CHIIR) in Oslo, Norway. The workshop focused on discussing the challenges and difficulties of researching and studying collaborative information retrieval and seeking (CIS/CIR). After an introductory and scene setting overview of developments in CIR/CIS, participants were challenged with devising a range of possible CIR/CIS tasks that could be used for evaluation purposes. Through the brainstorming and discussions, valuable insights regarding the evaluation of CIR/CIS tasks become apparent ? for particular tasks efficiency and/or effectiveness is most important, however for the majority of tasks the success and quality of outcomes along with knowledge sharing and sense-making were most important ? of which these latter attributes are much more difficult to measure and evaluate. Thus the major challenge for CIR/CIS research is to develop methods, measures and methodologies to evaluate these high order attributes

Beyond Traditional Collaborative Search: Understanding the Effect of Awareness on Multi-Level Collaborative Information Retrieval

Although there has been a great deal of research into Collaborative Information Retrieval (CIR) and Collaborative Information Seeking (CIS), the majority has assumed that team members have the same level of unrestricted access to underlying information. However, observations from different domains (e.g. healthcare, business, etc.) have suggested that collaboration sometimes involves people with differing levels of access to underlying information. This type of scenario has been referred to as Multi-Level Collaborative Information Retrieval (MLCIR). To the best of our knowledge, no studies have been conducted to investigate the effect of awareness, an existing CIR/CIS concept, on MLCIR. To address this gap in current knowledge, we conducted two separate user studies using a total of 5 different collaborative search interfaces and 3 information access scenarios. A number of Information Retrieval (IR), CIS and CIR evaluation metrics, as well as questionnaires were used to compare the interfaces. Design interviews were also conducted after evaluations to obtain qualitative feedback from participants. Results suggested that query properties such as time spent on query, query popularity and query effectiveness could allow users to obtain information about team’s search performance and implicitly suggest better queries without disclosing sensitive data. Besides, having access to a history of intersecting viewed, relevant and bookmarked documents could provide similar positive effect as query properties. Also, it was found that being able to easily identify different team members and their actions is important for users in MLCIR. Based on our findings, we provide important design recommendations to help develop new CIR and MLCIR interfaces

De la recherche sociale d'information à la recherche collaborative d'information

International audienceIn this paper, we explain social information retrieval (SIR) and collaborative information retrieval (CIR). We see SIR as a way of knowing who to collaborate with in resolving an information problem while CIR entails the process of mutual understanding and solving of an information problem among collaborators. We are interested in the transition from SIR to CIR hence we developed a communication model to facilitate knowledge sharing during CIR

Collaborative knowledge creation and management in information retrieval

International audienceThe final goal of Information Retrieval (IR) is knowledge production. However, it has been argued that knowledge production is not an individual effort but a collaborative effort. Collaboration in information retrieval is geared towards knowledge sharing and creation of new knowledge by users. This paper discusses Collaborative Information Retrieval (CIR) and how it culminates to knowledge creation. It explains how created knowledge is organized and structured. It describes a functional architecture for the development of a CIR prototype called MECOCIR. Some of the features of the prototype are presented as well as how they facilitate collaborative knowledge exploitation. Knowledge creation is explained through the knowledge conversion/transformation processes proposed by Nonaka and CIR activities that facilitate these processes are high-lighted and discusse

Collaborative knowledge creation and management in information retrieval

International audienceThe final goal of Information Retrieval (IR) is knowledge production. However, it has been argued that knowledge production is not an individual effort but a collaborative effort. Collaboration in information retrieval is geared towards knowledge sharing and creation of new knowledge by users. This paper discusses Collaborative Information Retrieval (CIR) and how it culminates to knowledge creation. It explains how created knowledge is organized and structured. It describes a functional architecture for the development of a CIR prototype called MECOCIR. Some of the features of the prototype are presented as well as how they facilitate collaborative knowledge exploitation. Knowledge creation is explained through the knowledge conversion/transformation processes proposed by Nonaka and CIR activities that facilitate these processes are high-lighted and discusse

Collaborative Information Retrieval among Economic Intelligence Actors

International audienceIn this paper we present collaborative information retrieval (CIR) for decision making. We take Economic Intelligence (EI) as application domain and we propose an approach for managing CIR among actors in the domain. EI process has information as a central object. In EI, we consider the translation of a decision making problem to information search problem; retrieval of needed information and processing this information to determine indicators that can aid in decision making. All these stages require collaboration among the EI actors. There are two major factors that affect collaboration: collaboration technology and culture of openness. The question to be posed is how can we manage collaborative information retrieval among EI actors taking into consideration the technological and cultural constraints? In attempting to answer this question, we propose a framework and a model for facilitating and managing collaborative information retrieval among EI actors

Collaborative Information Retrieval: Concepts, Models and Evaluation

International audienceRecent work have shown the potential of collaboration for solving complex or exploratory search tasks allowing to achieve synergic effects with respect to individual search, which is the prevalent information retrieval (IR) setting this last decade. This interactive multiuser context gives rise to several challenges in IR. One main challenge relies on the adaptation of IR techniques or models [8] in order to build algo-rithmic supports of collaboration distributing documents among users. The second challenge is related to the design of Collaborative Information Retrieval (CIR) models and their effectiveness evaluation since individual IR frameworks and measures do not totally fit with the collaboration paradigms. In this tutorial, we address the second challenge and present first a general overview of collaborative search introducing the main underlying notions. Then, we focus on related work dealing with collaborative ranking models and their effectiveness evaluation. Our primary objective is to introduce these notions by highlighting how and why they should be different from individual IR in order to give participants the main clues for investigating new research directions in this domain with a deep understanding of current CIR work

Towards quantifying the impact of non-uniform information access in collaborative information retrieval

The majority of research into Collaborative Information Retrieval (CIR) has assumed a uniformity of information access and visibility between collaborators. However in a number of real world scenarios, information access is not uniform between all collaborators in a team e.g. security, health etc. This can be referred to as Multi-Level Collaborative Information Retrieval (MLCIR). To the best of our knowledge, there has not yet been any systematic investigation of the effect of MLCIR on search outcomes. To address this shortcoming, in this paper, we present the results of a simulated evaluation conducted over 4 different non-uniform information access scenarios and 3 different collaborative search strategies. Results indicate that there is some tolerance to removing access to the collection and that there may not always be a negative impact on performance. We also highlight how different access scenarios and search strategies impact on search outcomes

Collaborative searching: social searching, together

Information Retrieval (IR) is typically an individual pursuit where an individual searcher will engage with a search system, working alone, until their information need is satisfied. Yet in the real world there are many scenarios, both work-related and related to leisure, entertainment or hobbies, where we want to search as part of a team, maybe even a group of only two people. Collaborative Information Retrieval (CIR) refers to technologies which support collaboration in the retrieval process.  In this presentation we will present both synchronous and asynchronous CIR as well as covering remote and co-located search, and the various combinations of these. In our work we are particularly interested in synchronous collaborative IR (SCIR) where a group of users work collectively to address some shared information need. We describe two systems we have developed to demonstrate SCIR, one on a gesture-based tabletop computer and the other on touch-based mobile devices (iPODs). We believe SCIR to be an important kind of social search even though the tools to support this are neither widespread nor reliable and are limited by the technology we currently use. Despite this we expect the importance of SCIR to grow as a consequential fallout of growth in social networks and the trend towards social networks now acting as platforms for applications, like search