Report of ECol Workshop Report on the First International Workshop on the Evaluation on Collaborative Information Seeking and Retrieval (ECol'2015)

Report of the ECol Workshop @ CIKM 2015The workshop on the evaluation of collaborative information retrieval and seeking (ECol) was held in conjunction with the 24 th Conference on Information and Knowledge Management (CIKM) in Melbourne, Australia. The workshop featured three main elements. First, a keynote on the main dimensions, challenges, and opportunities in collaborative information retrieval and seeking by Chirag Shah. Second, an oral presentation session in which four papers were presented. Third, a discussion based on three seed research questions: (1) In what ways is collaborative search evaluation more challenging than individual interactive information retrieval (IIIR) evaluation? (2) Would it be possible and/or useful to standardise experimental designs and data for collaborative search evaluation? and (3) For evaluating collaborative search, can we leverage ideas from other tasks such as diversified search, subtopic mining and/or e-discovery? The discussion was intense and raised many points and issues, leading to the proposition that a new evaluation track focused on collaborative information retrieval/seeking tasks, would be worthwhile

Report on the Second International Workshop on the Evaluation on Collaborative Information Seeking and Retrieval (ECol'2017 @ CHIIR)

The 2nd workshop on the evaluation of collaborative information retrieval and seeking (ECol) was held in conjunction with the ACM SIGIR Conference on Human Information Interaction & Retrieval (CHIIR) in Oslo, Norway. The workshop focused on discussing the challenges and difficulties of researching and studying collaborative information retrieval and seeking (CIS/CIR). After an introductory and scene setting overview of developments in CIR/CIS, participants were challenged with devising a range of possible CIR/CIS tasks that could be used for evaluation purposes. Through the brainstorming and discussions, valuable insights regarding the evaluation of CIR/CIS tasks become apparent ? for particular tasks efficiency and/or effectiveness is most important, however for the majority of tasks the success and quality of outcomes along with knowledge sharing and sense-making were most important ? of which these latter attributes are much more difficult to measure and evaluate. Thus the major challenge for CIR/CIS research is to develop methods, measures and methodologies to evaluate these high order attributes

Understanding the Impact of the Role Factor in Collaborative Information Retrieval

International audienceCollaborative information retrieval systems often rely on division of labor policies. Such policies allow work to be divided among collaborators with the aim of preventing redundancy and optimizing the synergic effects of collaboration. Most of the underlying methods achieve these goals by the means of explicit vs. implicit role-based mediation. In this paper, we investigate whether and how different factors, such as users' behavior, search strategies, and effectiveness, are related to role assignment within a collaborative exploratory search. Our main findings suggest that: (1) spontaneous and cohesive implicit roles might emerge during the collaborative search session implying users with no prior roles, and that these implicit roles favor the search precision, (2) role drift might occur alongside the search session performed by users with prior-assigned roles

QueryTogether: Enabling entity-centric exploration in multi-device collaborative search

Collaborative and co-located information access is becoming increasingly common. However, fairly little attention has been devoted to the design of ubiquitous computing approaches for spontaneous exploration of large information spaces enabling co-located collaboration. We investigate whether an entity-based user interface provides a solution to support co-located search on heterogeneous devices. We present the design and implementation of QueryTogether, a multi-device collaborative search tool through which entities such as people, documents, and keywords can be used to compose queries that can be shared to a public screen or specific users with easy touch enabled interaction. We conducted mixed-methods user experiments with twenty seven participants (nine groups of three people), to compare the collaborative search with QueryTogether to a baseline adopting established search and collaboration interfaces. Results show that QueryTogether led to more balanced contribution and search engagement. While the overall s-recall in search was similar, in the QueryTogether condition participants found most of the relevant results earlier in the tasks, and for more than half of the queries avoided text entry by manipulating recommended entities. The video analysis demonstrated a more consistent common ground through increased attention to the common screen, and more transitions between collaboration styles. Therefore, this provided a better fit for the spontaneity of ubiquitous scenarios. QueryTogether and the corresponding study demonstrate the importance of entity based interfaces to improve collaboration by facilitating balanced participation, flexibility of collaboration styles and social processing of search entities across conversation and devices. The findings promote a vision of collaborative search support in spontaneous and ubiquitous multi-device settings, and better linking of conversation objects to searchable entities

Group cohesion and collaborative information behaviour: An exploration of student experiences of university group work

Modern organisations heavily rely on using interdisciplinary teams to accomplish intellectually demanding tasks. The advent of the World Wide Web, the advancements in communication technological tools and easy access to high volumes of information through the Web provide expanded capacity for individuals to work together and fulfil their shared goal but true collaboration is far from straightforward. Teamwork skills are identified as a desirable and distinguishing attribute of the graduates whom employers seek to employ. Accordingly, higher education institutions lay particular emphasis on developing students’ collaborative skills by designing and incorporating group projects into courses. The findings of relevant research demonstrate that employers are still not satisfied with the newly-hired graduates’ performance and students also reflect negative attitudes towards university group work. In this regard, scholars attempt to gain a through and deep understanding of individuals’ collaborative information behaviour when working in group settings and identify the factors that can impact on this process. This research, guided by the primary question of ‘How does group cohesion shape students’ collaborative information behaviour over the duration of group tasks?’ sought to explore the development of cohesion in student groups which has been widely recognised as an influential element in motivating group members to work collectively. Through a series of supporting research questions addressing the role that task cohesion, social cohesion and perceived cohesion play in students’ collaborative information behaviour, the work also aimed to find out how different dimensions of cohesion can have an impact on the way students make sense of the group task components, search for information and use information to accomplish group projects. This study took a qualitative approach and used Straussian grounded theory methodology to collect and analyse the data. Data collection was conducted by taking an in-depth interview approach through 10 semi-structured focus group sessions with student participants recruited from an Australian university who were undertaking project units as part of their degrees across any number of discipline areas over two successive semesters. Data was analysed using open, axial and selective coding following the Strauss and Corbin approach. Constant comparison of similarities and differences in the data enabled the researcher to elaborate on the identified concepts in terms of their properties and dimensions. This study resulted in rich description of how different dimensions of group cohesion emerged and developed in student groups over the duration of completing the group task and its association with students’ collaborative and individual information behaviour practices. Results suggest that task cohesion exerted more meaningful impact on group process and outcomes in comparison with other aspects of cohesion. It was found that students’ collaborative information behaviour activities are shaped by their perceptions of group task cohesion developed through adopting shared leadership style, the level of task complexity and interdependence and group members’ composition in terms of similarity in aspirations and academic capability. With regard to social cohesion, familiarity was identified as a factor which had immediate impact on students’ feelings of attraction and liking towards the group which did not persist over time as it fell under the influence of group members’ commitment and active involvement in group task activities. Task cohesion was then recognised as an antecedent of social cohesion in student groups and participants’ interpretation of social cohesion was based on experiencing comfortable feeling with group members instead of developing collective sense of closeness and friendship. Experiencing such a feeling within the group plays a more influential role in motivating students to communicate easily and sharing their ideas In terms of perceived cohesion, the findings of this study indicated that students in this particular sample did not intend to develop a sense of belonging and attachment to the group. They were of the mindset that once they complete their group task, the group would be disbanded so there is no potential benefit of developing such a feeling in university group context. This study highlighted the role of task design and its features on students’ collaboration as well as their choice of communication method throughout the group’s lifespan. At the early stages of the group project, the level of collaboration for identification of needed information to create a shared focus and define the project’s problem statement was heavily dependent upon the nature of the assessment task and its perceived complexity. Individual information searching was also identified as a common characteristic among all the research participants in this study but the structure of the assessment task determined the level of collaboration among members in regard to sharing information and evaluating the retrieved information in terms of relevancy and credibility. The evaluation and use of information sources to fulfil group task requirements was seen to be a collaborative activity in similar research studies but the findings from this study showed that groups assigned a highly structured task did not feel a need to have regular communication because their sub-tasks were not so much related to each other. This finding suggests that the outcomes of collaboration are not what most academics expect them to be as too little emphasis placed on the role of the task and more on the scale of the work to be delivered. The key finding of this research is that the group ‘task’ drives the behaviours of students, as individuals and as a group member, and that assigning students a project to do as a group that is too large to be done individually will not drive genuine collaboration. This research suggests an addition to the Input, Mediator, Output, Input (IMOI) model that includes a Task Calibration step by academic staff, to define the primary outcome of any given assessment task as either ‘collaboration’ or ‘product’, rather than the hope that collaboration takes place in order to deliver a big product. This shows that true collaboration would not take place by assigning students a large-scale group project; instead the tasks should be designed and structured in a way to drive and reward collaboratio

Geoinformatics in Citizen Science

The book features contributions that report original research in the theoretical, technological, and social aspects of geoinformation methods, as applied to supporting citizen science. Specifically, the book focuses on the technological aspects of the field and their application toward the recruitment of volunteers and the collection, management, and analysis of geotagged information to support volunteer involvement in scientific projects. Internationally renowned research groups share research in three areas: First, the key methods of geoinformatics within citizen science initiatives to support scientists in discovering new knowledge in specific application domains or in performing relevant activities, such as reliable geodata filtering, management, analysis, synthesis, sharing, and visualization; second, the critical aspects of citizen science initiatives that call for emerging or novel approaches of geoinformatics to acquire and handle geoinformation; and third, novel geoinformatics research that could serve in support of citizen science

The Multisided Complexity of Fairness in Recommender Systems

Recommender systems are poised at the interface between stakeholders: for example, job applicants and employers in the case of recommendations of employment listings, or artists and listeners in the case of music recommendation. In such multisided platforms, recommender systems play a key role in enabling discovery of products and information at large scales. However, as they have become more and more pervasive in society, the equitable distribution of their benefits and harms have been increasingly under scrutiny, as is the case with machine learning generally. While recommender systems can exhibit many of the biases encountered in other machine learning settings, the intersection of personalization and multisidedness makes the question of fairness in recommender systems manifest itself quite differently. In this article, we discuss recent work in the area of multisided fairness in recommendation, starting with a brief introduction to core ideas in algorithmic fairness and multistakeholder recommendation. We describe techniques for measuring fairness and algorithmic approaches for enhancing fairness in recommendation outputs. We also discuss feedback and popularity effects that can lead to unfair recommendation outcomes. Finally, we introduce several promising directions for future research in this area

Addressing marine and coastal governance conflicts at the interface of multiple sectors and jurisdictions

Marine and coastal activities are closely interrelated, and conflicts among different sectors can undermine management and conservation objectives. Governance systems for fisheries, power generation, irrigation, aquaculture, marine biodiversity conservation, and other coastal and maritime activities are typically organized to manage conflicts within sectors, rather than across them. Based on the discussions around eight case studies presented at a workshop held in Brest in June 2019, this paper explores institutional approaches to move beyond managing conflicts within a sector. We primarily focus on cases where the groups and sectors involved are heterogeneous in terms of: the jurisdiction they fall under; their objectives; and the way they value ecosystem services. The paper first presents a synthesis of frameworks for understanding and managing cross-sectoral governance conflicts, drawing from social and natural sciences. We highlight commonalities but also conceptual differences across disciplines to address these issues. We then propose a novel analytical framework which we used to evaluate the eight case studies. Based on the main lessons learned from case studies, we then discuss the feasibility and key determinants of stakeholder collaboration as well as compensation and incentive schemes. The discussion concludes with future research needs to support policy development and inform integrated institutional regimes that consider the diversity of stakeholder interests and the potential benefits of cross-sectoral coordination

The future of Earth observation in hydrology

In just the past 5 years, the field of Earth observation has progressed beyond the offerings of conventional space-agency-based platforms to include a plethora of sensing opportunities afforded by CubeSats, unmanned aerial vehicles (UAVs), and smartphone technologies that are being embraced by both for-profit companies and individual researchers. Over the previous decades, space agency efforts have brought forth well-known and immensely useful satellites such as the Landsat series and the Gravity Research and Climate Experiment (GRACE) system, with costs typically of the order of 1 billion dollars per satellite and with concept-to-launch timelines of the order of 2 decades (for new missions). More recently, the proliferation of smart-phones has helped to miniaturize sensors and energy requirements, facilitating advances in the use of CubeSats that can be launched by the dozens, while providing ultra-high (3-5 m) resolution sensing of the Earth on a daily basis. Start-up companies that did not exist a decade ago now operate more satellites in orbit than any space agency, and at costs that are a mere fraction of traditional satellite missions. With these advances come new space-borne measurements, such as real-time high-definition video for tracking air pollution, storm-cell development, flood propagation, precipitation monitoring, or even for constructing digital surfaces using structure-from-motion techniques. Closer to the surface, measurements from small unmanned drones and tethered balloons have mapped snow depths, floods, and estimated evaporation at sub-metre resolutions, pushing back on spatio-temporal constraints and delivering new process insights. At ground level, precipitation has been measured using signal attenuation between antennae mounted on cell phone towers, while the proliferation of mobile devices has enabled citizen scientists to catalogue photos of environmental conditions, estimate daily average temperatures from battery state, and sense other hydrologically important variables such as channel depths using commercially available wireless devices. Global internet access is being pursued via high-altitude balloons, solar planes, and hundreds of planned satellite launches, providing a means to exploit the "internet of things" as an entirely new measurement domain. Such global access will enable real-time collection of data from billions of smartphones or from remote research platforms. This future will produce petabytes of data that can only be accessed via cloud storage and will require new analytical approaches to interpret. The extent to which today's hydrologic models can usefully ingest such massive data volumes is unclear. Nor is it clear whether this deluge of data will be usefully exploited, either because the measurements are superfluous, inconsistent, not accurate enough, or simply because we lack the capacity to process and analyse them. What is apparent is that the tools and techniques afforded by this array of novel and game-changing sensing platforms present our community with a unique opportunity to develop new insights that advance fundamental aspects of the hydrological sciences. To accomplish this will require more than just an application of the technology: in some cases, it will demand a radical rethink on how we utilize and exploit these new observing systems