Benchmarking News Recommendations in a Living Lab

Most user-centric studies of information access systems in literature suffer from unrealistic settings or limited numbers of users who participate in the study. In order to address this issue, the idea of a living lab has been promoted. Living labs allow us to evaluate research hypotheses using a large number of users who satisfy their information need in a real context. In this paper, we introduce a living lab on news recommendation in real time. The living lab has first been organized as News Recommendation Challenge at ACM RecSys’13 and then as campaign-style evaluation lab NEWSREEL at CLEF’14. Within this lab, researchers were asked to provide news article recommendations to millions of users in real time. Different from user studies which have been performed in a laboratory, these users are following their own agenda. Consequently, laboratory bias on their behavior can be neglected. We outline the living lab scenario and the experimental setup of the two benchmarking events. We argue that the living lab can serve as reference point for the implementation of living labs for the evaluation of information access systems

Systematising Learning and Research Information

This paper considers the ways in which information of relevance to the learning and research communities is organised and used. It contends that there is considerable overlap between the different types of online resources and information currently available within education. It describes some of the structured environments and data stores that have emerged in recent years, along with standards which are attempting to define the properties of discrete learning objects, through the specification of Learning Object Metadata (LOM). The paper contends that current developments of structured learning environments such as Managed and Virtual Learning Environments (MLEs and VLEs) are occurring on the whole in parallel to resource data stores, such as information gateways and portals. This discrepancy has arisen in part because these developments have occurred independently of one another and in part because there has to date been no rigorous definition of the underlying theoretical models. Furthermore, it argues that these predefined structured environments are unlikely to be sufficient to meet the information needs of users in different contexts. The paper goes on to describe an information toolkit, which provides a way of systematising information handling in learning and research, which helps users articulate information plans within specific contexts. The paper concludes with a description of two case studies which illustrate how this toolkit can be used

Benchmarking news recommendations: the CLEF NewsREEL use case

The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to evaluate and optimize news recommender algorithms. The goal is to create an algorithm that is able to generate news items that users would click, respecting a strict time constraint. The lab challenges participants to compete in either a "living lab" (Task 1) or perform an evaluation that replays recorded streams (Task 2). In this report, we discuss the objectives and challenges of the NewsREEL lab, summarize last year's campaign and outline the main research challenges that can be addressed by participating in NewsREEL 2016

Quality-testing the legal Internet: finding law with the SOSIG Law Gateway

English language version of a preprint of an article by Steven Whittle, Information Systems Manager at the Institute of Advanced Legal Studies, published in Recht Bibliothek Dokumentation, the journal of the AjBD Arbeitsgemeinschaft für juristisches Bibliotheks-und Dokumentationswese

Users' reading habits in online news portals

The aim of this study is to survey reading habits of users of an online news portal. The assumption motivating this study is that insight into the reading habits of users can be helpful to design better news recommendation systems. We estimated the transition probabilities that users who read an article of one news category will move to read an article of another (not necessarily distinct) news category. For this, we analyzed the users' click behavior within plista data set. Key findings are the popularity of category local, loyalty of readers to the same category, observing similar results when addressing enforced click streams, and the case that click behavior is highly influenced by the news category

Benchmarking performance management systems

The Balanced Scorecard and associated performance management approaches, has become a widely practiced and popular management reporting method in recent times. Moreover, enabling technology, which assists in the delivery and personalisation of corporate performance information, is having a deeper and more rapid impact than ever before. This paper presents a brief comparative benchmarking study of leading enterprise performance management systems. Also, the author discusses the merits of bespoke internet technology development and out-of-the-box portal functionalities. An analysis of key business drivers and implementation risks of such approaches is highlighted via a case study example, and concludes the paper

Proxy dynamic delegation in grid gateway

Nowadays one of the main obstacles the research comes up against is the difficulty in accessing the required computational resources. Grid is able to offer the user a wide set of resources, even if they are often too hard to exploit for non expert end user. Use simplification has today become a common practice in the access and utilization of Cloud, Grid, and data center resources. With the launch of L-GRID gateway, we introduced a new way to deal with Grid portals. L-GRID is an extremely light portal developed in order to access the EGI Grid infrastructure via Web, allowing users to submit their jobs from whatever Web browser in a few minutes, without any knowledge about the underlying Grid infrastructure.Comment: 6 page

Report of the user requirements and web based access for eResearch workshops

The User Requirements and Web Based Access for eResearch Workshop, organized jointly by NeSC and NCeSS, was held on 19 May 2006. The aim was to identify lessons learned from e-Science projects that would contribute to our capacity to make Grid infrastructures and tools usable and accessible for diverse user communities. Its focus was on providing an opportunity for a pragmatic discussion between e-Science end users and tool builders in order to understand usability challenges, technological options, community-specific content and needs, and methodologies for design and development. We invited members of six UK e-Science projects and one US project, trying as far as possible to pair a user and developer from each project in order to discuss their contrasting perspectives and experiences. Three breakout group sessions covered the topics of user-developer relations, commodification, and functionality. There was also extensive post-meeting discussion, summarized here. Additional information on the workshop, including the agenda, participant list, and talk slides, can be found online at http://www.nesc.ac.uk/esi/events/685/ Reference: NeSC report UKeS-2006-07 available from http://www.nesc.ac.uk/technical_papers/UKeS-2006-07.pd