Benchmarking News Recommendations in a Living Lab

Most user-centric studies of information access systems in literature suffer from unrealistic settings or limited numbers of users who participate in the study. In order to address this issue, the idea of a living lab has been promoted. Living labs allow us to evaluate research hypotheses using a large number of users who satisfy their information need in a real context. In this paper, we introduce a living lab on news recommendation in real time. The living lab has first been organized as News Recommendation Challenge at ACM RecSys’13 and then as campaign-style evaluation lab NEWSREEL at CLEF’14. Within this lab, researchers were asked to provide news article recommendations to millions of users in real time. Different from user studies which have been performed in a laboratory, these users are following their own agenda. Consequently, laboratory bias on their behavior can be neglected. We outline the living lab scenario and the experimental setup of the two benchmarking events. We argue that the living lab can serve as reference point for the implementation of living labs for the evaluation of information access systems

Evaluating global e-government sites: A view using web diagnostics tools

This is the post-print version of the Article. The official published version can be accessed from the link below - Copyright @ 2004 The AuthorsSeveral governments across the world have embraced the digital revolution and continue to take advantage of the information and communication facilities offered by the Internet to offer public services. Conversely, citizens’ awareness and expectations of Internet based online-public-services have also increased in recent times. Although the numbers of the different national e-Government web portals have rapidly increased in the last three years, the success of these portals will largely depend on their accessibility, quality and privacy. This paper reports the results of an evaluative study of a cross-section of e-Government portals from these three perspectives, using a common set of performance metrics and Web diagnostic engines. Results show that not only are there wide variations in the spectrum of information and services provided by these portals, but that significant work still needs to be undertaken in order to make the portals examples of ‘best practice’ e-Government services

Report of the user requirements and web based access for eResearch workshops

The User Requirements and Web Based Access for eResearch Workshop, organized jointly by NeSC and NCeSS, was held on 19 May 2006. The aim was to identify lessons learned from e-Science projects that would contribute to our capacity to make Grid infrastructures and tools usable and accessible for diverse user communities. Its focus was on providing an opportunity for a pragmatic discussion between e-Science end users and tool builders in order to understand usability challenges, technological options, community-specific content and needs, and methodologies for design and development. We invited members of six UK e-Science projects and one US project, trying as far as possible to pair a user and developer from each project in order to discuss their contrasting perspectives and experiences. Three breakout group sessions covered the topics of user-developer relations, commodification, and functionality. There was also extensive post-meeting discussion, summarized here. Additional information on the workshop, including the agenda, participant list, and talk slides, can be found online at http://www.nesc.ac.uk/esi/events/685/ Reference: NeSC report UKeS-2006-07 available from http://www.nesc.ac.uk/technical_papers/UKeS-2006-07.pd

Geoportals: an internet marketing perspective

A geoportal is a web site that presents an entry point to geo-products (including geo-data) on the web. Despite their importance in (spatial) data infrastructures, literature suggest stagnating or even declining trends in visitor numbers. In this paper relevant ideas and techniques for improving performance are derived from internet marketing literature. We tested the extent to which these ideas are already applied in practice through a survey among 48 geoportals worldwide. Results show in many cases positive correlation with trends in visitor numbers. The ideas can be useful for geoportal managers developing their marketing strateg

The organizational dynamics enabling patient portal impacts upon organizational performance and patient health: a qualitative study of Kaiser Permanente.

BackgroundPatient portals may lead to enhanced disease management, health plan retention, changes in channel utilization, and lower environmental waste. However, despite growing research on patient portals and their effects, our understanding of the organizational dynamics that explain how effects come about is limited.MethodsThis paper uses qualitative methods to advance our understanding of the organizational dynamics that influence the impact of a patient portal on organizational performance and patient health. The study setting is Kaiser Permanente, the world's largest not-for-profit integrated delivery system, which has been using a portal for over ten years. We interviewed eighteen physician leaders and executives particularly knowledgeable about the portal to learn about how they believe the patient portal works and what organizational factors affect its workings. Our analytical framework centered on two research questions. (1) How does the patient portal impact care delivery to produce the documented effects?; and (2) What are the important organizational factors that influence the patient portal's development?ResultsWe identify five ways in which the patient portal may impact care delivery to produce reported effects. First, the portal's ability to ease access to services improves some patients' satisfaction as well as changes the way patients seek care. Second, the transparency and activation of information enable some patients to better manage their care. Third, care management may also be improved through augmented patient-physician interaction. This augmented interaction may also increase the 'stickiness' of some patients to their providers. Forth, a similar effect may be triggered by a closer connection between Kaiser Permanente and patients, which may reduce the likelihood that patients will switch health plans. Finally, the portal may induce efficiencies in physician workflow and administrative tasks, stimulating certain operational savings and deeper involvement of patients in medical decisions. Moreover, our analysis illuminated seven organizational factors of particular importance to the portal's development--and thereby ability to impact care delivery: alignment with financial incentives, synergy with existing IT infrastructure and operations, physician-led governance, inclusive decision making and knowledge sharing, regional flexibility to implementation, continuous innovation, and emphasis on patient-centered design.ConclusionsThese findings show how organizational dynamics enable the patient portal to affect care delivery by summoning organization-wide support for and use of a portal that meets patient needs

Adding dimensions to the analysis of the quality of health information of websites returned by Google. Cluster analysis identifies patterns of websites according to their classification and the type of intervention described.

Background and aims: Most of the instruments used to assess the quality of health information on the Web (e.g. the JAMA criteria) only analyze one dimension of information quality, trustworthiness. We try to compare these characteristics with the type of treatments the website describe, whether evidence-based medicine or note, and correlate this with the established criteria. Methods: We searched Google for “migraine cure” and analyzed the first 200 websites for: 1) JAMA criteria (authorship, attribution, disclosure, currency); 2) class of websites (commercial, health portals, professional, patient groups, no-profit); and 3) type of intervention described (approved drugs, alternative medicine, food, procedures, lifestyle, drugs still at the research stage). We used hierarchical cluster analysis to assess associations between classes of websites and types of intervention described. Subgroup analysis on the first 10 websites returned was performed. Results: Google returned health portals (44%), followed by commercial websites (31%) and journalism websites (11%). The type of intervention mentioned most often was alternative medicine (55%), followed by procedures (49%), lifestyle (42%), food (41%) and approved drugs (35%). Cluster analysis indicated that health portals are more likely to describe more than one type of treatment while commercial websites most often describe only one. The average JAMA score of commercial websites was significantly lower than for health portals or journalism websites, and this was mainly due to lack of information on the authors of the text and indication of the date the information was written. Looking at the first 10 websites from Google, commercial websites are under-represented and approved drugs over-represented. Conclusions: This approach allows the appraisal of the quality of health-related information on the Internet focusing on the type of therapies/prevention methods that are shown to the patient

SPEIR: Scottish Portals for Education, Information and Research. Final Project Report: Elements and Future Development Requirements of a Common Information Environment for Scotland

The SPEIR (Scottish Portals for Education, Information and Research) project was funded by the Scottish Library and Information Council (SLIC). It ran from February 2003 to September 2004, slightly longer than the 18 months originally scheduled and was managed by the Centre for Digital Library Research (CDLR). With SLIC's agreement, community stakeholders were represented in the project by the Confederation of Scottish Mini-Cooperatives (CoSMiC), an organisation whose members include SLIC, the National Library of Scotland (NLS), the Scottish Further Education Unit (SFEU), the Scottish Confederation of University and Research Libraries (SCURL), regional cooperatives such as the Ayrshire Libraries Forum (ALF)1, and representatives from the Museums and Archives communities in Scotland. Aims; A Common Information Environment For Scotland The aims of the project were to: o Conduct basic research into the distributed information infrastructure requirements of the Scottish Cultural Portal pilot and the public library CAIRNS integration proposal; o Develop associated pilot facilities by enhancing existing facilities or developing new ones; o Ensure that both infrastructure proposals and pilot facilities were sufficiently generic to be utilised in support of other portals developed by the Scottish information community; o Ensure the interoperability of infrastructural elements beyond Scotland through adherence to established or developing national and international standards. Since the Scottish information landscape is taken by CoSMiC members to encompass relevant activities in Archives, Libraries, Museums, and related domains, the project was, in essence, concerned with identifying, researching, and developing the elements of an internationally interoperable common information environment for Scotland, and of determining the best path for future progress

Global Grids and Software Toolkits: A Study of Four Grid Middleware Technologies

Grid is an infrastructure that involves the integrated and collaborative use of computers, networks, databases and scientific instruments owned and managed by multiple organizations. Grid applications often involve large amounts of data and/or computing resources that require secure resource sharing across organizational boundaries. This makes Grid application management and deployment a complex undertaking. Grid middlewares provide users with seamless computing ability and uniform access to resources in the heterogeneous Grid environment. Several software toolkits and systems have been developed, most of which are results of academic research projects, all over the world. This chapter will focus on four of these middlewares--UNICORE, Globus, Legion and Gridbus. It also presents our implementation of a resource broker for UNICORE as this functionality was not supported in it. A comparison of these systems on the basis of the architecture, implementation model and several other features is included.Comment: 19 pages, 10 figure