When situativity meets objectivity in peer-production of knowledge:the case of the WikiRate platform

PurposeThe purpose of this paper is to further the debate on Knowledge Artefacts (KAs), by presenting the design of WikiRate, a Collective Awareness platform whose goal is to support a wider public contributing to the generation of knowledge on environmental, social and governance (ESG) performance of companies.Design/methodology/approachThe material presented in the paper comes from the first-hand experience of the authors as part of the WikiRate design team. This material is reflexively discussed using concepts from the field of science and technology studies.FindingsUsing the concept of the “funnel of interest”, the authors discuss how the design of a KA like WikiRate relies on the designers’ capacity to translate general statements into particular design solutions. The authors also show how this funnelling helps understanding the interplay between situativity and objectivity in a KA. The authors show how WikiRate is a peer-production platform based on situativity, which requires a robust level of objectivity for producing reliable knowledge about the ESG performance of companies.Originality/valueThis paper furthers the debate on KAs. It presents a relevant design example and offers in the discussion a set of design and community building recommendations to practitioners

Benchmarking performance management systems

The Balanced Scorecard and associated performance management approaches, has become a widely practiced and popular management reporting method in recent times. Moreover, enabling technology, which assists in the delivery and personalisation of corporate performance information, is having a deeper and more rapid impact than ever before. This paper presents a brief comparative benchmarking study of leading enterprise performance management systems. Also, the author discusses the merits of bespoke internet technology development and out-of-the-box portal functionalities. An analysis of key business drivers and implementation risks of such approaches is highlighted via a case study example, and concludes the paper

A Framework for the Evaluation of Semantics-based Service Composition Approaches

The benefits of service composition are being largely acknowledged in the literature nowadays. However, as the amount of available services increases, it becomes difficult to manage, discover, select and compose them, so that automation is required in these processes. This can be achieved by using semantic information represented in ontologies. Currently there are many different approaches that support semantics-based service composition. However, still little effort has been spent on creating a common methodology to evaluate and compare such approaches. In this paper we present our initial ideas to create an evaluation framework for semantics-based service composition approaches. We use a collection of existing services, and define a set of evaluation metrics, confusion matrix-based and time-based. Furthermore, we present how composition evaluation scenarios are generated from the collection of services and specify the strategy to be used in the evaluation process. We demonstrate the proposed framework through an example. Currently there are mechanisms and initiatives to address the evaluation of the semantics-based service discovery and matchmaking approaches. However, still few efforts have been spent on the creation of comprehensive evaluation mechanisms for semantics-based service composition approaches

WikiRate.org - leveraging collective awareness to understand companies' environmental, social and governance performance

Abstract. WikiRate is a Collective Awareness Platform for Sustainability and Social Innovation (CAPS) project with the aim of \crowdsourcing better companies" through analysis of their Environmental Social and Governance (ESG) performance. Research to inform the design of the platform involved surveying the current corporate ESG information landscape, and identifying ways in which an open approach and peer production ethos could be e ffectively mobilised to improve this landscape's fertility. The key requirement identi ed is for an open public repository of data tracking companies' ESG performance. Corporate Social Responsibility reporting is conducted in public, but there are barriers to accessing the information in a standardised analysable format. Analyses of and ratings built upon this data can exert power over companies' behaviour in certain circumstances, but the public at large have no access to the data or the most infuential ratings that utilise it. WikiRate aims to build an open repository for this data along with tools for analysis, to increase public demand for the data, allow a broader range of stakeholders to participate in its interpretation, and in turn drive companies to behave in a more ethical manner. This paper describes the quantitative Metrics system that has been designed to meet those objectives and some early examples of its use

The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises