Community Development Evaluation Storymap and Legend

Community based organizations, funders, and intermediary organizations working in the community development field have a shared interest in building stronger organizations and stronger communities. Through evaluation these organizations can learn how their programs and activities contribute to the achievement of these goals, and how to improve their effectiveness and the well-being of their communities. Yet, evaluation is rarely seen as part of a non-judgemental organizational learning process. Instead, the term "evaluation" has often generated anxiety and confusion. The Community Development Storymap project is a response to those concerns.Illustrations found in this document were produced by Grove Consultants

The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises

The role of handbooks in knowledge creation and diffusion: A case of science and technology studies

Genre is considered to be an important element in scholarly communication and in the practice of scientific disciplines. However, scientometric studies have typically focused on a single genre, the journal article. The goal of this study is to understand the role that handbooks play in knowledge creation and diffusion and their relationship with the genre of journal articles, particularly in highly interdisciplinary and emergent social science and humanities disciplines. To shed light on these questions we focused on handbooks and journal articles published over the last four decades belonging to the research area of Science and Technology Studies (STS), broadly defined. To get a detailed picture we used the full-text of five handbooks (500,000 words) and a well-defined set of 11,700 STS articles. We confirmed the methodological split of STS into qualitative and quantitative (scientometric) approaches. Even when the two traditions explore similar topics (e.g., science and gender) they approach them from different starting points. The change in cognitive foci in both handbooks and articles partially reflects the changing trends in STS research, often driven by technology. Using text similarity measures we found that, in the case of STS, handbooks play no special role in either focusing the research efforts or marking their decline. In general, they do not represent the summaries of research directions that have emerged since the previous edition of the handbook.Comment: Accepted for publication in Journal of Informetric

South American Expert Roundtable : increasing adaptive governance capacity for coping with unintended side effects of digital transformation

This paper presents the main messages of a South American expert roundtable (ERT) on the unintended side effects (unseens) of digital transformation. The input of the ERT comprised 39 propositions from 20 experts representing 11 different perspectives. The two-day ERT discussed the main drivers and challenges as well as vulnerabilities or unseens and provided suggestions for: (i) the mechanisms underlying major unseens; (ii) understanding possible ways in which rebound effects of digital transformation may become the subject of overarching research in three main categories of impact: development factors, society, and individuals; and (iii) a set of potential action domains for transdisciplinary follow-up processes, including a case study in Brazil. A content analysis of the propositions and related mechanisms provided insights in the genesis of unseens by identifying 15 interrelated causal mechanisms related to critical issues/concerns. Additionally, a cluster analysis (CLA) was applied to structure the challenges and critical developments in South America. The discussion elaborated the genesis, dynamics, and impacts of (groups of) unseens such as the digital divide (that affects most countries that are not included in the development of digital business, management, production, etc. tools) or the challenge of restructuring small- and medium-sized enterprises (whose service is digitally substituted by digital devices). We identify specific issues and effects (for most South American countries) such as lack of governmental structure, challenging geographical structures (e.g., inclusion in high-performance transmission power), or the digital readiness of (wide parts) of society. One scientific contribution of the paper is related to the presented methodology that provides insights into the phenomena, the causal chains underlying “wanted/positive” and “unwanted/negative” effects, and the processes and mechanisms of societal changes caused by digitalization

Innovation failure in the eye of the beholder: Towards a theory of innovation shaped by competing agendas within higher education

This paper examines a case of perceived innovation failure in higher education, a service dominated by conflicting institutional logics of professionalism and markets. Through a mixed methodology investigating student attitudes to, and behaviour around, technological innovation, the paper makes a contribution to the public service innovation literature by focusing on duality in innovation outcomes. This is suggestive of an innovation typology in public services: professionalism-driven and consumerism-driven innovation

Citation gaming induced by bibliometric evaluation: a country-level comparative analysis

It is several years since national research evaluation systems around the globe started making use of quantitative indicators to measure the performance of researchers. Nevertheless, the effects on these systems on the behavior of the evaluated researchers are still largely unknown. We attempt to shed light on this topic by investigating how Italian researchers reacted to the introduction in 2011 of national regulations in which key passages of professional careers are governed by bibliometric indicators. A new inwardness measure, able to gauge the degree of scientific self-referentiality of a country, is defined as the proportion of citations coming from the country itself compared to the total number of citations gathered by the country. Compared to the trends of the other G10 countries in the period 2000-2016, Italy's inwardness shows a net increase after the introduction of the new evaluation rules. Indeed, globally and also for a large majority of the research fields, Italy became the European country with the highest inwardness. Possible explanations are proposed and discussed, concluding that the observed trends are strongly suggestive of a generalized strategic use of citations, both in the form of author self-citations and of citation clubs. We argue that the Italian case offers crucial insights on the constitutive effects of evaluation systems. As such, it could become a paradigmatic case in the debate about the use of indicators in science-policy contexts

Building governance capability in online social production : insights from Wikipedia

This article investigates a form of governance that makes online social production possible. Drawing on the concepts of capability and routine, we develop a dynamic, process-oriented view that departs from past research focused on static comparative analysis. We theorize that online social production systems develop a collective governance capability to steer the process of integrating distributed knowledge resources to the production of value. Governance mechanisms emerge from individual and collective learning that is made possible by new technology, and they evolve over time, as routines are developed to respond to new problems faced by a growing production system. Using Wikipedia as a paradigmatic example of online social production, we characterize governance as an evolving, enabling and embedded process and discuss implications for a dynamic theory of governance

Social Vulnerability and Touristification of Historic Centers

Historic centers have su ered di erent processes of neglect, occupation, segregation, gentrification, and touristification as a result of changes in demand and policies. Currently, they are going through a homogenization process motivated by tourist pressure, which is causing the expulsion of the local population; this is a common topic of interest for media and political agendas, which requires scientific analysis. This research aims at identifying the winning and the losing tourist groups in the historic center of Seville. It is structured in two parts: a conceptual one based on the bibliographic review with which one wants to know how the current society responds to tourist pressure through defining and characterizing the processes of substitution of uses and inhabitants, and another empirical one in which the analysis of statistical indicators (demographic, economic, and residential) treated with Geographic Information System (GIS) allows us to measure the degree of existing vulnerability and analyze social and spatial e ects caused by the tourism in Sevill

Decoding the "Free/Open Source(F/OSS) Software Puzzle" a survey of theoretical and empirical contributions

F/OSS software has been described by many as a puzzle. In the past five years, it has stimulated the curiosity of scholars in a variety of fields, including economics, law, psychology, anthropology and computer science, so that the number of contributions on the subject has increased exponentially. The purpose of this paper is to provide a sufficiently comprehensive account of these contributions in order to draw some general conclusions on the state of our understanding of the phenomenon and identify directions for future research. The exercise suggests that what is puzzling about F/OSS is not so much the fact that people freely contribute to a good they make available to all, but rather the complexity of its institutional structure and its ability to organizationally evolve over time.F/OSS software, Innovation, Incentives, Governance, Intellectual Property Rights