Writing impact case studies: a comparative study of high-scoring and low-scoring case studies from REF2014

This paper reports on two studies that used qualitative thematic and quantitative linguistic analysis, respectively, to assess the content and language of the largest ever sample of graded research impact case studies, from the UK Research Excellence Framework 2014 (REF). The paper provides the first empirical evidence across disciplinary main panels of statistically significant linguistic differences between high- versus low-scoring case studies, suggesting that implicit rules linked to written style may have contributed to scores alongside the published criteria on the significance, reach and attribution of impact. High-scoring case studies were more likely to provide specific and high-magnitude articulations of significance and reach than low-scoring cases. High-scoring case studies contained attributional phrases which were more likely to attribute research and/or pathways to impact, and they were written more coherently (containing more explicit causal connections between ideas and more logical connectives) than low-scoring cases. High-scoring case studies appear to have conformed to a distinctive new genre of writing, which was clear and direct, and often simplified in its representation of causality between research and impact, and less likely to contain expressions of uncertainty than typically associated with academic writing. High-scoring case studies in two Main Panels were significantly easier to read than low-scoring cases on the Flesch Reading Ease measure, although both high-scoring and low-scoring cases tended to be of “graduate” reading difficulty. The findings of our work enable impact case study authors to better understand the genre and make content and language choices that communicate their impact as effectively as possible. While directly relevant to the assessment of impact in the UK’s Research Excellence Framework, the work also provides insights of relevance to institutions internationally who are designing evaluation frameworks for research impact

'Joined up' policy making : group decision and negotiation practice

Creating public value is problematic in a world of austerity. Joint private and public, and public-public, projects are now an established way of creating public value. Establishing joint goals within a context of different ‘own goals’ is important and difficult. A particular issue facing many organisations in seeking to negotiate joint goals in a collaborative project is that of getting all the key managers from both organisations together over enough of a sequence of meetings for agreements to be meaningful and owned by those who will deliver the project. The development of such goals can be significantly enhanced by i) using a Group Decision Support System (GDSS) and ii) using a powerful conceptualisation of a goals framework comprising: a goals system; ‘core goals’; ‘meta-goals’; ‘negative’ goals; and ‘above and beyond’ goals. In the case of negotiating joint goals the use of a GDSS has increased productivity to such an extent that powerful negotiated agreements can be achieved with all key managers in the room in as little as one meeting. The combination of high productivity, anonymity, and the structuring of the data has also facilitated the uncovering of ‘negative goals’, and the development of ‘meta-goals’ and ‘above and beyond’ goals. This paper uses three real cases to illustrate the significance of a Group Support System’s contribution and to illustrate the different types of goals. In doing so the paper suggests that facilitating such outcomes requires a carefully designed strategic conversation necessarily supported by a Group Decision Support System to enable group decision and negotiation in practice. Two of the cases are from public-public collaboration in the health field, and the other from a private-public setting

Italian Public Administration Reform: what are the limits of financial performance measures?

The Italian public sector reforms in recent years have demonstrated an over-reliance on accounting-based financial measurements which has essentially created a sort of ‘hierarchization’ of performance. This chapter aims to demonstrate whether and how this predominance leads to negative consequences in the evaluation (and management) of public sector organizations: First, because in definitive governments, performance should be assessed coupling financial parameters with non-financial measures and qualitative judgements (Jones and Pendlebury in Public Sector Accounting, 6th ed., Pearson Prentice Hall, London, 2010); second, for the lack of a systemic approach, financial performance should not be the ultimate objective of public management but instead an instrument to evaluate the financial comparability of various priorities to purse (public value, social, environmental, etc.) (Esposito and Ricci in Public Money Manage 35(3):227–231, 2015)