Persuading developers to buy into software process improvement: an exploratory analysis

In order to investigate practitioners' opinions of software process and software process improvement, we have collected information from 13 companies, in a variety of ways i.e. the use of Repertory Grid Technique, survey and focus group discussions. Both the Repertory Grid Technique and the focus group discussions (43 discussions occurred, in total) produced a large volume of qualitative data. At the same time, other researchers have reported--investigations of practitioners, and we are interested in how their reports may relate to our own. Thus, other research publications can also be treated as a form of qualitative data. In this paper, we review advice on a method, content analysis, that is used to analyse qualitative data. Content analysis is a method for identifying and classifying words and phrases used in--ordinary language. We use content analysis to describe and analyse discussions on software--process and software process improvement. We report preliminary findings from an analysis--of both the focus group evidence and some publications. Our main finding is that there is an--apparent contradiction between developers saying that they want evidence for software process improvement, and what developers will accept as evidence. This presents a serious problem for research: even if researchers could demonstrate a strong, reliable relationship between software process improvement and improved organisational performance, there would still be the problem of convincing practitioners that the evidence applies to their particular situation

Persuading developers to buy into software process improvement: a local opinion and empirical evidence

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.---- Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.In order to investigate practitioners’ opinions of software process and software process improvement, we have collected a large volume of qualitative evidence from 13 companies. At the same time, other researchers have reported investigations of practitioners, and we are interested in how their reports may relate to our evidence. Thus, other research publications can also be treated as a form of qualitative data. In this paper, we review advice on a method, content analysis, that is used to analyse qualitative data. We use content analysis to describe and analyse discussions on software process and software process improvement. We report preliminary findings from an analysis of both the focus group evidence and four publications

Doing pedagogical research in engineering

This is a book

Citations: Indicators of Quality? The Impact Fallacy

We argue that citation is a composed indicator: short-term citations can be considered as currency at the research front, whereas long-term citations can contribute to the codification of knowledge claims into concept symbols. Knowledge claims at the research front are more likely to be transitory and are therefore problematic as indicators of quality. Citation impact studies focus on short-term citation, and therefore tend to measure not epistemic quality, but involvement in current discourses in which contributions are positioned by referencing. We explore this argument using three case studies: (1) citations of the journal Soziale Welt as an example of a venue that tends not to publish papers at a research front, unlike, for example, JACS; (2) Robert Merton as a concept symbol across theories of citation; and (3) the Multi-RPYS ("Multi-Referenced Publication Year Spectroscopy") of the journals Scientometrics, Gene, and Soziale Welt. We show empirically that the measurement of "quality" in terms of citations can further be qualified: short-term citation currency at the research front can be distinguished from longer-term processes of incorporation and codification of knowledge claims into bodies of knowledge. The recently introduced Multi-RPYS can be used to distinguish between short-term and long-term impacts.Comment: accepted for publication in Frontiers in Research Metrics and Analysis; doi: 10.3389/frma.2016.0000

Trialing project-based learning in a new EAP ESP course: A collaborative reflective practice of three college English teachers

Currently in many Chinese universities, the traditional College English course is facing the risk of being ‘marginalized’, replaced or even removed, and many hours previously allocated to the course are now being taken by EAP or ESP. At X University in northern China, a curriculum reform as such is taking place, as a result of which a new course has been created called ‘xue ke’ English. Despite the fact that ‘xue ke’ means subject literally, the course designer has made it clear that subject content is not the target, nor is the course the same as EAP or ESP. This curriculum initiative, while possibly having been justified with a rationale of some kind (e.g. to meet with changing social and/or academic needs of students and/or institutions), this is posing a great challenge for, as well as considerable pressure on, a number of College English teachers who have taught this single course for almost their entire teaching career. In such a context, three teachers formed a peer support group in Semester One this year, to work collaboratively co-tackling the challenge, and they chose Project-Based Learning (PBL) for the new course. This presentation will report on the implementation of this project, including the overall designing, operational procedure, and the teachers’ reflections. Based on discussion, pre-agreement was reached on the purpose and manner of collaboration as offering peer support for more effective teaching and learning and fulfilling and pleasant professional development. A WeChat group was set up as the chief platform for messaging, idea-sharing, and resource-exchanging. Physical meetings were supplementary, with sound agenda but flexible time, and venues. Mosoteach cloud class (lan mo yun ban ke) was established as a tool for virtual learning, employed both in and after class. Discussions were held at the beginning of the semester which determined only brief outlines for PBL implementation and allowed space for everyone to autonomously explore in their own way. Constant further discussions followed, which generated a great deal of opportunities for peer learning and lesson plan modifications. A reflective journal, in a greater or lesser detailed manner, was also kept by each teacher to record the journey of the collaboration. At the end of the semester, it was commonly recognized that, although challenges existed, the collaboration was overall a success and they were all willing to continue with it and endeavor to refine it to be a more professional and productive approach

Drawing Elena Ferrante's Profile. Workshop Proceedings, Padova, 7 September 2017

Elena Ferrante is an internationally acclaimed Italian novelist whose real identity has been kept secret by E/O publishing house for more than 25 years. Owing to her popularity, major Italian and foreign newspapers have long tried to discover her real identity. However, only a few attempts have been made to foster a scientific debate on her work. In 2016, Arjuna Tuzzi and Michele Cortelazzo led an Italian research team that conducted a preliminary study and collected a well-founded, large corpus of Italian novels comprising 150 works published in the last 30 years by 40 different authors. Moreover, they shared their data with a select group of international experts on authorship attribution, profiling, and analysis of textual data: Maciej Eder and Jan Rybicki (Poland), Patrick Juola (United States), Vittorio Loreto and his research team, Margherita Lalli and Francesca Tria (Italy), George Mikros (Greece), Pierre Ratinaud (France), and Jacques Savoy (Switzerland). The chapters of this volume report the results of this endeavour that were first presented during the international workshop Drawing Elena Ferrante's Profile in Padua on 7 September 2017 as part of the 3rd IQLA-GIAT Summer School in Quantitative Analysis of Textual Data. The fascinating research findings suggest that Elena Ferrante\u2019s work definitely deserves \u201cmany hands\u201d as well as an extensive effort to understand her distinct writing style and the reasons for her worldwide success

The BTWC Protocol: Proposed Complete Text for an Integrated Regime

Ye

From bench to bedside: Tracing the payback forwards from basic or early clinical research – A preliminary exercise and proposals for a future study

EXECUTIVE SUMMARY Chapter 1 : Introduction • The members of the research team from HERG and the Wellcome Trust have conducted previous studies showing that it is possible both to assess the payback from applied health research, and to use bibliometrics to trace the links between generations of research and clinical guidelines. In another of the team’s studies, however, it proved difficult to replicate the major study by Comroe and Dripps (1976) that had identified clinical advances and then worked backwards to show that they had relied on earlier basic research. Therefore, the study reported here sets out to use the methods developed in our previous studies of payback to undertake analysis that starts with more basic or early clinical research and traces the research lines forwards to clinical applications. Whilst this preliminary study involved preparation for a future large-scale study, it was hoped that it would also provide an interesting case study. • Starting with the research outputs of one team 20 years ago, called the 1st generation papers, the preliminary study has three main elements: standard bibliometric analysis through several generations of papers; categorisation of the citations; and qualitative analysis using questionnaires, critical pathway analysis and interviews to trace the impact of the 1st generation of research. • Diabetes and cardiology were suggested as possible topics on which to base the study. Initial reviews identified two bodies of research in diabetes as being potentially suitable for reasons such as the continuing activity of key members of the team. • The research into diabetes conducted in 1981 by George Alberti and his team at Newcastle, and collaborators elsewhere, was selected to provide the case study for this preliminary stage for several reasons. It was thought to have been important science and there was a belief that some of it had made a contribution to clinical practice. Chapter 2 : Bibliometric analysis • An original plan to look at publications produced over a three year period was changed to looking at the output of just one year, 1981, because in that year alone Alberti and colleagues published 29 articles. These form the 1st generation papers and the average number of citations they received is high. Identifying the citations given to these 29 papers resulted in 799 2nd generation papers and 12,891 3rd generation papers. The numbers involved meant that it was impractical to go beyond the 3rd generation. Within the high overall average, the variation in the number of citations per paper was iii considerable going from 76 to just one. Similarly, the half-lives of the 29 papers, ie the time taken for an article to receive 50% of its citations, ranged from two years to 11. • Articles can be given a Research Level (ie one of four levels from clinical observation to basic) based on the journals in which they appear. Such analysis demonstrates the breadth of Alberti’s work because the 29 articles are spread across all four Research Levels. Crucially, there was not a shift from basic to more clinical levels across the generations. The higher than average number of authors and addresses per paper is testimony to Alberti’s extensive collaborations. • The funding acknowledgements reveal the high proportion of papers supported, at least partially, by one funder: the British Diabetic Association, now Diabetes UK, which provided core support for Alberti’s Newcastle team. Chapter 3 : Categorisation of citations • Traditional citation analysis does not allow identification of the importance of the cited article to the citing article, and therefore limits the ability to use citation analysis to trace the impact of basic or early research on later research. We conducted a review of the literature of the meaning of citations. • From this review, a template was devised that allowed the location, nature and importance of citations to be recorded as well as the type of research (basic or clinical) described in the paper. This was used by six assessors on a sample of papers and inter-rater reliability was tested. Further work is required to refine the template and its definitions, and to improve its consistency in application. • Nevertheless, for initial analysis, it was applied to 623 out of the 799 2nd generation papers. A four point scale was used for the importance of the cited paper to the citing paper. In just 9% of cases was the cited 1st generation paper thought to be in one of the top two categories, ie of Considerable or Essential importance to the citing paper. • Statistical analysis revealed no relationship between the number of citations a paper received and the proportion of citations where the cited paper was classified as being of high (ie. Considerable or Essential) importance to the citing paper. Self-citations, however, were shown to be significantly more likely to be in this category. • The classification of the type of research (basic or clinical) by our analysis of each paper broadly agreed with the classification of the journals by Research Level. • The time constraints involved in applying the template, plus the lack of any overall pattern in terms of correlations between number and importance of citations, might point to the desirability of adopting a more selective approach, guided by qualitative analysis. In any selective approach, however, it is likely that self-citations should feature. iv Chapter 4 : Qualitative analysis • Given the number of co-authors, it seemed appropriate to send them a questionnaire rather than attempt to interview them. Therefore the interviewing was rather more concentrated than originally intended. Only one formal critical pathway was created, but it was undertaken by an expert in the field who worked with Alberti at Newcastle. • Some problems emerged in taking 1981 as the starting point for the study. Alberti identified 10 selected papers from the 1970s and 1980s that he felt had had most impact on clinical practice. These helped to give us both a better understanding of the payback from our 1st generation, or 1981, papers, and provided further material for analysis. • Attempting to describe the impact from the 1981 body of work, and from the 10 selected papers, underlines the complex reality of how science advances and influences clinical practice. If they make a contribution at all, most studies make a small, incremental one. • A few papers, however, have been shown to have a considerably greater impact. A possible key to the level of payback indicated is the enormous breadth of Alberti’s contacts, and fields and methods of working, to which various references were made. This is well illustrated in the account of how the idea for subcutaneous pumps came about. Similarly, the ability to produce the very important guidelines on treating diabetics during surgery, and diabetic coma, partly resulted from the application to clinical problems of the understandings gained from some of the basic/early clinical studies. It is significant that the key papers on these issues, all of which come from the list of 10 selected papers from the 1970s and 1980s, were having an impact on the 1981 work. • How far the collection of papers from 1981 have been drawn upon in similar ways is less clear. Nevertheless, papers on treating diabetics during open heart surgery, and on bolus delivery of insulin at meal times, were key parts of these wider streams, despite variable citation levels. Furthermore, various papers, including on acarbose, on portal infusion of insulin, and on semi-human insulin, were important steps in bodies of work in their respective areas. The complexity was illustrated by a paper that helped debunk the Chlorpropamide alcohol flushing hypothesis, and thus end a line of scientific enquiry: there was payback in stopping an incorrect line of inquiry, but nothing on which to build. • Each technique in the qualitative study produced information about the successful subsequent careers followed by many researchers trained through working with Alberti. • Historical perspectives, and insider expert opinions, were important in the qualitative analysis. Overall, the qualitative methods highlighted some limitations in the bibliometric approach but also showed how aspects of the citation analysis can complement the opinions expressed, for example about the importance of the breadth of Alberti’s work. v Chapter 5 : Lessons learnt and the way forward • Lessons learnt: a variety of methods can be used successfully to gather considerable data about the payback from a body of research undertaken 20 years ago. Traditional citation analysis alone, however, is not sufficient: the importance of the surgery papers despite their relatively low citation rates illustrates this. The qualitative methods are important and much of the analysis is strengthened by drawing on multiple approaches. Several problems remain, including: identifying a coherent starting point for the analysis; coping with the enormous number of papers involved in later generations; and refining the template for categorising citations and developing ways of fully utilising the results from applying it. • Preparing for the large-scale study: this preliminary study provides a basis on which to attempt to undertake the larger study we envisaged. Issues now being addressed include identification of the level of bibliometric/citation analysis necessary to complement any qualitative studies. To provide confidence in the findings from an eventual large-scale study, we will need to expand the focus. The study will need to cover at least four sets of case studies. Ideally, each set should focus on a number of research groups working in a country in the same field. We hope there will be sets of case studies in two or three fields and in at least two countries. The issues to be explored will include ones highlighted by this study such as breadth of work, level of collaboration, and the role of core funding. • Methods for the large-scale study: for each case study we now propose to employ two methodological elements based on the qualitative and quantitative techniques adopted in the preliminary study. They will work in parallel but the quantitative bibliometric analysis would be applied selectively to parts of ‘research lines’ (ie discrete themes of research) identified in the qualitative studies as being important in influencing clinical practice. • Presenting the findings: each research line could be written-up in a standardised document that would use the HERG payback model and categories to describe the impact of that research. We shall use the qualitative and quantitative data to compare and contrast the ‘payback’ of research lines by country and disease, and then identify common factors that correlate with the translation of basic or early clinical research. • Concluding comments: in the era of ‘evidence based policy’, research funders are looking for value for money in the research they support and for evidence on the effectiveness of different research strategies. In this study we have begun developing a methodology that will allow us to understand the complexity of research development over a series of generations. The utility of the policy research we propose here will only be realised when it is scaled up to cover a number of different fields in different settings.NHS Executive, London Regio