Supporting the Quality Assurance of a Scientific Framework

The quality assurance of scientific software has to deal with special challenges of this type of software, including missing test oracles, the need for high performance computing, and the high priority of non-functional requirements. A scientific framework consists of common code, which provides solutions for several similar mathematical problems. The various possible uses of a scientific framework lead to a large variability in the framework. In addition to the challenges of scientific software, the quality assurance of a scientific framework needs to find a way of dealing with the large variability. In software product line engineering (SPLE), the idea is to develop a software platform and then use mass customization for the creation of a group of similar applications. In this thesis, we show how SPLE, in particular variability modeling, can be applied to support the quality assurance of scientific frameworks. One of the main contributions of this thesis is a process for the creation of reengineering variability models for a scientific framework based on its mathematical requirements. Reengineering means the adjustment of a software system to improve the software quality, mostly without changing the software’s functionality. In our research, the variability models are created for existing software and therefore we call them reengineering variability models. The created variability models are used for a systematic development of system test applications for the framework. Additionally, we developed a model-based method for test case derivation for the system test applications based on the variability models. Furthermore, we contribute a software product line test strategy for scientific frameworks. A test strategy strongly influences the test activities performed. Another main contribution of this thesis is the design of a quality assurance process for scientific frameworks, which combines the test activities of the test strategy with other quality assurance activities. We introduce a list of special characteristics for scientific software, which we use as rationale for the design of this process. We report on a case study, analyzing the feasibility and acceptance by developers for two parts of the design of the quality assurance process: variability model creation and desk-checking, a kind of lightweight review. Using FeatureIDE, an environment for feature-oriented software development as well as an automated test environment, we prototypically demonstrate the applicability of our approach

Innovation dynamics and the role of infrastructure

This report shows how the role of the infrastructure – standards, measurement, accreditation, design and intellectual property – can be integrated into a quantitative model of the innovation system and used to help explain levels and changes in labour productivity and growth in turnover and employment. The summary focuses on the new results from the project, set out in more detail in Sections 5 and 6. The first two sections of the report provide contextual material on the UK innovation system, the nature and content of the infrastructure knowledge and the institutions that provide it. Mixed modes of innovation, the typology of innovation practices developed and applied here, is constituted of six mixed modes, derived from many variables taken from the UK Innovation Survey. These are: Investing in intangibles Technology with IP innovating Using codified knowledge Wider (managerial) innovating Market-led innovating External process modernising. The composition of the innovation modes, and the approach used to compute them, is set out in more detail in Section 4. Modes can be thought of as the underlying process of innovation, a bundle of activities undertaken jointly by firms, and whose working out generates well known indicators such as new product innovations, R&D spending and accessing external information, that are the partial indicators gathered from the innovation survey itself

Ireland

This report will focuses on the current innovations and the future development of the practices and approaches to the assessment of learning in the area of work-based Vocational Education & Training in Ireland. The report is written from the perspective of the Irish Partner (Dublin City University) of the Leonardo da Vinci QualPraxis Research Project. In Ireland Vocational Education and Training (VET) exists mainly in the further education sector and this report will focus on this area

The transformation of steering and governance in Higher Education: funding and evaluation as policy instruments.

This paper focuses on policy implementation in higher education (HE) to be analysed through the evolution and transformation of the policy instruments, namely those related to the Government funding and evaluation. The research questions are: to what extent instruments can reveal the evolution of policy rationales and justifications? How instruments emerged, and become institutionalised, affecting and being affected by the characteristics of national configuration of HE systems? Whether and how they produce desired effects or evolve in unpredictable ways, generating unexpected results, playing new roles and functionalities? The evolution of the instruments seems to be dependent on some characteristics of the context and some key features of the instruments. The development has been often inspired by NPM principles, which aimed at increasing steering capacity of the policy maker on one side, and university role and autonomy on the other. The common narrative is then declined in very different ways among countries, and instruments implementation reveals the extent to which it is adapted to the existing characters (dominant paradigm) of the HE system.Higher Education, Funding, Evaluation, Policy instruments, Policy implementation

Clinical audit project in undergraduate medical education curriculum: An assessment validation study

Objectives: To evaluate the merit of the Clinical Audit Project (CAP) in an assessment program for undergraduate medical education using a systematic assessment validation framework. Methods: A cross-sectional assessment validation study at one medical school in Western Australia, with retrospective qualitative analysis of the design, development, implementation and outcomes of the CAP, and quantitative analysis of assessment data from four cohorts of medical students (2011-2014). Results: The CAP is fit for purpose with clear external and internal alignment to expected medical graduate outcomes. Substantive validity in students’ and examiners’ response processes is ensured through relevant methodological and cognitive processes. Multiple validity features are built-in to the design, planning and implementation process of the CAP. There is evidence of high internal consistency reliability of CAP scores (Cronbach’s alpha \u3e 0.8) and inter-examiner consistency reliability (intra-class correlation\u3e0.7). Aggregation of CAP scores is psychometrically sound, with high internal consistency indicating one common underlying construct. Significant but moderate correlations between CAP scores and scores from other assessment modalities indicate validity of extrapolation and alignment between the CAP and the overall target outcomes of medical graduates. Standard setting, score equating and fair decision rules justify consequential validity of CAP scores interpretation and use. Conclusions: This study provides evidence demonstrating that the CAP is a meaningful and valid component in the assessment program. This systematic framework of validation can be adopted for all levels of assessment in medical education, from individual assessment modality, to the validation of an assessment program as a whole

External examining: fit for purpose?

In a context of international concern about academic standards, the practice of external examining is widely admired for its role in defending standards. Yet a contradiction exists between this faith in examining and continuing concerns about standards. This article argues that external examining rests on assumptions about standards which are significantly open to challenge. Six assumptions relating to the conceptual context, the operation and the nature of examiners themselves are analysed drawing on a review of the available evidence. The analysis challenges the notion of a consensus on standards and the potential to vest in individuals the ability to represent that consensus when judging the comparability of academic standards in a stable and appropriate way. The issues raised have relevance to the UK and to other national systems using external examiners or seeking to guarantee academic standards by, in some cases, adopting quality assurance approaches developed in the UK

Reclaiming scholarship as an integrating dimension of academic work for the impact of research on teaching and learning in higher education

This paper offers reflections on Ernest Boyer’s conceptions of scholarship which were first outlined twenty years earlier. It considers the ways in which such conceptions have become meaninglessness as part of a wider process of reductionism and how this relates to policy discussions on curriculum, competence and teaching. It argues for the central place of research in, and the central place of, Higher Education in the professional education of teachers. It proposes that in order to develop cultures of inquiry in Higher Education we need to do so in ways that integrate research, teaching and learning. The integration process is seen to be achieved through an understanding of the nature of wholes and through a process of seeing, and therefore thinking and acting, holistically. It is argued that this is part of a wider struggle over values which the academic community needs to advance by reclaiming the meaning of scholarship

The discipline of Natural Design

If we define design work as those cognitive and practical things to which designers give their valuable effort, then our Natural Design framework allows the recording and replaying of design work. Natural Design provides a meta-structural framework that has developed through our observations of engineering design in safety and mission critical industries, such as aircraft design. Our previous work has produced parametrisable models of design work for software intensive systems, and we now look to make an initial assessment of our natural design framework for its fit to the more creative design practices. In this paper we briefly sketch the framework and subsequently attempt to locate ‘creativity’ in it. We find that, although there are good strong hooks for what the designer does, we are forced to find a role for the community of the designer in the creative process in our framework, something that was only implicit in our previous work. Keywords: Natural design; Engineering design; Creativity</p