Implementation of computer assisted assessment: lessons from the literature

This paper draws attention to literature surrounding the subject of computer-assisted assessment (CAA). A brief overview of traditional methods of assessment is presented, highlighting areas of concern in existing techniques. CAA is then defined, and instances of its introduction in various educational spheres are identified, with the main focus of the paper concerning the implementation of CAA. Through referenced articles, evidence is offered to inform practitioners, and direct further research into CAA from a technological and pedagogical perspective. This includes issues relating to interoperability of questions, security, test construction and testing higher cognitive skills. The paper concludes by suggesting that an institutional strategy for CAA coupled with staff development in test construction for a CAA environment can increase the chances of successful implementation

The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises

ALT-SURF seminar: ePortfolios and Digital Repositories, 22 and 23 April 2004, Edinburgh UK

ALT (Association for Learning Technology) and SURF (collaborative organisation for IT innovation in Dutch HE) have similar goals and started working together in 1999. In 2001 they signed a Memorandum of Understanding to bring together learning technologists from the UK and the Netherlands in a variety of exchange activities. Sharing experiences and views internationally between peers proved to be an excellent way to learn new things, find inspiration to meet joint challenges and engage in healthy reflection. ALT and SURF organised mutual study visits (50 Dutch learning technologists visited universities in the UK in 2002 and 40 UK colleagues visited Dutch institutes in 2003), joint conferences and seminars, and a expert study trip to Australia (8 UK and 8 Dutch experts visited 14 institutes in 2002). SURF and ALT reported on these in presentations and publications. A new activity is the joint working seminar, where a specific theme can be discussed by experts in the field with the purpose of describing in a so-called ‘briefing paper’ the current state of affairs, the underlying beliefs, hopes and desired situation, and possible ‘next steps’ in joint conceptualisation and/or developing activities. In the process of composing the briefing papers differences between the countries will surface, e.g. organisational, cultural or political. The themes are therefore not isolated topics but seen as developments within a context. The first joint ALT-SURF working seminar took place in April 2004 in Edinburgh, immediately following an ALT spring conference, and addressed two related topics: e-(or Digital) Portfolios and Digital Repositories. The aim was to establish working relationships between experts in both countries through discussion and collaborative writing of briefing papers aimed at experts, researchers, managers and policy makers. The objectives of the working seminar were threefold: • describe the basic belief that leads to so much energy being put into these concepts • outline current thinking in ePortfolios and Digital repositories as a common reference point for deeper understanding • devise a roadmap to meet required conditions to further develop ePortfolios and Digital Repository concepts and use. This report is the result of that seminar, namely a briefing paper on ePortfolios and another on Digital Repositories. In the papers the potential and ultimate goal of using ePortfolios and Digital Repositories for learning and teaching is outlined; the current state of achievements is described, and thirdly the briefing paper concludes with desired next steps to reach the ultimate goals, what conditions need to be in place, what activities need to be set in motion. The papers highlight any apparent differences in approaches between UK and Netherlands as well as opportunities for future collaboration. SURF and ALT will use the conclusions and suggestions as input for joint future development and networking activities

HaIRST: Harvesting Institutional Resources in Scotland Testbed. Final Project Report

The HaIRST project conducted research into the design, implementation and deployment of a pilot service for UK-wide access of autonomously created institutional resources in Scotland, the aim being to investigate and advise on some of the technical, cultural, and organisational requirements associated with the deposit, disclosure, and discovery of institutional resources in the JISC Information Environment. The project involved a consortium of Scottish higher and further education institutions, with significant assistance from the Scottish Library and Information Council. The project investigated the use of technologies based on the Open Archives Initiative (OAI), including the implementation of OAI-compatible repositories for metadata which describe and link to institutional digital resources, the use of the OAI protocol for metadata harvesting (OAI-PMH) to automatically copy the metadata from multiple repositories to a central repository, and the creation of a service to search and identify resources described in the central repository. An important aim of the project was to identify issues of metadata interoperability arising from the requirements of individual institutional repositories and their impact on services based on the aggregation of metadata through harvesting. The project also sought to investigate issues in using these technologies for a wide range of resources including learning, teaching and administrative materials as well as the research and scholarly communication materials considered by many of the other projects in the JISC Focus on Access to Institutional Resources (FAIR) Programme, of which HaIRST was a part. The project tested and implemented a number of open source software packages supporting OAI, and was successful in creating a pilot service which provides effective information retrieval of a range of resources created by the project consortium institutions. The pilot service has been extended to cover research and scholarly communication materials produced by other Scottish universities, and administrative materials produced by a non-educational institution in Scotland. It is an effective testbed for further research and development in these areas. The project has worked extensively with a new OAI standard for 'static repositories' which offers a low-barrier, low-cost mechanism for participation in OAI-based consortia by smaller institutions with a low volume of resources. The project identified and successfully tested tools for transforming pre-existing metadata into a format compliant with OAI standards. The project identified and assessed OAI-related documentation in English from around the world, and has produced metadata for retrieving and accessing it. The project created a Web-based advisory service for institutions and consortia. The OAI Scotland Information Service (OAISIS) provides links to related standards, guidance and documentation, and discusses the findings of HaIRST relating to interoperability and the pilot harvesting service. The project found that open source packages relating to OAI can be installed and made to interoperate to create a viable method of sharing institutional resources within a consortium. HaIRST identified issues affecting the interoperability of shared metadata and suggested ways of resolving them to improve the effectiveness and efficiency of shared information retrieval environments based on OAI. The project demonstrated that application of OAI technologies to administrative materials is an effective way for institutions to meet obligations under Freedom of Information legislation

Object lessons : a learning object approach to e-learning for social work education

Learning objects are bite-sized digital learning resources designed to tackle the e-learning adoption problem by virtue of their scale, adaptability, and interoperability. The learning object approach advocates the creation of small e-learning resources rather than whole courses: resources that can be mixed and matched; used in a traditional or online learning environment; and adapted for reuse in other discipline areas and in other countries. Storing learning objects within a subject specific digital repository to enable search, discovery, sharing and use adds considerable value to the model. This paper explores the rationale for a learning object approach to e-learning and reflects on early experiences in developing a national learning object repository for social work education in Scotland

INSPIRAL: investigating portals for information resources and learning. Final project report

INSPIRAL's aims were to identify and analyse, from the perspective of the UK HE learner, the nontechnical, institutional and end-user issues with regard to linking VLEs and digital libraries, and to make recommendations for JISC strategic planning and investment. INSPIRAL's objectives -To identify key stakeholders with regard to the linkage of VLEs, MLEs and digital libraries -To identify key stakeholder forum points and dissemination routes -To identify the relevant issues, according to the stakeholders and to previous research, pertaining to the interaction (both possible and potential) between VLEs/MLEs and digital libraries -To critically analyse identified issues, based on stakeholder experience and practice; output of previous and current projects; and prior and current research -To report back to JISC and to the stakeholder communities, with results situated firmly within the context of JISC's strategic aims and objectives

From flowers to palms: 40 years of policy for online learning

This year sees the 40th anniversary of the first policy paper regarding the use of computers in higher education in the United Kingdom. The publication of this paper represented the beginning of the field of learning technology research and practice in higher education. In the past 40 years, policy has at various points drawn from different communities and provided the roots for a diverse field of learning technology researchers and practitioners. This paper presents a review of learning technology-related policy over the past 40 years. The purpose of the review is to make sense of the current position in which the field finds itself, and to highlight lessons that can be learned from the implementation of previous policies. Conclusions drawn from the review of 40 years of learning technology policy suggest that there are few challenges that have not been faced before as well as a potential return to individual innovation

Optimising metadata to make high-value content more accessible to Google users

Purpose: This paper shows how information in digital collections that have been catalogued using high-quality metadata can be retrieved more easily by users of search engines such as Google. Methodology/approach: The research and proposals described arose from an investigation into the observed phenomenon that pages from the Glasgow Digital Library (gdl.cdlr.strath.ac.uk) were regularly appearing near the top of Google search results shortly after publication, without any deliberate effort to achieve this. The reasons for this phenomenon are now well understood and are described in the second part of the paper. The first part provides context with a review of the impact of Google and a summary of recent initiatives by commercial publishers to make their content more visible to search engines. Findings/practical implications: The literature research provides firm evidence of a trend amongst publishers to ensure that their online content is indexed by Google, in recognition of its popularity with Internet users. The practical research demonstrates how search engine accessibility can be compatible with use of established collection management principles and high-quality metadata. Originality/value: The concept of data shoogling is introduced, involving some simple techniques for metadata optimisation. Details of its practical application are given, to illustrate how those working in academic, cultural and public-sector organisations could make their digital collections more easily accessible via search engines, without compromising any existing standards and practices

DRIVER Technology Watch Report

This report is part of the Discovery Workpackage (WP4) and is the third report out of four deliverables. The objective of this report is to give an overview of the latest technical developments in the world of digital repositories, digital libraries and beyond, in order to serve as theoretical and practical input for the technical DRIVER developments, especially those focused on enhanced publications. This report consists of two main parts, one part focuses on interoperability standards for enhanced publications, the other part consists of three subchapters, which give a landscape picture of current and surfacing technologies and communities crucial to DRIVER. These three subchapters contain the GRID, CRIS and LTP communities and technologies. Every chapter contains a theoretical explanation, followed by case studies and the outcomes and opportunities for DRIVER in this field

The NHS-HE Forum

This paper discusses the formation of the NHS-HE Forum by Professor Roland Rosner in 2001. It looks at the aims and objectives of the Forum and maps current progress. The second part of the paper looks particularly at the NHS-HE Procurement Group and the later JISC NHS-HE Procurement Group, which are part of the Forum. These groupings have been partially successful in conducting joint procurement activity across the NHS and HE sectors. The publication of Dr Ian Gibson?s Parliamentary Report Scientific Publications: Free for All? is seminal in progressing this joint activity. As of 1 February 2006, joint membership of the Public Library of Science has been achieved, but one joint procurement activity of commercial content has failed. Further joint procurement is planned, along with a mapping study of procurement activities and work on an Athens Account Linking Project. The library and informatics research components are coming to the fore with the use of a community-wide survey to establish the current situation and future plans for NHS-HE network connectivity and similarly the Mapping Study of Procurement Practices in the NHS and HE for content such as e-journals. This is moving to a more systematic approach than previously. The attempted joint procurement of selected journals has two joint aims: to support both research and clinical practice in the NHS and HE. This paper discusses the formation of the NHS-HE Forum by Professor Roland Rosner in 2001. It looks at the aims and objectives of the Forum and maps current progress. The second part of the paper looks particularly at the NHS-HE Procurement Group and the later JISC NHS-HE Procurement Group, which are part of the Forum. These groupings have been partially successful in conducting joint procurement activity across the NHS and HE sectors. The publication of Dr Ian Gibson?s Parliamentary Report Scientific Publications: Free for All? is seminal in progressing this joint activity. As of 1 February 2006, joint membership of the Public Library of Science has been achieved, but one joint procurement activity of commercial content has failed. Further joint procurement is planned, along with a mapping study of procurement activities and work on an Athens Account Linking Project. The library and informatics research components are coming to the fore with the use of a community-wide survey to establish the current situation and future plans for NHS-HE network connectivity and similarly the Mapping Study of Procurement Practices in the NHS and HE for content such as e-journals. This is moving to a more systematic approach than previously. The attempted joint procurement of selected journals has two joint aims: to support both research and clinical practice in the NHS and HE