Information systems evaluation methodologies

Due to the prevalent use of Information Systems (IS) in modern organisations nowadays, evaluation research in this field is becoming more and more important. In light of this, a set of rigorous methodologies were developed and used by IS researchers and practitioners to evaluate the increasingly complex IS implementation used. Moreover, different types of IS and different focusing perspectives of the evaluation require the selection and use of different evaluation approaches and methodologies. This paper aims to identify, explore, investigate and discuss the various key methodologies that can be used in IS evaluation from different perspectives, namely in nature (e.g. summative vs. formative evaluation) and in strategy (e.g. goal-based, goal-free and criteria-based evaluation). The paper concludes that evaluation methodologies should be selected depending on the nature of the IS and the specific goals and objectives of the evaluation. Nonetheless, it is also proposed that formative criteria-based evaluation and summative criteria-based evaluation are currently among the most and more widely used in IS research. The authors suggest that the combines used of one or more of these approaches can be applied at different stages of the IS life cycle in order to generate more rigorous and reliable evaluation outcomes

Quality-testing the legal Internet: finding law with the SOSIG Law Gateway

English language version of a preprint of an article by Steven Whittle, Information Systems Manager at the Institute of Advanced Legal Studies, published in Recht Bibliothek Dokumentation, the journal of the AjBD Arbeitsgemeinschaft für juristisches Bibliotheks-und Dokumentationswese

Developing communication tools for resource management in western Alaska: an evaluation of the Western Alaska Landscape Conservation Cooperative coastal projects database

Master's Project (M.S.) University of Alaska Fairbanks, 2017Science communication is an essential component in decision-making for resource management in Alaska. This field aids in bridging knowledge gaps between scientists and diverse stakeholders. In 2014, the Western Alaska LCC developed a database cataloging the current coastal change projects in order to facilitate collaboration amongst researchers, managers, and the surrounding communities. In order to better inform similar outreach projects in other LCC regions, this MNRM project entailed an evaluation of this database between April and September 2016 and comprised a ten-question phone interview with the database participants and other involved personnel. Results from this evaluation can help refine the database to better suit its users' needs in the future, and it can also inform the creation of similar tools in other LCC regions. This project evaluated the use and usability of the Western Alaska LCC Coastal Change Database. First, I review coastal change and its impacts on Western Alaska. Next, I explore how institutions can respond to these changes and what resources they can use, including decision-support tools. I then provide examples of different decision-support tools (both in academic literature and in Alaskan projects) and discuss methodologies for evaluating their use. Interview results are then reported. The evaluation of the WALCC Coastal Change Database indicated that the tool was mostly used to enhance general understanding of the research occurring in the region. Respondents were less likely to use it for time-intensive tasks such as collaboration. Respondents also indicated that a place exists for tools like this database to flourish, but they need 1) persistent outreach, 2) a dynamic design, and 3) immediate benefits for users' time. In the future, regular updates and frequent outreach could improve the database's usability and help maintain its credibility

An introduction to STRIKE : STRuctured Interpretation of the Knowledge Environment

Knowledge forms a critical part of the income generation of the system and the complex environment in which actors participate in the creation of knowledge assets merits robust, eclectic consideration. STRIKE - STRuctured Interpretation of the Knowledge Environment affords an unobtrusive and systematic framework to observe, record, evaluate and articulate concrete and abstract elements of a setting, across internal and external dimensions. Inter-relationships between actor and environment are preserved. STRIKE is supported by underlying techniques to enrich data and enhance the authenticity of its representation. Adoption of photography and videography tools provides illustrative and interpretive benefits and facilitates researcher reflexivity. This structured approach to data analysis and evaluation mitigates criticisms of methodological rigour in observational research and affords standardisation potential, germane for application in a verification or longitudinal capacity. Advancing exploratory validation studies, the method is employed to evaluate the knowledge environments of two enterprises in the UK creative sector. These occupy a critical role in fostering entrepreneurial innovation alongside participant self-efficacy. Access Space in Sheffield and the Bristol Hackspace are committed to open software, open knowledge and open participation; sharing peer learning, creativity and socio-technical aims to address broadly similar community needs. Drawing on Wittgenstein’s Picture Theory of Meaning, the knowledge management perspective is abstracted from the STRIKE assessment. It is argued that the tiered analytical approach which considers a breadth of dimensions enhances representation and interpretation of the knowledge environment and presents a diagnostic and prescriptive capability to actualise change. The paper concludes by evaluating framework effectiveness, findings application and future direction

Investigation to enhance sustainable improvements in high speed rail transport

Transport systems are essential for the life of modern societies and economies. A sustainable transport system can shape a sustainable development pattern and socio-economic attributes of urban centres and regions. The use of private cars and trucks is increasing in most countries, creating more congestion, accidents, pollution and energy consumption. Many governments desire to achieve growth in public transport to overcome these adverse trends. A massive shift toward an environmentally sound type of transport is crucial and railways are deemed to be one of the most sustainable modes. All over the world the railway industry is involved in a renewal to reform and up-date rail, prompted largely by environmental concerns. The trend is to develop speed-competitive systems to expand transportation capacity. The focus of the current research, which is at its commencing stages, is to investigate the opportunities to apply an alternative approach to railway operations to overcome the difficulty of high speed transport in servicing larger amounts of demand, while achieving minimum point to point travel time, in a viable and integrated environment for both passenger and freight services. The expected outcome of the research project is to present a framework that may be used to identify and evaluate the most cost-effective transport solution to service not only major cities, but also regional centres along an interregional rail corridor, thus providing greater benefits on local economies and to build a spine for future development

The Widening and Deepening of Innovation Policy: What Conditions Provide for Effective Governance?

In relation to the gradual and steady introduction of the systemic perspective and of new public management techniques in innovation policy-making during the past decade, many countries in the developed and developing world have been substantially widening and deepening their innovation policies. The introduction of new and more sophisticated policy instruments (deepening) has been accompanied by an expansion of the realm of action for innovation policy (widening). The main argument of this paper is that this remarkable governmental activism and experimentalism raises important analytical questions about the conditions under which innovation policy contributes to an effective governance of the innovation system. Hence, this paper has two main purposes. Firstly, it characterises in an unambiguous way the widening and deepening trends in innovation policy, problematising their possible effects on governance. And, secondly, it develops an analytical toolbox based on a series of theoretical assumptions about the political conditions for effective governance of innovation systems.Innovation Policy; Innovation System; Governmental Activism; Governmental Experimentalism