Information systems evaluation methodologies

Due to the prevalent use of Information Systems (IS) in modern organisations nowadays, evaluation research in this field is becoming more and more important. In light of this, a set of rigorous methodologies were developed and used by IS researchers and practitioners to evaluate the increasingly complex IS implementation used. Moreover, different types of IS and different focusing perspectives of the evaluation require the selection and use of different evaluation approaches and methodologies. This paper aims to identify, explore, investigate and discuss the various key methodologies that can be used in IS evaluation from different perspectives, namely in nature (e.g. summative vs. formative evaluation) and in strategy (e.g. goal-based, goal-free and criteria-based evaluation). The paper concludes that evaluation methodologies should be selected depending on the nature of the IS and the specific goals and objectives of the evaluation. Nonetheless, it is also proposed that formative criteria-based evaluation and summative criteria-based evaluation are currently among the most and more widely used in IS research. The authors suggest that the combines used of one or more of these approaches can be applied at different stages of the IS life cycle in order to generate more rigorous and reliable evaluation outcomes

Usability of Data Warehousing and Data Mining for Interactive Decision Making in Textile Sector

Data warehouse is one of the most rapidly growing areas in management information system. With this approach, data for Executive Information System (EIS) and Decision Support System (DSS) applications are separated from operational data and stored in a separate database. This process is called data warehousing. The major advantages of this approach are: improved in performance, better data quality, and the ability to consolidate and summarize data from heterogeneous systems. A data warehouse is part of a larger infrastructure that includes legacy data sources, external data sources, a repository, data acquisition software, and user interface and related analytical tools. The aim of this research work is to elaborate that how the textile industry can manage and improve their production capacity and resources at optimum level to produce a good quality result using data warehousing and data mining techniques. This research work is conduction in Masood Textile Mills Limited, Faisalabad, Pakistan (MTML). The results may hopefully opened-up an era of research and methodology that could further benefit the Industry to support in decision support system

Toiminnanohjausjärjestelmän käytettävyyden parantaminen aloitteleville ja kokeneille käyttäjille

Despite benefits and importance of ERP systems, they suffer from many usability problems. They have user interfaces that are complex and suffer from "daunting usability problems". Also, their implementation success rate is relatively low and their usability significantly influences this implementation success. As a company offering an ERP system to ferry operators was planning to renew the user interface of this system in future, we investigated usability of the current system so this could guide future implementation of the new user interface. We studied new and long time users by conducting sessions where the users told about their experiences, performed tasks with the system and filled usability questionnaire (System Usability Scale). Many novice and long time users reported problems. The scores from usability questionnaire show all but two participants perceived the usability of the system as below average and in adjective rating "not acceptable". Two users rated the usability as "excellent". We reasoned that there could be a group of users who use the system in such a way and in such context that they do not experience these problems. The results indicate novices have trouble, for example, navigating and completing tasks. Also some long time users reported navigation issues. The system seems to require that it’s users remember lots of things in order to use it well. The interviews and tasks indicate the system is complex and hard to use and both novices and experts face problems. This is supported by perceived usability scores. While experts could in most cases finish all tasks, during interview some of them reported problems such as finding products the customers needed, error reporting being unclear, configuration being tedious, and need for lots of manual typing, for example. We gave recommendations on what to consider when implementing new user interface for this ERP system. For example, navigation should be improved and users should be provided with powerful search tools. ERP usability is not studied much. Our study supports use of already developed heuristics in classifying usability problems. Our recommendations how to improve usability of the ERP system studied should give some guidelines on what could be done, although not much is backed by laboratory studies. More work is needed in this field to find and test solutions to usability problems users face

The Chicken and the Pig: User Involvement in Developing Usability Heuristics

Traditionally, users have not been involved in certain usability engineering methods, although they arguably are the most important stakeholders. This paper explores the possibility of involving users in developing a set of usability heuristics for a specific type of application, an activity they are not usually involved in. Using a qualitative approach based on interviews, a focus group, and an online survey, usability experts and software users evaluated existing sets of heuristics in terms of their applicability to a specific type of application and developed new heuristics to supplement them. The results indicate that the users provide a valuable contribution to the adaptation of existing heuristics to a specific type of application. Users add a new perspective and can address problem areas that usability experts, especially those with little or no experience with the specific application area, would not have identified

Value-driven Security Agreements in Extended Enterprises

Today organizations are highly interconnected in business networks called extended enterprises. This is mostly facilitated by outsourcing and by new economic models based on pay-as-you-go billing; all supported by IT-as-a-service. Although outsourcing has been around for some time, what is now new is the fact that organizations are increasingly outsourcing critical business processes, engaging on complex service bundles, and moving infrastructure and their management to the custody of third parties. Although this gives competitive advantage by reducing cost and increasing flexibility, it increases security risks by eroding security perimeters that used to separate insiders with security privileges from outsiders without security privileges. The classical security distinction between insiders and outsiders is supplemented with a third category of threat agents, namely external insiders, who are not subject to the internal control of an organization but yet have some access privileges to its resources that normal outsiders do not have. Protection against external insiders requires security agreements between organizations in an extended enterprise. Currently, there is no practical method that allows security officers to specify such requirements. In this paper we provide a method for modeling an extended enterprise architecture, identifying external insider roles, and for specifying security requirements that mitigate security threats posed by these roles. We illustrate our method with a realistic example

Accessibility of websites of the European national tourism boards

Purpose: The aim is to find out the current state of accessibility of the websites of European national tourism boards. Furthermore, the identification of the most common errors in terms of accessibility as well as recommendations leading to their correction is aimed for. Design/methodology/approach: The study is based on methods of testing the availability of web systems. The testing included automated tools, namely AChecker and Accessibility Evaluation Tool, as well as the WCAG 2.1 checklist developed by WebAIM initiative. Findings: The research has shown a relatively high accessibility of those websites. Nevertheless, some accessibility violations have been identified that can significantly complicate the accessibility of those websites for users using various assistive devices or other alternative hardware or software means. The most commonly identified errors include: failure to use alternative text for content-relevant images, the absence of text or audio transcripts for videos shared via Youtube, missing descriptions for text form elements and missing label for search form. Practical implications: The results of the research can be used in the evaluation of web presentations at the level of tourism boards and destination management. Originality/Value: The main output of this article is the application of web testing methodology on a comprehensive set of national tourist boards.peer-reviewe