Information systems evaluation methodologies

Due to the prevalent use of Information Systems (IS) in modern organisations nowadays, evaluation research in this field is becoming more and more important. In light of this, a set of rigorous methodologies were developed and used by IS researchers and practitioners to evaluate the increasingly complex IS implementation used. Moreover, different types of IS and different focusing perspectives of the evaluation require the selection and use of different evaluation approaches and methodologies. This paper aims to identify, explore, investigate and discuss the various key methodologies that can be used in IS evaluation from different perspectives, namely in nature (e.g. summative vs. formative evaluation) and in strategy (e.g. goal-based, goal-free and criteria-based evaluation). The paper concludes that evaluation methodologies should be selected depending on the nature of the IS and the specific goals and objectives of the evaluation. Nonetheless, it is also proposed that formative criteria-based evaluation and summative criteria-based evaluation are currently among the most and more widely used in IS research. The authors suggest that the combines used of one or more of these approaches can be applied at different stages of the IS life cycle in order to generate more rigorous and reliable evaluation outcomes

Software component testing : a standard and the effectiveness of techniques

This portfolio comprises two projects linked by the theme of software component testing, which is also often referred to as module or unit testing. One project covers its standardisation, while the other considers the analysis and evaluation of the application of selected testing techniques to an existing avionics system. The evaluation is based on empirical data obtained from fault reports relating to the avionics system. The standardisation project is based on the development of the BC BSI Software Component Testing Standard and the BCS/BSI Glossary of terms used in software testing, which are both included in the portfolio. The papers included for this project consider both those issues concerned with the adopted development process and the resolution of technical matters concerning the definition of the testing techniques and their associated measures. The test effectiveness project documents a retrospective analysis of an operational avionics system to determine the relative effectiveness of several software component testing techniques. The methodology differs from that used in other test effectiveness experiments in that it considers every possible set of inputs that are required to satisfy a testing technique rather than arbitrarily chosen values from within this set. The three papers present the experimental methodology used, intermediate results from a failure analysis of the studied system, and the test effectiveness results for ten testing techniques, definitions for which were taken from the BCS BSI Software Component Testing Standard. The creation of the two standards has filled a gap in both the national and international software testing standards arenas. Their production required an in-depth knowledge of software component testing techniques, the identification and use of a development process, and the negotiation of the standardisation process at a national level. The knowledge gained during this process has been disseminated by the author in the papers included as part of this portfolio. The investigation of test effectiveness has introduced a new methodology for determining the test effectiveness of software component testing techniques by means of a retrospective analysis and so provided a new set of data that can be added to the body of empirical data on software component testing effectiveness

Intelligent sampling for the measurement of structured surfaces

Uniform sampling in metrology has known drawbacks such as coherent spectral aliasing and a lack of efficiency in terms of measuring time and data storage. The requirement for intelligent sampling strategies has been outlined over recent years, particularly where the measurement of structured surfaces is concerned. Most of the present research on intelligent sampling has focused on dimensional metrology using coordinate-measuring machines with little reported on the area of surface metrology. In the research reported here, potential intelligent sampling strategies for surface topography measurement of structured surfaces are investigated by using numerical simulation and experimental verification. The methods include the jittered uniform method, low-discrepancy pattern sampling and several adaptive methods which originate from computer graphics, coordinate metrology and previous research by the authors. By combining the use of advanced reconstruction methods and feature-based characterization techniques, the measurement performance of the sampling methods is studied using case studies. The advantages, stability and feasibility of these techniques for practical measurements are discussed

INCENTIVE CONTRACTS TO MEET FUNCTIONAL CHARACTERISTICS IN WHEAT PURCHASING

Consistency of functional characteristics in hard red spring (HRS) wheat is a concern confronting sellers and buyers. This research analyzes contract incentives for importers with respect to cost and potential risk of acceptance. A principal-agent framework is utilized to examine contract incentives. In the principal-agent contract, the principal offers the contract, the agent rejects or accepts the contract, and then decides how much effort to apply. All this is subject to risk for the agent and moral hazard for the principal. An example is presented, for which equilibrium contract terms are a base price of 454 cents per bushel for low quality wheat and a premium of 36 cents per bushel if high quality is achieved. The premium for high quality is unchanged as the agent's outside option increases, but increases as the probability of conformance with high effort declines or as the agent's cost of high effort increases. Small changes in several of the parameter values (agent's outside option, agent's cost of high/low effort, principal value for high/low effort, and principal's outside options if the contract was not extended or if the agent rejects the contract) result in the principal not offering a contract.Incentive Contact, Functional Characteristic, Wheat, Principal-Agent, Crop Production/Industries,

Synthesis and final recommendations on the development of a European Information System for Organic Markets. = Deliverable D6 of the European Project EISfOM QLK5-2002-02400

Executive summary European markets for organic products are growing rapidly, but the market information available in most European countries is woefully inadequate. Often only very basic data such as certified organic holdings and land area are reported, and sometimes not even individual crop areas or livestock numbers. Important market data, such as the amount of production, consumption, international trade or producer and consumer prices, do not exist in most European countries. In some European countries there are only rough estimates of the levels of production and consumption. There is no standardisation and data are seldom comparable. Furthermore, detailed information on specific commodities is missing. Hence, investment decisions are taken under conditions of great uncertainty. Policy evaluation, including periodic monitoring of the European Action Plan for Organic Food and Farming and RDP 2007-2013, will require many other data in addition to those regarding production structures and financial data that are already available, but obtaining this information would require a new EU-wide data collection and processing system (DCPS) to be put in place. The European Information System for Organic Markets (EISfOM) project is an EUfunded Concerted Action which has analysed and documented the current situation and proposed ways in which organic data collection and processing systems (DCPS) can be improved by means of: • improvement in the current situation of data collecting and processing systems for the organic sector • innovation in data collection and processing systems for the organic sector • integration of conventional and organic data collection and processing systems This report summarises the most relevant findings of the EISfOM project, which are analysed in the main project reports: Wolfert, S., Kramer, K. J., Richter, T., Hempfling, G., Lux. S. and Recke, G. (eds.) (2004). Review of data collection and processing systems for organic and conventional markets. EISfOM (QLK5-2002-02400) project deliverable submitted to European Commission. www.eisfom.org/publications. Recke, G., Hamm, U., Lampkin, N., Zanoli, R., Vitulano, S. and Olmos, S. (eds.) (2004a) Report on proposals for the development, harmonisation and quality assurance of organic data collection and processing systems (DCPS). EISfOM (QLK5-2002-02400) project deliverable submitted to European Commission. www.eisfom.org/publications. Recke, G., Willer, H., Lampkin, N. and Vaughan, A. (eds.) (2004b). Development of a European Information System for Organic Markets – Improving the Scope and Quality of Statistical Data. Proceedings of the 1st EISfOM European Seminar, Berlin, Germany, 26-27 April, 2004. Research Institute of Organic Agriculture (FiBL), Frick, Switzerland. www.eisfom.org/publications. Gleirscher, N., Schermer, M., Wroblewska, M. and Zakowska-Biemans, S. (2005) Report on the evaluation of the pilot case studies. EISfOM (QLK5-2002-02400) project deliverable submitted to European Commission. www.eisfom.org/publications. QLK5-2002-02400 European Information System for Organic Markets (EISfOM) D6 final report Rippin, M. and Lampkin, N. (eds.) (2005) Framework for a European Information System for Organic Markets. Unpublished report of the project European Information System for Organic Markets (EISfOM) (QLK5-2002-02400). Rippin, M., Willer, H., Lampkin, N., and Vaughan A. (2006). Towards a European Framework for Organic Market information, Proceedings of the 2nd EISfOM European Seminar, Brussels, November 10 and 11, 2005. Research Institute of Organic Agriculture (FiBL), Frick, Switzerland. www.eisfom.org/publications