Performance evaluation of consumer decision support systems

Consumer decision support systems (CDSSs) help online users make purchasing decisions in e-commerce Web sites. To more effectively compare the usefulness of the various functionalities and interface features of such systems, we have developed a simulation environment for decision tasks of any scale and structure. Furthermore, we have identified three criteria in an evaluation framework for assessing the quality of such CDSSs: users' cognitive effort, preference expression effort, and decision accuracy. A set of experiments carried out in such simulation environments showed that most CDSSs employed in e-commerce Web sites are suboptimal. On the other hand, a hybrid decision strategy based on four existing ones was found to be more effective. The interface improvements based on the new strategy correspond to some of the advanced tools already developed in the research field. This result is therefore consistent with our earlier work on evaluating CDSSs with real users. That is, some advanced tools do produce more accurate decisions while requiring a comparable amount of user effort. However, the simulation environment enables us to efficiently compare more advanced tools among themselves, and indicate further opportunities for functionality and interface improvement

Heuristic Optimization of Consumer Electricity Costs Using a Generic Cost Model

Many new demand response strategies are emerging for energy management in smart grids. Real-Time Energy Pricing (RTP) is one important aspect of consumer Demand Side Management (DSM), which encourages consumers to participate in load scheduling. This can help reduce peak demand and improve power system efficiency. The use of Intelligent Decision Support Systems (IDSSs) for load scheduling has become necessary in order to enable consumers to respond to the changing economic value of energy across different hours of the day. The type of scheduling problem encountered by a consumer IDSS is typically NP-hard, which warrants the search for good heuristics with efficient computational performance and ease of implementation. This paper presents an extensive evaluation of a heuristic scheduling algorithm for use in a consumer IDSS. A generic cost model for hourly pricing is utilized, which can be configured for traditional on/off peak pricing, RTP, Time of Use Pricing (TOUP), Two-Tier Pricing (2TP) and combinations thereof. The heuristic greedily schedules controllable appliances to minimize smart appliance energy costs and has a polynomial worst-case computation time. Extensive computational experiments demonstrate the effectiveness of the algorithm and the obtained results indicate the gaps between the optimal achievable costs are negligible

State Strategies to Improve Quality and Efficiency: Making the Most of Opportunities in National Health Reform

Examines ten states' initiatives to address key components of quality and efficiency improvement, including data collection, aggregation, and standardization; public reporting; payment reform; consumer engagement; and provider engagement

MOVIEMOD: An Implementable Decision-Support System for Prerelease Market Evaluation of Motion Pictures

In spite of the high financial stakes involved in marketing new motion pictures, marketing science models have not been applied to the prerelease market evaluation of motion pictures. The motion picture industry poses some unique challenges. For example, the consumer adoption process for movies is very sensitive to word-of-mouth interactions, which are difficult to measure and predict before the movie has been released. In this article, we undertake the challenge to develop and implement MOVIEMOD—a prerelease market evaluation model for the motion picture industry. MOVIEMOD is designed to generate box-office forecasts and to support marketing decisions for a new movie after the movie has been produced (or when it is available in a rough cut) but before it has been released. Unlike other forecasting models for motion pictures, the calibration of MOVIEMOD does not require any actual sales data. Also, the data collection time for a product with a limited lifetime such as a movie should not take too long. For MOVIEMOD it takes only three hours in a “consumer clinic” to collect the data needed for the prediction of box-office sales and the evaluation of alternative marketing plans. The model is based on a behavioral representation of the consumer adoption process for movies as a macroflow process. The heart of MOVIEMOD is an interactive Markov chain model describing the macro-flow process. According to this model, at any point in time with respect to the movie under study, a consumer can be found in one of the following behavioral states: undecided, considerer, rejecter, positive spreader, negative spreader, and inactive. The progression of consumers through the behavioral states depends on a set of movie-specific factors that are related to the marketing mix, as well as on a set of more general behavioral factors that characterize the movie-going behavior in the population of interest. This interactive Markov chain model allows us to account for word-of-mouth interactions among potential adopters and several types of word-of-mouth spreaders in the population. Marketing variables that influence the transitions among the states are movie theme acceptability, promotion strategy, distribution strategy, and the movie experience. The model is calibrated in a consumer clinic experiment. Respondents fill out a questionnaire with general items related to their movie-going and movie communication behavior, they are exposed to different sets of information stimuli, they are actually shown the movie, and finally, they fill outpostmovie evaluations, including word-of-mouth intentions.These measures are used to estimate the word-of-mouth parameters and other behavioral factors, as well as the movie-specific parameters of the model. MOVIEMOD produces forecasts of the awareness, adoption intention, and cumulative penetration for a new movie within the population of interest for a given base marketing plan. It also provides diagnostic information on the likely impact of alternative marketing plans on the commercial performance of a new movie. We describe two applications of MOVIEMOD: One is a pilot study conducted without studio cooperation in the United States, and the other is a full-fledged implementation conducted with cooperation of the movie\u27s distributor and exhibitor in the Netherlands. The implementations suggest that MOVIEMOD produces reasonably accurate forecasts of box-office performance. More importantly, the model offers the opportunity to simulate the effects of alternative marketing plans. In the Dutch application, the effects of extra advertising, extra magazine articles, extra TV commercials, and higher trailer intensity (compared to the base marketing plan of the distributor) were analyzed. We demonstrate the value of these decision-support capabilities of MOVIEMOD in assisting managers to identify a final plan that resulted in an almost 50% increase in the test movie\u27s revenue performance, compared to the marketing plan initially contemplated. Management implemented this recommended plan, which resulted in box-office sales that were within 5% of the MOVIEMOD prediction. MOVIEMOD was also tested against several benchmark models, and its prediction was better in all cases. An evaluation of MOVIEMOD jointly by the Dutch exhibitor and the distributor showed that both parties were positive about and appreciated its performance as a decision-support tool. In particular, the distributor, who has more stakes in the domestic performance of its movies, showed a great interest in using MOVIEMOD for subsequent evaluations of new movies prior to their release. Based on such evaluations and the initial validation results, MOVIEMOD can fruitfully (and inexpensively) be used to provide researchers and managers with a deeper understanding of the factors that drive audience response to new motion pictures, and it can be instrumental in developing other decision-support systems that can improve the odds of commercial success of new experiential products

Exploiting Qualitative Information for Decision Support in Scenario Analysis

The development of scenario analysis (SA) to assist decision makers and stakeholders has been growing over the last few years through mainly exploiting qualitative information provided by experts. In this study, we present SA based on the use of qualitative data for strategy planning. We discuss the potential of SA as a decision-support tool, and provide a structured approach for the interpretation of SA data, and an empirical validation of expert evaluations that can help to measure the consistency of the analysis. An application to a specific case study is provided, with reference to the European organic farming business

Applying performance measures to support informed decision making at an operational level

Performance Measurement Systems (PMS) have commonly been applied to evaluate and reward performances at managerial levels, especially in the context of supply chain management. However, evidence suggests that the effective use of PMS can also positively influence the behaviour and improve performance at an operational level. The purpose of the study described in this paper is to develop a conceptual framework that adopts performance measures for ex-ante decision-making at an operational level within the supply chain. A case study at Coca-Cola Enterprises has been carried out and as a result, a conceptual framework of the PMS has been developed

An evaluation methodology for ergonomic design of electronic consumer products based on fuzzy axiomatic design

This article is posted with permission of OCP Science imprint. Copyright @ 2008 Old City Publishing Group.The development life cycle of software and electronic products has been shortened by the growth of rapid prototyping techniques. The evaluation of electronic consumer products should consider hardware and software as well as the ergonomic usability, emotional appeal and aesthetic integrity of the design. This research follows a systematic approach to develop an evaluation methodology for electronic mobile products on ergonomic design. The proposed methodology is based on fuzzy multi attribute decision making and fuzzy axiomatic design realized in three steps; determination of ergonomic attributes for electronic consumer products, determination of a representative set of alternatives, and selection of the best alternative in terms of ergonomic design by utilizing fuzzy axiomatic design. A case study is also provided to support the proposed methodology

Organic Action Plans. Development, implementation and evaluation. A resource manual for the organic food and farming sector

In 2004, the European Action Plan for Organic Food and Farming was launched. Many European countries have also developed national Organic Action Plans to promote and support organic agriculture. As part of the EU funded ORGAP project (“European Action Plan of Organic Food and Farming - Development of criteria and procedures for the evaluation of the EU Action Plan for Organic Agriculture”) a toolbox to evaluate and monitor the implementation of national and European Action Plans has been developed. In order to communicate the results of this project as widely as possible, a practical manual for initiating and evaluating Organic Action Plans has been produced. This manual has been created to inspire the people, organisations and institutions involved, or with an interest, in the organic food and farming sector to engage in the initiation, review, revision and renewal of regional, national and European Organic Action Plans. The objectives of the manual are to provide: • a tool for stakeholder involvement in future Action Plan development and implementation processes at EU, national and regional level • a guide to the use of the Organic Action Plan Evaluation Toolbox (ORGAPET) developed through the project The manual summarises the key lessons learnt from more than 10 years experience of development, implementation and evaluation of Organic Action Plans throughout Europe. The Organic Action Plan Evaluation Toolbox (ORGAPET), which includes comprehensive information to support the Organic Action Plan development and evaluation process is included with the manual as a CD-ROM, and is also accessible on-line at www.orgap.org/orgapet. The ORGAP website www.orgap.org provides a further information on the project and the European and national organic action plans. Published by: Research Institute of Organic Agriculture (FiBL), Frick, Switzerland; IFOAM EU Group, Brussels Table of contents Foreword 1 1 Introduction 3 1.1 About this manual 3 1.2 Organic farming – origins, definition & principles 6 1.3 Development of organic food & farming in Europe 8 1.3.1 Organic food and farming regulation in Europe 10 1.3.2 Policy support for organic food and farming in Europe 11 2 Organic Action Plans – what are they about? 16 2.1 Why Organic Action Plans? 16 2.2 European Organic Action Plan 21 2.3 Overview of national and regional Organic Action Plans 23 3 Planning and implementing Organic Action Plans 28 3.1 Policy development 28 3.2 Defining organic sector development needs and potential 31 3.3 Defining policy goals and objectives 34 3.4 Involving stakeholders 40 3.4.1 The case for stakeholder involvement 40 3.4.2 Identifying relevant stakeholders 42 3.4.3 Participatory approaches for stakeholders involvement 44 3.5 Decision making: selecting, integrating and prioritising relevant measures 46 3.5.1 Deciding on policy instruments and action points 47 3.5.2 Priorities for action – allocating resources 50 3.6 Implementing Organic Action Plans 52 3.7 Including monitoring and evaluation of Organic Action Plans from outset 56 3.8 Managing communication 58 3.9 Development of Action Plans in countries that joined the EU in 2004 and later 59 4 Evaluating Organic Action Plans 61 4.1 Principles of evaluation 61 4.2 Conducting an evaluation 64 4.3 Evaluating Action Plan design and implementation 70 4.3.1 Evaluating programme design and implementation processes 70 4.3.2 Evaluating programme coherence 72 4.3.3 Evaluating stakeholder involvement 74 4.4 Evaluating Action Plan effects 78 4.4.1 Developing and using indicators for evaluation 78 4.5 Overall evaluation of Organic Action Plans – judging success 85 4.6 Evaluating Action Plans in countries that joined the EU in 2004 and later 89 5 Organic Action Plans – the Golden Rules 91 5.1 Key elements of Organic Action Plan development 91 5.2 The Golden rules for Organic Action Plan 93 References 96 Annex Detailed synopsis of ORGAPET 10