Effort and accuracy analysis of choice strategies for electronic product catalogs

One crucial task for e-commerce systems is to help buyers find products that not only satisfy their preferences but also reduce their search effort. Usually the amount of available products is far beyond the upper limit that any individual could process by hand; thus product search tools are em-ployed to generate target product(s) by eliciting the buyer’s preferences and then executing some kind of choice strate-gies. We propose in this paper an extended effort–accuracy framework for measuring the performance of various choice strategies in terms of cognitive effort, elicitation effort and decision accuracy. The performance of a variety of basic choice strategies is further studied by theoretical analysis as well as empirical simulations. It shows that the perfor-mance of a given choice strategy is a tradeoff between choice accuracy and effort required from the users. The proposed framework also suggests a new efficient method of evaluat-ing the user interfaces of e-commerce systems by analyzing the performance of the underlying choice strategies. Categories and Subject Descriptor

Effort and accuracy analysis of choice strategies for electronic product catalogs

One crucial task for e-commerce systems is to help buyers find products that not only satisfy their preferences but also reduce their search effort. Usually the amount of available products is far beyond the upper limit that any individual could process by hand; thus product search tools are employed to generate target product (s) by eliciting the buyer's preferences and then executing some kind of choice strategies. We propose in this paper an extended effort-accuracy framework for measuring the performance of various choice strategies in terms of cognitive effort, elicitation effort and decision accuracy. The performance of a variety of basic choice strategies is further studied by theoretical analysis as well as empirical simulations. It shows that the performance of a given choice strategy is a tradeoff between choice accuracy and effort required from the users. The proposed framework also suggests a new efficient method of evaluating the user interfaces of e-commerce systems by analyzing the performance of the underlying choice strategies. Copyright 2005 ACM

Overcoming Incomplete User Models in Recommendation Systems Via an Ontology

To make accurate recommendations, recommendation systems currently require more data about a customer than is usually available. We conjecture that the weaknesses are due to a lack of inductive bias in the learning methods used to build the prediction models. We propose a new method that extends the utility model and assumes that the structure of user preferences follows an ontology of product attributes. Using the data of the MovieLens system, we show experimentally that real user preferences indeed closely follow an ontology based on movie attributes. Furthermore, a recommender based just on a single individual’s preferences and this ontology performs better than collaborative filtering, with the greatest differences when little data about the user is available. This points the way to how proper inductive bias can be used for significantly more powerful recommender systems in the future

Performance evaluation of consumer decision support systems

Consumer decision support systems (CDSSs) help online users make purchasing decisions in e-commerce Web sites. To more effectively compare the usefulness of the various functionalities and interface features of such systems, we have developed a simulation environment for decision tasks of any scale and structure. Furthermore, we have identified three criteria in an evaluation framework for assessing the quality of such CDSSs: users' cognitive effort, preference expression effort, and decision accuracy. A set of experiments carried out in such simulation environments showed that most CDSSs employed in e-commerce Web sites are suboptimal. On the other hand, a hybrid decision strategy based on four existing ones was found to be more effective. The interface improvements based on the new strategy correspond to some of the advanced tools already developed in the research field. This result is therefore consistent with our earlier work on evaluating CDSSs with real users. That is, some advanced tools do produce more accurate decisions while requiring a comparable amount of user effort. However, the simulation environment enables us to efficiently compare more advanced tools among themselves, and indicate further opportunities for functionality and interface improvement

Effort and Accuracy Analysis of Choice Strategies for Electronic Product Catalogs

One crucial task for e-commerce systems is to help buyers find products that not only satisfy their preferences but also reduce their search effort. Usually the amount of available products is far beyond the upper limit that any individual could process by hand; thus product search tools are employed to generate target product(s) by eliciting the buyer’s preferences and then executing some kind of choice strategies. We propose in this paper an extended effort–accuracy framework for measuring the performance of various choice strategies in terms of cognitive effort, elicitation effort and decision accuracy. The performance of a variety of basic choice strategies is further studied by theoretical analysis as well as empirical simulations. It shows that the performance of a given choice strategy is a tradeoff between choice accuracy and effort required from the users. The proposed framework also suggests a new efficient method of evaluating the user interfaces of e-commerce systems by analyzing the performance of the underlying choice strategies. Categories and Subject Descriptors H.5.2 [Information Interfaces and Presentation]: Use

User decision improvement and trust building in product recommender systems

As online stores are offering an almost unlimited shelf space, users must increasingly rely on product search and recommender systems to find their most preferred products and decide which item is the truly best one to buy. However, much research work has emphasized on developing and improving the underlying algorithms whereas many of the user issues such as preference elicitation and trust formation received little attention. In this thesis, we aim at designing and evaluating various decision technologies, with emphases on how to improve users' decision accuracy with intelligent preference elicitation and revision tools, and how to build their competence-inspired subjective constructs via trustworthy recommender interfaces. Specifically, two primary technologies are proposed: one is called example critiquing agents aimed to stimulate users to conduct tradeoff navigation and freely specify feedback criteria to example products; another termed as preference-based organization interfaces designed to take two roles: explaining to users why and how the recommendations are computed and displayed, and suggesting critique suggestions to guide users to understand existing tradeoff potentials and to make concrete decision navigations from the top candidate for better choices. To evaluate the two technologies' true performance and benefits to real-users, an evaluation framework was first established, that includes important assessment standards such as the objective/subjective accuracy-effort measures and trust-related subjective aspects (e.g., competence perceptions and behavioral intentions). Based on the evaluation framework, a series of nine experiments has been conducted and most of them were participated by real-users. Three user studies focused on the example critiquing (EC) agent, which first identified the significant impact of tradeoff process with the help of EC on users' decision accuracy improvement, and then in depth explored the advantage of multi-item strategy (for critiquing coverage) against single-item display, and higher user-control level reflected by EC in supporting users to freely compose critiquing criteria for both simple and complex tradeoffs. Another three experiments studied the preference-based organization technique. Regarding its explanation role, a carefully conducted user survey and a significant-scale quantitative evaluation both demonstrated that it can be likely to increase users' competence perception and return intention, and reduce their cognitive effort in information searching, relative to the traditional "why" explanation method in ranked list views. In addition, a retrospective simulation revealed its superior algorithm accuracy in predicting critiques and product choices that real-users intended to make, in comparison with other typical critiquing generation approaches. Motivated by the empirically findings in terms of the two technologies' respective strengths, a hybrid system has been developed with the purpose of combining them into a single application. The final three experiments evaluated its two design versions and particularly validated the hybrid system's universal effectiveness among people from different types of cultural backgrounds: oriental culture and western culture. In the end, a set of design guidelines is derived from all of the experimental results. They should be helpful for the development of a preference-based recommender system, making it capable of practically benefiting its users in improving decision accuracy, expending effort they are willing to invest, and even promoting trust in the system with resulting behavioral intentions to purchase chosen products and return to the system for repeated uses