Validation of Expert Systems: Personal Choice Expert -- A Flexible Employee Benefit System

A method for validating expert systems, based on psychological validation literature and Turing\u27s imitation game, is applied to a flexible benefits expert system. Expert system validation entails determining if a difference exists between expert and novice decisions (construct validity), if the system uses the same inputs and processes to make its decisions as experts (content validity), and if the system produces the same results as experts (criterionrelated validity). If these criteria are satisfied, then the system is indistinguishable from experts for its domain and satisfies Turing\u27s imitation game. The methods developed in this paper are applied to a human resource expert system, Personal Choice Expert (PCE), designed to help employees choose a benefits package in a flexible benefits system. Expert and novice recommendations are compared to those generated by PCE. PCE\u27s recommendations do not significantly differ from those given by experts. High inter-expert agreement exists for some benefit recommendations (e.g. Dental Care and Long-Term Disability) but not for others (e.g. Short-Term Disability and Life Insurance). Insights offered by this method are illustrated and examined

Design knowledge capture for a corporate memory facility

Currently, much of the information regarding decision alternatives and trade-offs made in the course of a major program development effort is not represented or retained in a way that permits computer-based reasoning over the life cycle of the program. The loss of this information results in problems in tracing design alternatives to requirements, in assessing the impact of change in requirements, and in configuration management. To address these problems, the problem was studied of building an intelligent, active corporate memory facility which would provide for the capture of the requirements and standards of a program, analyze the design alternatives and trade-offs made over the program's lifetime, and examine relationships between requirements and design trade-offs. Early phases of the work have concentrated on design knowledge capture for the Space Station Freedom. Tools are demonstrated and extended which helps automate and document engineering trade studies, and another tool is being developed to help designers interactively explore design alternatives and constraints

Eliciting Expertise

Since the last edition of this book there have been rapid developments in the use and exploitation of formally elicited knowledge. Previously, (Shadbolt and Burton, 1995) the emphasis was on eliciting knowledge for the purpose of building expert or knowledge-based systems. These systems are computer programs intended to solve real-world problems, achieving the same level of accuracy as human experts. Knowledge engineering is the discipline that has evolved to support the whole process of specifying, developing and deploying knowledge-based systems (Schreiber et al., 2000) This chapter will discuss the problem of knowledge elicitation for knowledge intensive systems in general

The challenge of complexity for cognitive systems

Complex cognition addresses research on (a) high-level cognitive processes – mainly problem solving, reasoning, and decision making – and their interaction with more basic processes such as perception, learning, motivation and emotion and (b) cognitive processes which take place in a complex, typically dynamic, environment. Our focus is on AI systems and cognitive models dealing with complexity and on psychological findings which can inspire or challenge cognitive systems research. In this overview we first motivate why we have to go beyond models for rather simple cognitive processes and reductionist experiments. Afterwards, we give a characterization of complexity from our perspective. We introduce the triad of cognitive science methods – analytical, empirical, and engineering methods – which in our opinion have all to be utilized to tackle complex cognition. Afterwards we highlight three aspects of complex cognition – complex problem solving, dynamic decision making, and learning of concepts, skills and strategies. We conclude with some reflections about and challenges for future research

Making mentoring work: The need for rewiring epistemology

To help produce expert coaches at both participation and performance levels, a number of governing bodies have established coach mentoring systems. In light of the limited literature on coach mentoring, as well as the risks of superficial treatment by coach education systems, this paper therefore critically discusses the role of the mentor in coach development, the nature of the mentor-mentee relationship and, most specifically, how expertise in the mentee may best be developed. If mentors are to be effective in developing expert coaches then we consequently argue that a focus on personal epistemology is required. On this basis, we present a framework that conceptualizes mentee development on this level through a step by step progression, rather than unrealistic and unachievable leap toward expertise. Finally, we consider the resulting implications for practice and research with respect to one-on-one mentoring, communities of practice, and formal coach education

Building the foundations of professional expertise: creating a dialectic between work and formal learning

Recent critiques of management and teacher education curricula and teaching pay particular attention to the disconnection between the de-contextualised, formal knowledge and analytical techniques conveyed in university programs and the messy, ill-structured nature of practice. At the same time research into professional expertise suggests that its development requires bringing together different forms of knowledge and the integration of formal and non-formal learning with the development of cognitive flexibility. Such complex learning outcomes are unlikely to be achieved through a 'knowledge transmission' approach to curriculum design. In this article we argue that in many ways current higher education practices create barriers to developing ways of knowing which can underpin the formation of expertise. Using examples from two practice-focused distance learning courses, we explore the role of distance learning in enabling a dialogue between academic and workplace learning and the use of 'practice dialogues' among course participants to enable integration of learning experiences. Finally, we argue that we need to find ways in higher education of enabling students to engage in relevant communities of expertise, rather than drawing them principally into a community of academic discourse which is not well aligned with practice

The Effects of a Flexible Benefits Expert System on Employee Decisions and Satisfaction

Anecdotal reports and recent reviews assert that expert systems are potentially useful decision aids in human resource management. This study examines the effects of an expert system designed to aid employees when they make their choices in a flexible bellcfit program. A four group quasi-field experimental design is used to examine the relative effects of the expert system compared to a conventional spreadsheet decision aid. Eighty employees at an NCR-AT&T facility were randomly selected and assigned to the groups. Employees using the expert system expressed greater benefits satisfaction compared to those using the spreadsheet aid. The spreadsheet did not have any effect on employees\u27 decisions. When the benefit choices recommended by the expert system differed from the employees\u27 current choices, employees are more likely to change their choices. Consequently, the expert system is likely to affect employees\u27 decisions. Implications are discussed and future research needs are suggested

Reading the Readers: Modelling Complex Humanities Processes to Build Cognitive Systems

The ink and stylus tablets discovered at the Roman Fort of Vindolanda are a unique resource for scholars of ancient history. However, the stylus tablets have proved particularly difficult to read. This paper describes the initial stages in the development of a computer system designed to aid historians in the reading of the stylus tablets. A detailed investigation was undertaken, using Knowledge Elicitation techniques borrowed from Artificial IntelliJOURce, Cognitive Psychology, and Computational Linguistics, to elicit the processes experts use whilst reading an ancient text. The resulting model was used as the basis of a computer architecture to construct a system which takes in images of the tablets and outputs plausible interpretations of the documents. It is demonstrated that using Knowledge Elicitation techniques can further the understanding of complex processes in the humanities, and that these techniques can provide an underlying structure for the basis of a computer system that replicates that process. As such it provides significant insight into how experts work in the humanities, whilst providing the means to develop tools to assist them in their complex task

Towards an alternative to Benner’s theory of expert intuition in nursing: A discussion paper

Several authors have highlighted the role of intuition in expertise. In particular, a large amount of data has been collected about intuition in expert nursing, and intuition plays an important role in the influential theory of nursing expertise developed by Benner (1984). We discuss this theory, and highlight both data that support it and data that challenge it. Based on this assessment, we propose a new theory of nursing expertise and intuition, which emphasizes how perception and conscious problem solving are intimately related. In the discussion, we propose that this theory opens new avenues of enquiry for research into nursing expertise