Validating Expert Systems: A Demonstration Using Personal Choice Expert, a Flexible Employee Benefit System

A method for validating expert systems, based on validation approaches from psychology and Turing\u27s “imitation game,” is demonstrated using a flexible employee benefits expert system. Psychometric validation has three aspects: the extent to which the system and expert decisions agree (criterionrelated validity), the inputs and processes used by experts compared to the system (content validity), and differences between expert and novice decisions (construct validity). If these criteria are satisfied, then the system is indistinguishable from experts for its domain and satisfies the Turing Test. Personal Choice Expert (PCE) was designed to help employees of a Fortune 500 firm choose benefits in their flexible benefits system. Its recommendations do not significantly differ from those given by independent experts. Hence, if the system-independent expert agreement (criterion-related validity) were the only standard, PCE could be considered valid. However, construct analysis suggests that re-engineering may be required. High intra-expert agreement exists only for some benefit recommendations (e.g., dental care and long-term disability) and not for others (e.g., short-term disability, accidental death and dismemberment, and life insurance). Insights offered by these methods are illustrated and examined

Validation of Expert Systems: Personal Choice Expert -- A Flexible Employee Benefit System

A method for validating expert systems, based on psychological validation literature and Turing\u27s imitation game, is applied to a flexible benefits expert system. Expert system validation entails determining if a difference exists between expert and novice decisions (construct validity), if the system uses the same inputs and processes to make its decisions as experts (content validity), and if the system produces the same results as experts (criterionrelated validity). If these criteria are satisfied, then the system is indistinguishable from experts for its domain and satisfies Turing\u27s imitation game. The methods developed in this paper are applied to a human resource expert system, Personal Choice Expert (PCE), designed to help employees choose a benefits package in a flexible benefits system. Expert and novice recommendations are compared to those generated by PCE. PCE\u27s recommendations do not significantly differ from those given by experts. High inter-expert agreement exists for some benefit recommendations (e.g. Dental Care and Long-Term Disability) but not for others (e.g. Short-Term Disability and Life Insurance). Insights offered by this method are illustrated and examined

Validation of Expert System Performance

Most definitions of an expert system include some reference to the ability of the system to perform at a level close to human expert performance. Yet the validation of expert systems, that is, the testing of systems so as to ascertain that they achieve an acceptable level of performance, has (with a few exceptions) been ad-hoc, informal, and in some cases of dubious value. This paper attempts to establish validation as an important concern in expert systems research and development. The problems in validating an expert system are discussed, and a number of methods for validating expert systems, both qualitative and quantitative, are presented

Discussant\u27s response to Interim report on the development of an expert system for the auditor\u27s loan loss evaluation

https://egrove.olemiss.edu/dl_proceedings/1171/thumbnail.jp

Leading without bleeding: an information technology case study at union pacific railroad

In the railroad industry, the ability to assess damages to rail units in an accurate and timely manner is critical to the success and profits of a company. Accurate damage assessment of rail units also plays a key role in dispute resolution and negotiation with key vendors and suppliers (my.uprr.com/pub/dam-prev). This paper describes and presents information about Union Pacific Railroads (UPRR) and Science Applications International Corporations (SAIC) highly successful efforts in fully automating the data collection, inspection, assessment and reporting of damage claims to rail equipment. UPRR and SAIC used an innovative and highly creative approach to develop and implement the Automated Gate System (AGS) by integrating a portfolio of leading edge high resolution imaging and optical character recognition technologies. AGS is a unique and revolutionary system in the transportation industry and has yielded significant strategic and long-term benefits to the company. The reengineering efforts that preceded the development of the system have helped the company to sustain its position as a leader in the railroad industry

A fuzzy expert system for small business loan processing

In recent years, managerial applications of artificial intelligence, especially in the area of financial services, has received considerable attention. In this paper, a fiizzy logic expert system is developed for approvals of small business loans. Previous studies have used non-fuzzv expert systems. But the fuzzification of the variables used in a business loan approval decisionmaking promises more efficient results. Furthermore, another distinct feature of this paper is its focus on small business loans. CubiCalc fuzzy expert system shell is used to develop the expert system. Knowledge acquisition is made using the resources and expertise of a Small Business Development Center. Tests to establish generalizability and validity of the results are conducted

An Expert System for Local Flood Response Coordination and Training

Flood response is an essential component of flood management to rescue people, reduce property loss, and limit the impact to the environment. Effective flood response depends on a sound coordination structure with unified responsibilities, smooth communications, and scalable response plans. An efficient coordination system, including command and management structures, is built on a thorough understanding of the responsibilities and actions of each role for delivering the response core capabilities. Collecting, sharing, using, and handling the knowledge require great efforts in knowledge management. To further enhance such efforts, an expert system for local flood response coordination and training (LFRS) was developed and introduced in this paper. LFRS can help emergency managers construct scalable, flexible, and adaptable coordination structures and support educating flood response entities, such as individuals, communities, nongovernmental organizations, private sector entities, and local governments. The output of the prototype expert system contains two CSV formatted reports as well as prompt screens. The operational structure report hierarchically depicts the crisscross linkages among all responders, their primary functions, and contact information. Another report summarizes the responsibilities and actions of a certain role of flood responders from commanders to individuals

Introduction to the Special Issue on Human Resource Information Systems and Human Computer Interaction

In this special issue, we focus on the role that human-computer interaction (HCI) can play in the development and successful use of human resource information systems (HRIS) in organizations. There is no doubt that information systems have transformed the practice of human resources. From online/e-recruiting to e-learning and the growing interest in data analytics, the practice of human resources has become technology centric. Given the overlap of human resource practice and information systems, both fields need to work together to develop models and theories that advance the practice of HRIS in organizations. Therefore, this special issue a) briefly reviews the history of the HRIS field, b) advances theory and research that stands at the intersection of HRIS and HCI, and c) suggest new directions for research at the intersection of HRIS and HCI

INTEGRATION OF INFORMATION SYSTEMS TECHNOLOGIES TO SUPPORT CONSULTATION IN AN INFORMATION CENTER

This paper presents an approach for integrating different types of information systems technologies to support the functions of an Information Center (IC). A knowledge based system, Information Center Expert/Help Service (ICE/H), has been developed to provide support for the help services of an IC. A general process model to represent the consultation process in an IC is described. Based on this model, an architecture to support the consultation process has been developed. The architecture depicts the use of a knowledge management system, a data management system and a communication (E-mail) system to emulate the consultation process. The ICE/H system has been implemented using this architecture to support an IC with 5000 users