An hierarchical approach to performance evaluation of expert systems

The number and size of expert systems is growing rapidly. Formal evaluation of these systems - which is not performed for many systems - increases the acceptability by the user community and hence their success. Hierarchical evaluation that had been conducted for computer systems is applied for expert system performance evaluation. Expert systems are also evaluated by treating them as software systems (or programs). This paper reports many of the basic concepts and ideas in the Performance Evaluation of Expert Systems Study being conducted at the University of Southwestern Louisiana

The Effects of a Flexible Benefits Expert System on Employee Decisions and Satisfaction

Anecdotal reports and recent reviews assert that expert systems are potentially useful decision aids in human resource management. This study examines the effects of an expert system designed to aid employees when they make their choices in a flexible bellcfit program. A four group quasi-field experimental design is used to examine the relative effects of the expert system compared to a conventional spreadsheet decision aid. Eighty employees at an NCR-AT&T facility were randomly selected and assigned to the groups. Employees using the expert system expressed greater benefits satisfaction compared to those using the spreadsheet aid. The spreadsheet did not have any effect on employees\u27 decisions. When the benefit choices recommended by the expert system differed from the employees\u27 current choices, employees are more likely to change their choices. Consequently, the expert system is likely to affect employees\u27 decisions. Implications are discussed and future research needs are suggested

Innovation, skills development and labour: a European perspective

LSE Enterprise was approached by Microsoft, a software developer engaged in lobbying the European Commission and Parliament, to appoint an expert team of academic consultants to research and write a series of four reports on various aspects of innovation. Working with our nominated academic director, LSE Enterprise put together a strong research team, who continue to work closely with the client to deliver each of the reports to a standard suitable for consumption by top EU policy-makers, the client's industry sector, the press and the public. The reports are designed and printed professionally to ensure maximum quality and consistency across the series

The roles of reputation and transparency on the behavior of biased experts

We analyze situations in which an expert is biased toward some decision but cares also about his reputation in the market for experts. The information the expert reveals decreases as his bias moves toward stronger preference for the status quo. We show that it is optimal to publicly disclose both the expert's contribution and his identity. Surprisingly, revealing the intensity of the expert's bias doesn't always improve the information he reveals in equilibrium. The presence of a second expert raises the first expert's incentives to report truthfully when reports are public, but reduces them when they are secret. In particular, having an option to call another expert may be detrimental in terms of information production if reports are not public. Finally, sequential consultation of experts reduces the information obtained when reports are public, but raises it when they are secret.Experts, Bias, Reputation

Expertise and Bias in Decision Making

In this paper, we develop a model of a decision maker using an expert to obtain information. The expert is biased toward some favoured decision but cares also about its reputation on the market for experts. We then analyse the corresponding decision game depending on the nature of the informational linkage with the market. In the case where the expert is biased in favour of the status quo, the final decision is always biased in the same direction. Moreover, it is better to rely on experts biased against the status quo. We also show that it is optimal to publically disclose the expert report. Finally, we prove that the intuitive results that hiring an honest inside expert raises the outside expert's incentives to report truthfully holds when reports are public but not when they are secret.Experts, Bias, Reputation, Merger Control

Implementation of a data management software system for SSME test history data

The implementation of a software system for managing Space Shuttle Main Engine (SSME) test/flight historical data is presented. The software system uses the database management system RIM7 for primary data storage and routine data management, but includes several FORTRAN programs, described here, which provide customized access to the RIM7 database. The consolidation, modification, and transfer of data from the database THIST, to the RIM7 database THISRM is discussed. The RIM7 utility modules for generating some standard reports from THISRM and performing some routine updating and maintenance are briefly described. The FORTRAN accessing programs described include programs for initial loading of large data sets into the database, capturing data from files for database inclusion, and producing specialized statistical reports which cannot be provided by the RIM7 report generator utility. An expert system tutorial, constructed using the expert system shell product INSIGHT2, is described. Finally, a potential expert system, which would analyze data in the database, is outlined. This system could use INSIGHT2 as well and would take advantage of RIM7's compatibility with the microcomputer database system RBase 5000

Cooperation between expert knowledge and data mining discovered knowledge: Lessons learned

Expert systems are built from knowledge traditionally elicited from the human expert. It is precisely knowledge elicitation from the expert that is the bottleneck in expert system construction. On the other hand, a data mining system, which automatically extracts knowledge, needs expert guidance on the successive decisions to be made in each of the system phases. In this context, expert knowledge and data mining discovered knowledge can cooperate, maximizing their individual capabilities: data mining discovered knowledge can be used as a complementary source of knowledge for the expert system, whereas expert knowledge can be used to guide the data mining process. This article summarizes different examples of systems where there is cooperation between expert knowledge and data mining discovered knowledge and reports our experience of such cooperation gathered from a medical diagnosis project called Intelligent Interpretation of Isokinetics Data, which we developed. From that experience, a series of lessons were learned throughout project development. Some of these lessons are generally applicable and others pertain exclusively to certain project types

Domain-based perceptions of risk:a case study of lay and technical community attitudes towards managed aquifer recharge

Despite growing water scarcity, communities in many parts of the developed world often reject technically and economically sound options for water augmentation. This paper reports findings from a study investigating risk perceptions associated with a proposed Managed Aquifer Recharge scheme in Australia. Q-Methodology was used to compare decision-making frameworks of lay community and „technical expert‟ participants. Technical expert participants were also asked to approximate the decision-making framework of a „typical‟ community member. The emerging contrasts between lay community frameworks and those approximated by technical experts suggest that there are prevailing yet errant assumptions about lay community attitudes towards new technologies. The findings challenge the characterisation of the lay community and technical experts as being in entrenched opposition with one another