Assessment of Intelligence Complexity in Embedded Intelligent Real Time Systems

Intelligent systems and their applications are proliferating. Embedded Intelligent Real-Time Systems (EIRTS) are one type of intelligent system. Defining and measuring the complexity of this kind of system may help with better design, development, maintenance, and performance of EIRTS. In this paper, we propose a set of evaluation criteria to measure the complexity of Embedded Intelligent Real-Time Systems (EIRTS). We show an operationalization of the criteria with a sample EIRTS

The Missing Link in the Integration of Knowledge Management Practices and Technological Solutions

The development of knowledge management tools shows a difference between the information system practice and the non-technological solutions: most of the time, the direction of the efforts is not presenting the same way, and not integrating each other. It is the same in the real life practice: there is a problem that the technological tools are not able to give an efficient support to the practice of the company. In this paper an integrated framework is developed to assign the right technological solutions for organisational demands

Improving AI systems' dependability by utilizing historical knowledge

A Turing Test is a promising way to validate AI systems which usually have no way to proof correctness. However, human experts (validators) are often too busy to participate in it and sometimes have different opinions per person as well as per validation session. To cope with these and increase the validation dependability, a Validation Knowledge Base (VKB) in Turing Test - like validation is proposed. The VKB is constructed and maintained across various validation sessions. Primary benefits are (1) decreasing validators' workload, (2) refining the methodology itself, e.g. selecting dependable validators using V KB, and (3) increasing AI systems' dependabilities through dependable validation, e.g. support to identify optimal solutions. Finally, Validation Experts Software Agents (VESA) are introduced to further break limitations of human validator's dependability. Each VESA is a software agent corresponding to a particular human validator. This suggests the ability to systematically "construct" human-like validators by keeping personal validation knowledge per corresponding validator. This will bring a new dimension towards dependable AI systems

Avaliação de desempenho dos sistemas periciais

Mestrado em Gestão de Sistemas de InformaçãoAs divergências existentes entre os autores que consideram que os Sistemas de Informação proporcionam benefícios consideráveis às organizações e aqueles que consideram que estes sistemas não proporcionam benefícios significativos, levou-nos a investigar a relação entre a adopção de Sistemas de Informação e o desempenho da organização relativamente a um tipo específico de sistemas, os Sistemas Periciais. Assim, com base na literatura existente é proposta uma Metodologia de Avaliação de Desempenho de Sistemas Periciais, bem como as métricas que lhe estão associadas. A metodologia e as métricas propostas são aplicadas a um caso de estudo onde se demonstra que um Sistema Pericial de baixo custo pode, para o caso de estudo em questão, proporcionar benefícios significativos.The existing divergences between authors who consider that Information Systems provide considerable benefits to the organizations and those who consider that these systems failed to provide significant benefits, took us to investigate the relationship between the adoption of Information Systems and organizations performance regarding a specific type of systems, the Expert Systems. Thus, based on existing literature is proposed a Methodology for Expert Systems Performance Evaluation, as well as the metrics that are associated. The methodology and metrics proposed are applied to a case study where is evidenced that a low cost Expert System can, for the case study in question, provide significant benefits

Applications Of Rule-Base Coverage Measures To Expert System Evaluation

A rule-base coverage analysis method has been developed which provides an assessment of both the rule-base under review and the test set that has been used for evaluation. Lack of coverage can result from either incompleteness of the test data or errors in the rule-base. A series of heuristics have been developed which use coverage information and meta-knowledge about the larger population to suggest additional test cases from the population, in the event that the initial test set is incomplete. This forms the basis of an incremental approach which allows us to both increase completeness of the test suite and improve coverage of the rule-base. Rule-based system testing usually faces the difficult dual problems of incompleteness and errors in both the rule-base and the test data. Performance of a system on a limited suite of test data is never sufficient to predict performance on a larger set of data in routine use without additional assumptions. An important one of these is the assumption of representative coverage of the population for which the system is intended. The heuristic approach to test data selection is demonstrated using information generated by TRUBAC, a tool which implements the coverage analysis methods. We have applied these techniques to analyze a number of prototype systems for diagnosis of rheumatological diseases. In addition, we demonstrate the use of coverage information to identify class dependencies and guide rule-base pruning. We also introduce a complexity metric for rule-bases. Finally, we discuss extensions of the coverage measures for rule-based systems with dynamic computation of certainty factors.Technical report DCS-TR-34

Applications of Rule-Base Coverage Measures to Expert System Evaluation

Often a rule-based system is tested by checking its performance on a number of test cases with known solutions, modifying the system until it gives the correct results for all or a sufficiently high proportion of the test cases. This method cannot guarantee that the rule-base has been adequately or completely covered during the testing process. We introduce an approach to testing of rule-based systems which uses coverage measures to guide and evaluate the testing process. In addition, the coverage measures can be used to assist rule-base pruning and identification of class dependencies, and serve as the foundation for a set of test data selection heuristics. We also introduce a complexity metric for rule-bases. 1 Introduction Evaluation of a knowledge-based system is a multi-faceted problem, with numerous approaches and techniques. The results generated by the system must be evaluated, along with its features, the usability of the system, how easily it can be enhanced, and whether or ..

Tool Support for Finding and Preventing Faults in Rule Bases

This thesis analyzes challenges for the correct creation of rule bases. Based on experiences and data from three rule base development projects, dedicated experiments and a survey of developers, ten main problem areas are identified. Four approaches in the area of Testing, Debugging, Anomaly Detection and Visualization are proposed and evaluated as remedies for these problem areas

Context-driven methodologies for context-aware and adaptive systems

Applications which are both context-aware and adapting, enhance users’ experience by anticipating their need in relation with their environment and adapt their behavior according to environmental changes. Being by definition both context-aware and adaptive these applications suffer both from faults related to their context-awareness and to their adaptive nature plus from a novel variety of faults originated by the combination of the two. This research work analyzes, classifies, detects, and reports faults belonging to this novel class aiming to improve the robustness of these Context-Aware Adaptive Applications (CAAAs). To better understand the peculiar dynamics driving the CAAAs adaptation mechanism a general high-level architectural model has been designed. This architectural model clearly depicts the stream of information coming from sensors and being computed all the way to the adaptation mechanism. The model identifies a stack of common components representing increasing abstractions of the context and their general interconnections. Known faults involving context data can be re-examined according to this architecture and can be classified in terms of the component in which they are happening and in terms of their abstraction from the environment. Resulting from this classification is a CAAA-oriented fault taxonomy. Our architectural model also underlines that there is a common evolutionary path for CAAAs and shows the importance of the adaptation logic. Indeed most of the adaptation failures are caused by invalid interpretations of the context by the adaptation logic. To prevent such faults we defined a model, the Adaptation Finite-State Machine (A-FSM), describing how the application adapts in response to changes in the context. The A-FSM model is a powerful instrument which allows developers to focus in those context-aware and adaptive aspects in which faults reside. In this model we have identified a set of patterns of faults representing the most common faults in this application domain. Such faults are represented as violation of given properties in the A-FSM. We have created four techniques to detect such faults. Our proposed algorithms are based on three different technologies: enumerative, symbolic and goal planning. Such techniques compensate each other. We have evaluated them by comparing them to each other using both crafted models and models extracted from existing commercial and free applications. In the evaluation we observe the validity, the readability of the reported faults, the scalability and their behavior in limited memory environments. We conclude this Thesis by suggesting possible extensions