Improving AI systems' dependability by utilizing historical knowledge

A Turing Test is a promising way to validate AI systems which usually have no way to proof correctness. However, human experts (validators) are often too busy to participate in it and sometimes have different opinions per person as well as per validation session. To cope with these and increase the validation dependability, a Validation Knowledge Base (VKB) in Turing Test - like validation is proposed. The VKB is constructed and maintained across various validation sessions. Primary benefits are (1) decreasing validators' workload, (2) refining the methodology itself, e.g. selecting dependable validators using V KB, and (3) increasing AI systems' dependabilities through dependable validation, e.g. support to identify optimal solutions. Finally, Validation Experts Software Agents (VESA) are introduced to further break limitations of human validator's dependability. Each VESA is a software agent corresponding to a particular human validator. This suggests the ability to systematically "construct" human-like validators by keeping personal validation knowledge per corresponding validator. This will bring a new dimension towards dependable AI systems

Compiling experience into knowledge

Typical application fields of Knowledge Based Systems are usually characterized by having human expertise as the only one source to specify their desired behavior. Therefore, their design, evaluation and refinement has to make effective use of this valuable source. After an introduction to the concept of collecting validation experience in a Validation Knowledge Base (VKB), the paper introduces an estimation of the significance of the cases collected in the VKB. A high significance signalizes that a VKB should not longer serve as a case-based source of external (outside the Knowledge Base) knowledge, but compiled into the Knowledge Base instead. Based on this significance estimation, a technology to compile well selected cases into the Knowledge Base of the system under evaluation is presented