Sampling with Confidence: Using k-NN Confidence Measures in Active Learning

Active learning is a process through which classifiers can be built from collections of unlabelled examples through the cooperation of a human oracle who can label a small number of examples selected as most informative. Typically the most informative examples are selected through uncertainty sampling based on classification scores. However, previous work has shown that, contrary to expectations, there is not a direct relationship between classification scores and classification confidence. Fortunately, there exists a collection of particularly effective techniques for building measures of classification confidence from the similarity information generated by k-NN classifiers. This paper investigates using these confidence measures in a new active learning sampling selection strategy, and shows how the performance of this strategy is better than one based on uncertainty sampling using classification scores

EGAL: Exploration Guided Active Learning for TCBR

The task of building labelled case bases can be approached using active learning (AL), a process which facilitates the labelling of large collections of examples with minimal manual labelling effort. The main challenge in designing AL systems is the development of a selection strategy to choose the most informative examples to manually label. Typical selection strategies use exploitation techniques which attempt to refine uncertain areas of the decision space based on the output of a classifier. Other approaches tend to balance exploitation with exploration, selecting examples from dense and interesting regions of the domain space. In this paper we present a simple but effective exploration only selection strategy for AL in the textual domain. Our approach is inherently case-based, using only nearest-neighbour-based density and diversity measures. We show how its performance is comparable to the more computationally expensive exploitation-based approaches and that it offers the opportunity to be classifier independent

Retrieval, reuse, revision and retention in case-based reasoning

El original está disponible en www.journals.cambridge.orgCase-based reasoning (CBR) is an approach to problem solving that emphasizes the role of prior experience during future problem solving (i.e., new problems are solved by reusing and if necessary adapting the solutions to similar problems that were solved in the past). It has enjoyed considerable success in a wide variety of problem solving tasks and domains. Following a brief overview of the traditional problem-solving cycle in CBR, we examine the cognitive science foundations of CBR and its relationship to analogical reasoning. We then review a representative selection of CBR research in the past few decades on aspects of retrieval, reuse, revision, and retention.Peer reviewe

Case-based reasoning approach for decision-making in building retrofit: A review

The rapid development of computer science has brought inspirations to building retrofit. Artificial intelligence (AI) provides more possibilities in decision-making for building retrofit, could be regarded as an alternative strategy compared to the abundant research time spent in the early decision-making stage of traditional retrofit approaches. This paper reviews the application of the statistic algorithm and AI approach, including CBR, in building retrofit decision-making, and the essential process of CBR, such as workflow, similarity degree calculation method, weight factors correction manner, and input or output content using building design to provide a synthetic overview of CBR utilisation in the building retrofit realm. Among those different models, Case-Based Reasoning (CBR) is valuable in providing references and avoiding possible failures, which is a promising approach for building retrofit. Yet, current research mainly focused on its utilisation to solve specific issues. There is still a lack of systematically summarised research on Case-Based Reasoning solution. Therefore, this study analyses the methods used for CBR approach in the field of building retrofit decision-making process, aiming to find the characteristics of internal commonness. It concludes that CBR has two significant impact factors: similarity attribute type and similarity calculation manner, which determines the judgement process. The results show that the CBR solution has great application potential in further building retrofit design

Organization based multiagent architecture for distributed environments

[EN]Distributed environments represent a complex field in which applied solutions should be flexible and include significant adaptation capabilities. These environments are related to problems where multiple users and devices may interact, and where simple and local solutions could possibly generate good results, but may not be effective with regards to use and interaction. There are many techniques that can be employed to face this kind of problems, from CORBA to multi-agent systems, passing by web-services and SOA, among others. All those methodologies have their advantages and disadvantages that are properly analyzed in this documents, to finally explain the new architecture presented as a solution for distributed environment problems. The new architecture for solving complex solutions in distributed environments presented here is called OBaMADE: Organization Based Multiagent Architecture for Distributed Environments. It is a multiagent architecture based on the organizations of agents paradigm, where the agents in the architecture are structured into organizations to improve their organizational capabilities. The reasoning power of the architecture is based on the Case-Based Reasoning methology, being implemented in a internal organization that uses agents to create services to solve the external request made by the users. The OBaMADE architecture has been successfully applied to two different case studies where its prediction capabilities have been properly checked. Those case studies have showed optimistic results and, being complex systems, have demonstrated the abstraction and generalizations capabilities of the architecture. Nevertheless OBaMADE is intended to be able to solve much other kind of problems in distributed environments scenarios. It should be applied to other varieties of situations and to other knowledge fields to fully develop its potencial.[ES]Los entornos distribuidos representan un campo de conocimiento complejo en el que las soluciones a aplicar deben ser flexibles y deben contar con gran capacidad de adaptación. Este tipo de entornos está normalmente relacionado con problemas donde varios usuarios y dispositivos entran en juego. Para solucionar dichos problemas, pueden utilizarse sistemas locales que, aunque ofrezcan buenos resultados en términos de calidad de los mismos, no son tan efectivos en cuanto a la interacción y posibilidades de uso. Existen múltiples técnicas que pueden ser empleadas para resolver este tipo de problemas, desde CORBA a sistemas multiagente, pasando por servicios web y SOA, entre otros. Todas estas mitologías tienen sus ventajas e inconvenientes, que se analizan en este documento, para explicar, finalmente, la nueva arquitectura presentada como una solución para los problemas generados en entornos distribuidos. La nueva arquitectura aquí se llama OBaMADE, que es el acrónimo del inglés Organization Based Multiagent Architecture for Distributed Environments (Arquitectura Multiagente Basada en Organizaciones para Entornos Distribuidos). Se trata de una arquitectura multiagente basasa en el paradigma de las organizaciones de agente, donde los agentes que forman parte de la arquitectura se estructuran en organizaciones para mejorar sus capacidades organizativas. La capacidad de razonamiento de la arquitectura está basada en la metodología de razonamiento basado en casos, que se ha implementado en una de las organizaciones internas de la arquitectura por medio de agentes que crean servicios que responden a las solicitudes externas de los usuarios. La arquitectura OBaMADE se ha aplicado de forma exitosa a dos casos de estudio diferentes, en los que se han demostrado sus capacidades predictivas. Aplicando OBaMADE a estos casos de estudio se han obtenido resultados esperanzadores y, al ser sistemas complejos, se han demostrado las capacidades tanto de abstracción como de generalización de la arquitectura presentada. Sin embargo, esta arquitectura está diseñada para poder ser aplicada a más tipo de problemas de entornos distribuidos. Debe ser aplicada a más variadas situaciones y a otros campos de conocimiento para desarrollar completamente el potencial de esta arquitectura

Modelo de desenvolvimento de arquitecturas de sistemas de informação

Tese de doutoramento em Tecnologias e Sistemas de InformaçãoA arquitectura de um Sistema de Informação desempenha um papel importante na função Sistemas de Informação, uma vez que permite manter uma visão global dos seus vários aspectos. Em actividades como a de desenvolvimento da arquitectura de um Sistema de Informação é comum recorrer-se à experiência adquirida. De facto, uma das estratégias utilizadas pelos profissionais em Sistemas de Informação é relembrar situações anteriormente resolvidas para aplicação a novos problemas. O desenvolvimento da arquitectura de um Sistema de Informação é actualmente realizado nas organizações sem recurso à experiência adquirida, para além da que foi acumulada pelas pessoas que participaram em processos similares anteriores. Como tal, cada vez que se desenvolve uma Arquitectura de Sistema de Informação, o processo decorre como se o mesmo se estivesse a realizar pela primeira vez. Este trabalho visa suprir a lacuna referida, isto é, propõe um modelo e uma aplicação informática que permite às organizações tirarem partido de forma sistemática da experiência adquirida. Para o efeito, conjugam-se factores de ordem teórica e prática. Em primeiro lugar, relaciona-se conhecimento existente na bibliografia do domínio Sistemas de Informação para assim se propor um modelo de desenvolvimento de arquitecturas de Sistemas de Informação. Posteriormente, realiza-se o teste e ajuste do anteriormente proposto para se verificar se o modelo se ajusta às necessidades das organizações. O resultado a que chegámos, denominado MODASI (Modelo de Desenvolvimento de Arquitecturas de Sistemas de Informação), é constituído por um modelo que assenta num referencial de desenvolvimento de arquitecturas de Sistemas de Informação e por uma aplicação informática que contempla um módulo que implementa técnicas de Raciocínio Baseado em Casos. Serve de suporte ao MODASI um metamodelo que permite a utilização de várias técnicas e ferramentas. Esta conjugação de características do MODASI permite que este seja um elemento potenciador do desenvolvimento de Arquitecturas de Sistemas de Informação. A fim de validar o MODASI, foi empreendido um vasto grupo de experiências em nove hospitais portugueses, abrangendo aspectos particulares e distintos em cada um deles. Verificou-se a adequação do modelo e a utilidade do recurso, automatizado, à experiência passada.Information Systems architecture plays an important role in Information Systems function, since it allows to maintain a global vision over its various aspects. In activities such as developing Information Systems architecture, it is common to resort to acquired experience. In fact, one of the strategies used by Information Systems professionals is to remember previously resolved situations for application in new problems. Information Systems architecture development is carried currently out in the organizations without recourse to acquired experience besides that which was accumulated by the people who previously participated in similar processes. As such, each time that an Information Systems architecture description is developed, the process happens as if it were being carried out for the first time. The aim of this work is to overcome this gap, that is, it considers a model and a computer application that allows organizations to systematically take advantage of acquired experience. To this end, theoretical and practical considerations converge. Firstly, existing knowledge in Information Systems literature is analyzed so as to propose an Information Systems architecture development model. Later, a test is carried out and adjusted in relation to the previous proposition in order to verify whether the proposed model suits organizations’ needs. The result obtained, called MODASI, consists of a model based on an Information Systems architecture framework and on a computer application which includes a module of Case-Based Reasoning. The MODASI has a meta-model associated to it that allows/enables the use of several techniques and tools. These characteristics of the MODASI make it a fundamental aid in Information Systems architecture development. In order to validate the MODASI a vast set of experiments in nine Portuguese hospitals was undertaken, taking each one’s particular and distinct aspects into account. The adequacy of the model was verified as well as the usefulness of resorting to past experience in an automated fashion.Programa de Desenvolvimento Educativo para Portugal (PRODEP)

Collaborative Case Retention Strategies for CBR Agents

Empirical experiments have shown that storing every case does not automatically improve the accuracy of a CBR system. Therefore, several retain policies have been proposed in order to select which cases to retain. However, all the research done in case retention strategies is done in centralized CBR systems. We focus on multiagent CBR systems, where each agent has a local case base, and where each agent can interact with other agents in the system to solve problems in a collaborative way. We propose several case retention strategies that directly deal with the issue of being in a multiagent CBR system. Those case retention strategies combine ideas from the CBR case retain strategies and from the active learning techniques. Empirical results show that strategies that use collaboration with other agents outperform those strategies where the agents work in isolation. We present experiments in two di#erent scenarios, the first one allowing multiple copies of one case and the second one only allowing one copy of each case. Although it may seem counterintuitive, we show and explain why not allowing multiple copies of each case achieves better results