Prediction error-driven memory consolidation for continual learning. On the case of adaptive greenhouse models

This work presents an adaptive architecture that performs online learning and faces catastrophic forgetting issues by means of episodic memories and prediction-error driven memory consolidation. In line with evidences from the cognitive science and neuroscience, memories are retained depending on their congruency with the prior knowledge stored in the system. This is estimated in terms of prediction error resulting from a generative model. Moreover, this AI system is transferred onto an innovative application in the horticulture industry: the learning and transfer of greenhouse models. This work presents a model trained on data recorded from research facilities and transferred to a production greenhouse.Comment: Revised version. Paper under review, submitted to Springer German Journal on Artificial Intelligence (K\"unstliche Intelligenz), Special Issue on Developmental Robotic

Sharing emotions and space - empathy as a basis for cooperative spatial interaction

Boukricha H, Nguyen N, Wachsmuth I. Sharing emotions and space - empathy as a basis for cooperative spatial interaction. In: Kopp S, Marsella S, Thorisson K, Vilhjalmsson HH, eds. Proceedings of the 11th International Conference on Intelligent Virtual Agents (IVA 2011). LNAI. Vol 6895. Berlin, Heidelberg: Springer; 2011: 350-362.Empathy is believed to play a major role as a basis for humans’ cooperative behavior. Recent research shows that humans empathize with each other to different degrees depending on several modulation factors including, among others, their social relationships, their mood, and the situational context. In human spatial interaction, partners share and sustain a space that is equally and exclusively reachable to them, the so-called interaction space. In a cooperative interaction scenario of relocating objects in interaction space, we introduce an approach for triggering and modulating a virtual humans cooperative spatial behavior by its degree of empathy with its interaction partner. That is, spatial distances like object distances as well as distances of arm and body movements while relocating objects in interaction space are modulated by the virtual human’s degree of empathy. In this scenario, the virtual human’s empathic emotion is generated as a hypothesis about the partner’s emotional state as related to the physical effort needed to perform a goal directed spatial behavior

Towards Adaptable and Interactive Image Captioning with Data Augmentation and Episodic Memory

Interactive machine learning (IML) is a beneficial learning paradigm in cases of limited data availability, as human feedback is incrementally integrated into the training process. In this paper, we present an IML pipeline for image captioning which allows us to incrementally adapt a pre-trained image captioning model to a new data distribution based on user input. In order to incorporate user input into the model, we explore the use of a combination of simple data augmentation methods to obtain larger data batches for each newly annotated data instance and implement continual learning methods to prevent catastrophic forgetting from repeated updates. For our experiments, we split a domain-specific image captioning dataset, namely VizWiz, into non-overlapping parts to simulate an incremental input flow for continually adapting the model to new data. We find that, while data augmentation worsens results, even when relatively small amounts of data are available, episodic memory is an effective strategy to retain knowledge from previously seen clusters

Introduction to the Use of Robotic Tools for Search and Rescue

Modern search and rescue workers are equipped with a powerful toolkit to address natural and man-made disasters. This introductory chapter explains how a new tool can be added to this toolkit: robots. The use of robotic assets in search and rescue operations is explained and an overview is given of the worldwide efforts to incorporate robotic tools in search and rescue operations. Furthermore, the European Union ICARUS project on this subject is introduced. The ICARUS project proposes to equip first responders with a comprehensive and integrated set of unmanned search and rescue tools, to increase the situational awareness of human crisis managers, such that more work can be done in a shorter amount of time. The ICARUS tools consist of assistive unmanned air, ground, and sea vehicles, equipped with victim-detection sensors. The unmanned vehicles collaborate as a coordinated team, communicating via ad hoc cognitive radio networking. To ensure optimal human-robot collaboration, these tools are seamlessly integrated into the command and control equipment of the human crisis managers and a set of training and support tools is provided to them to learn to use the ICARUS system

Chapter Introduction to the Use of Robotic Tools for Search and Rescue

Modern search and rescue workers are equipped with a powerful toolkit to address natural and man-made disasters. This introductory chapter explains how a new tool can be added to this toolkit: robots. The use of robotic assets in search and rescue operations is explained and an overview is given of the worldwide efforts to incorporate robotic tools in search and rescue operations. Furthermore, the European Union ICARUS project on this subject is introduced. The ICARUS project proposes to equip first responders with a comprehensive and integrated set of unmanned search and rescue tools, to increase the situational awareness of human crisis managers, such that more work can be done in a shorter amount of time. The ICARUS tools consist of assistive unmanned air, ground, and sea vehicles, equipped with victim-detection sensors. The unmanned vehicles collaborate as a coordinated team, communicating via ad hoc cognitive radio networking. To ensure optimal human-robot collaboration, these tools are seamlessly integrated into the command and control equipment of the human crisis managers and a set of training and support tools is provided to them to learn to use the ICARUS system

IST Austria Thesis

Traditionally machine learning has been focusing on the problem of solving a single task in isolation. While being quite well understood, this approach disregards an important aspect of human learning: when facing a new problem, humans are able to exploit knowledge acquired from previously learned tasks. Intuitively, access to several problems simultaneously or sequentially could also be advantageous for a machine learning system, especially if these tasks are closely related. Indeed, results of many empirical studies have provided justification for this intuition. However, theoretical justifications of this idea are rather limited. The focus of this thesis is to expand the understanding of potential benefits of information transfer between several related learning problems. We provide theoretical analysis for three scenarios of multi-task learning - multiple kernel learning, sequential learning and active task selection. We also provide a PAC-Bayesian perspective on lifelong learning and investigate how the task generation process influences the generalization guarantees in this scenario. In addition, we show how some of the obtained theoretical results can be used to derive principled multi-task and lifelong learning algorithms and illustrate their performance on various synthetic and real-world datasets

On-device mobile speech recognition

Despite many years of research, Speech Recognition remains an active area of research in Artificial Intelligence. Currently, the most common commercial application of this technology on mobile devices uses a wireless client – server approach to meet the computational and memory demands of the speech recognition process. Unfortunately, such an approach is unlikely to remain viable when fully applied over the approximately 7.22 Billion mobile phones currently in circulation. In this thesis we present an On – Device Speech recognition system. Such a system has the potential to completely eliminate the wireless client-server bottleneck. For the Voice Activity Detection part of this work, this thesis presents two novel algorithms used to detect speech activity within an audio signal. The first algorithm is based on the Log Linear Predictive Cepstral Coefficients Residual signal. These LLPCCRS feature vectors were then classified into voice signal and non-voice signal segments using a modified K-means clustering algorithm. This VAD algorithm is shown to provide a better performance as compared to a conventional energy frame analysis based approach. The second algorithm developed is based on the Linear Predictive Cepstral Coefficients. This algorithm uses the frames within the speech signal with the minimum and maximum standard deviation, as candidates for a linear cross correlation against the rest of the frames within the audio signal. The cross correlated frames are then classified using the same modified K-means clustering algorithm. The resulting output provides a cluster for Speech frames and another cluster for Non–speech frames. This novel application of the linear cross correlation technique to linear predictive cepstral coefficients feature vectors provides a fast computation method for use on the mobile platform; as shown by the results presented in this thesis. The Speech recognition part of this thesis presents two novel Neural Network approaches to mobile Speech recognition. Firstly, a recurrent neural networks architecture is developed to accommodate the output of the VAD stage. Specifically, an Echo State Network (ESN) is used for phoneme level recognition. The drawbacks and advantages of this method are explained further within the thesis. Secondly, a dynamic Multi-Layer Perceptron approach is developed. This builds on the drawbacks of the ESN and provides a dynamic way of handling speech signal length variabilities within its architecture. This novel Dynamic Multi-Layer Perceptron uses both the Linear Predictive Cepstral Coefficients (LPC) and the Mel Frequency Cepstral Coefficients (MFCC) as input features. A speaker dependent approach is presented using the Centre for spoken Language and Understanding (CSLU) database. The results show a very distinct behaviour from conventional speech recognition approaches because the LPC shows performance figures very close to the MFCC. A speaker independent system, using the standard TIMIT dataset, is then implemented on the dynamic MLP for further confirmation of this. In this mode of operation the MFCC outperforms the LPC. Finally, all the results, with emphasis on the computation time of both these novel neural network approaches are compared directly to a conventional hidden Markov model on the CSLU and TIMIT standard datasets

Predição para o uso da inteligência artificial no agronegócio na Caatinga

A ciência e a tecnologia, em diferentes formas, sempre exerceram um papel expressivo na solução de problemas, sendo usadas para o desenvolvimento de estratégias, produtos, métodos e ferramentas. Os avanços em ciência e tecnologia têm se mostrado promissores no intuito de aprimorar setores como o agronegócio. E essa visão tem sido justificada pelo constante avanço de dispositivos tecnológicos projetados para apresentar soluções aos problemas agrícolas. Sendo assim, este estudo tem por objetivo analisar o processo de inovação no contexto da Inteligência Artificial (IA), desde a produção do conhecimento científico até a fase de predição dessa tecnologia no agronegócio na Caatinga. Do ponto de vista dos aspectos metodológicos a pesquisa é classificada como exploratória, uma vez que essa investigação leva em consideração uma área na qual há pouco conhecimento acumulado e sistematizado. Em relação à técnica de pesquisa, é caracterizada como estudo de caso. Os resultados da aplicação dos métodos da IA no agronegócio no contexto geral apresentam diferentes abordagens como o uso de Visão de Máquina por meio de Sistema Agrícola Virtual, SVM e ELM na detecção precoce do patógeno de pragas e doenças; FIS e MLP para a exploração de culturas; propagação reversa para monitoramento dos limites da fazenda; ANN e MFNN para análise de estruturas de irrigação; e Árvore da Decisão e TDNN para a vigilância do rebanho. Com os dispositivos integrados no sistema de produção agrícola os sistemas das fazendas passam a oferecer recomendações e insights mais ricos para a tomada de decisão e melhoria da cadeia de suprimentos agrícola. Em relação ao levantamento das tecnologias atuais no agronegócio na Caatinga, o contexto local apresenta abordagens bem distintas, desde a utilização de técnicas de convivência com o semiárido como os métodos de manejo do solo, aproveitamento da água da chuva e preparo de ração animal. Já a análise do uso das tecnologias, o enfoco está na viabilidade da produção, diversificação e manejo da colheita em polos integrados de grande desenvolvimento tecnológico em polos de cultivo e manejo de culturas irrigadas. A perspectiva da adoção e o desenvolvimento de IA no agronegócio na Caatinga ainda se encontram em fase inicial, com os agentes buscando nas pesquisas, conhecer as oportunidades dessa tecnologia frente aos negócios no setor agrícola. Na Caatinga, os estudos ainda são reduzidos, mas já há exemplos como rastreabilidade de carne, predição da produtividade da palma forrageira, delineamento de zonas de manejo ou mesmo na estimativa da evapotranspiração de referência. Contudo, há etapas que devem ser superadas até a integração da IA como a habilidade de entender e manusear as ferramentas com IA e a integração dos sistemas dentro da cadeia de suprimentos. Já os resultados do levantamento sistemático apresentam ações como modelagem e previsão do fluxo de água; evapotranspiração; variabilidade, avaliação de terra; previsão de época ótima de semeadura e seleção de cultivares. De modo que, os achados apresentam os diferentes usos da IA, com iniciativas de sustentabilidade habilitadas por mudanças no sistema agrícola atual.Science and technology, in different forms, have always played an expressive role in problem solving, being used for the development of strategies, products, methods and tools. Advances in science and technology have shown promise in order to improve sectors such as agribusiness. And this vision has been justified by the constant advancement of technological devices designed to present solutions to agricultural problems. Therefore, this study aims to analyze the innovation process in the context of artificial intelligence, from the production of scientific knowledge to the prediction phase of this technology in agribusiness in the Caatinga. From the point of view of methodological aspects, the research is classified as exploratory, since this investigation takes into account an area in which there is little accumulated and systematized knowledge. Regarding the research technique, it is characterized as a case study. The results of the application of AI methods in agribusiness in the general context present different approaches such as the use of Machine Vision through Virtual Agricultural System, SVM and ELM in the early detection of the pathogen of pests and diseases; FIS and MLP for the exploitation of cultures; reverse propagation for monitoring farm boundaries; ANN and MFNN for analysis of irrigation structures; and Decision Tree and TDNN for herd surveillance. With the devices integrated into the agricultural production system. farm systems now offer richer recommendations and insights for decision making and agricultural supply chain improvement. Regarding the survey of current technologies in agribusiness in the Caatinga, the local context presents very different approaches, from the use of technologies of coexistence with the semi-arid region or social techniques such as methods of soil management, use of rainwater and preparation of feed animal. Even the use of technologies themselves aimed at the viability of production, diversification and management of the harvest in integrated poles of great technological development in poles of cultivation and management of irrigated cultures. The perspective of the adoption and development of AI in agribusiness in the Caatinga is still at an early stage, with agents seeking, in research, to know the opportunities of this technology in relation to business in the agricultural sector. In the Caatinga, studies are still very limited, but there are already examples such as meat traceability, prediction of forage cactus productivity, delineation of management zones or even in the estimation of reference evapotranspiration. However, there are steps that must be overcome until the integration of AI such as the ability to understand and handle the tools with AI and the integration of systems within the supply chain. On the other hand, the results of the systematic survey present actions such as modeling and forecasting the water flow; evapotranspiration; variability, land assessment; prediction of optimal sowing time and selection of cultivars. So, the findings present the different uses of AI, with sustainability initiatives enabled by changes in the current agricultural system

Computergestützte Film- und Videoanalyse

peer reviewe