Humanoid Robot NAO : developing behaviours for soccer humanoid robots

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

Information theoretic stochastic search

The MAP-i Doctoral Programme in Informatics, of the Universities of Minho, Aveiro and PortoOptimization is the research field that studies the design of algorithms for finding the best solutions to problems we may throw at them. While the whole domain is practically important, the present thesis will focus on the subfield of continuous black-box optimization, presenting a collection of novel, state-of-the-art algorithms for solving problems in that class. In this thesis, we introduce two novel general-purpose stochastic search algorithms for black box optimisation. Stochastic search algorithms aim at repeating the type of mutations that led to fittest search points in a population. We can model those mutations by a stochastic distribution. Typically the stochastic distribution is modelled as a multivariate Gaussian distribution. The key idea is to iteratively change the parameters of the distribution towards higher expected fitness. However we leverage information theoretic trust regions and limit the change of the new distribution. We show how plain maximisation of the fitness expectation without bounding the change of the distribution is destined to fail because of overfitting and the results in premature convergence. Being derived from first principles, the proposed methods can be elegantly extended to contextual learning setting which allows for learning context dependent stochastic distributions that generates optimal individuals for a given context, i.e, instead of learning one task at a time, we can learn multiple related tasks at once. However, the search distribution typically uses a parametric model using some hand-defined context features. Finding good context features is a challenging task, and hence, non-parametric methods are often preferred over their parametric counter-parts. Therefore, we further propose a non-parametric contextual stochastic search algorithm that can learn a non-parametric search distribution for multiple tasks simultaneously.Otimização é área de investigação que estuda o projeto de algoritmos para encontrar as melhores soluções, tendo em conta um conjunto de critérios, para problemas complexos. Embora todo o domínio de otimização tenha grande importância, este trabalho está focado no subcampo da otimização contínua de caixa preta, apresentando uma coleção de novos algoritmos novos de última geração para resolver problemas nessa classe. Nesta tese, apresentamos dois novos algoritmos de pesquisa estocástica de propósito geral para otimização de caixa preta. Os algoritmos de pesquisa estocástica visam repetir o tipo de mutações que levaram aos melhores pontos de pesquisa numa população. Podemos modelar essas mutações por meio de uma distribuição estocástica e, tipicamente, a distribuição estocástica é modelada como uma distribuição Gaussiana multivariada. A ideia chave é mudar iterativamente os parâmetros da distribuição incrementando a avaliação. No entanto, alavancamos as regiões de confiança teóricas de informação e limitamos a mudança de distribuição. Deste modo, demonstra-se como a maximização simples da expectativa de “fitness”, sem limites da mudança da distribuição, está destinada a falhar devido ao “overfitness” e à convergência prematura resultantes. Sendo derivado dos primeiros princípios, as abordagens propostas podem ser ampliadas, de forma elegante, para a configuração de aprendizagem contextual que permite a aprendizagem de distribuições estocásticas dependentes do contexto que geram os indivíduos ideais para um determinado contexto. No entanto, a distribuição de pesquisa geralmente usa um modelo paramétrico linear em algumas das características contextuais definidas manualmente. Encontrar uma contextos bem definidos é uma tarefa desafiadora e, portanto, os métodos não paramétricos são frequentemente preferidos em relação às seus semelhantes paramétricos. Portanto, propomos um algoritmo não paramétrico de pesquisa estocástica contextual que possa aprender uma distribuição de pesquisa não-paramétrica para várias tarefas simultaneamente.FCT - Fundação para a Ciência e a Tecnologia. As well as fundings by European Union’s FP7 under EuRoC grant agreement CP-IP 608849 and by LIACC (UID/CEC/00027/2015) and IEETA (UID/CEC/00127/2015)

Longterm Generalized Actions for Smart, Autonomous Robot Agents

Creating intelligent artificial systems, and in particular robots, that improve themselves just like humans do is one of the most ambitious goals in robotics and machine learning. The concept of robot experience exists for some time now, but has up to now not fully found its way into autonomous robots. This thesis is devoted to both, analyzing the underlying requirements for enabling robot learning from experience and actually implementing it on real robot hardware. For effective robot learning from experience I present and discuss three main requirements: (a ) Clearly expressing what a robot should do, on a vague, abstract level I introduce Generalized Plans as a means to express the intention rather than the actual action sequence of a task, removing as much task specific knowledge as possible. (a ) Defining, collecting, and analyzing robot experiences to enable robots to improve I present Episodic Memories as a container for all collected robot experiences for any arbitrary task and create sophisticated action (effect) prediction models from them, allowing robots to make better decisions. (a ) Properly abstracting from reality and dealing with failures in the domain they occurred in I propose failure handling strategies, a failure taxonomy extensible through experience, and discuss the relationship between symbolic/discrete and subsymbolic/continuous systems in terms of robot plans interacting with real world sensors and actuators. I concentrate on the domain of human-scale robot activities, specifically on doing household chores. Tasks in this domain offer many repeating patterns and are ideal candidates for abstracting, encapsulating, and modularizing robot plans into a more general form. This way, very similar plan structures are transformed into parameters that change the behavior of the robot while performing the task, making the plans more flexible. While performing tasks, robots encounter the same or similar situations over and over again. Albeit humans are able to benefit from this and improve at what they do, robots in general lack this ability. This thesis presents techniques for collecting and making robot experiences accessible to robots and outside observers alike, answering high level questions such as What are good spots to stand at for grasping objects from the fridge? or Which objects are especially difficult to grasp with two hands while they are in the oven? . By structuring and tapping into a robot's memory, it can make more informed decisions that are not based on manually encoded information, but self-improved behavior. To this end, I present several experience-based approaches to improve a robot's autonomous decisions, such as parameter choices, during execution time. Robots that interact with the real world are bound to deal with unexpected events and must properly react to failures of any kind of action. I present an extensible failure model that suits the structure of Generalized Plans and Episodic Memories and make clear how each module should deal with their own failures rather than directly handing them up to a governing cognitive architecture. In addition, I make a distinction between discrete parametrizations of Generalized Plans and continuous low level components, and how to translate between the two

Accessibility of Health Data Representations for Older Adults: Challenges and Opportunities for Design

Health data of consumer off-the-shelf wearable devices is often conveyed to users through visual data representations and analyses. However, this is not always accessible to people with disabilities or older people due to low vision, cognitive impairments or literacy issues. Due to trade-offs between aesthetics predominance or information overload, real-time user feedback may not be conveyed easily from sensor devices through visual cues like graphs and texts. These difficulties may hinder critical data understanding. Additional auditory and tactile feedback can also provide immediate and accessible cues from these wearable devices, but it is necessary to understand existing data representation limitations initially. To avoid higher cognitive and visual overload, auditory and haptic cues can be designed to complement, replace or reinforce visual cues. In this paper, we outline the challenges in existing data representation and the necessary evidence to enhance the accessibility of health information from personal sensing devices used to monitor health parameters such as blood pressure, sleep, activity, heart rate and more. By creating innovative and inclusive user feedback, users will likely want to engage and interact with new devices and their own data