On the hardness of unlabeled multi-robot motion planning

In unlabeled multi-robot motion planning several interchangeable robots operate in a common workspace. The goal is to move the robots to a set of target positions such that each position will be occupied by some robot. In this paper, we study this problem for the specific case of unit-square robots moving amidst polygonal obstacles and show that it is PSPACE-hard. We also consider three additional variants of this problem and show that they are all PSPACE-hard as well. To the best of our knowledge, this is the first hardness proof for the unlabeled case. Furthermore, our proofs can be used to show that the labeled variant (where each robot is assigned with a specific target position), again, for unit-square robots, is PSPACE-hard as well, which sets another precedence, as previous hardness results require the robots to be of different shapes

Curriculum Learning with a progression function

Whenever we, as humans, need to learn a complex task, our learning is usually organised in a specific order: starting from simple concepts and progressing onto more complex ones as our knowledge increases. Likewise, Reinforcement Learning agents can benefit from structure and guidance in their learning. The field of research that studies how to design the agent's training effectively is called Curriculum Learning, and it aims to increase its performance and learning speed. This thesis introduces a new paradigm for Curriculum Learning based on progression and mapping functions. While progression functions specify the complexity of the environment at any given time, mapping functions generate environments of a specific complexity. This framework does not impose any restriction on the tasks that can be included in the curriculum, and it allows to change the task the agent is training on up to each action. The problem of creating a curriculum tailored to each agent is explored in the context of the framework. This is achieved through adaptive progression functions, which specify the complexity of the environment based on the agent's performance. Furthermore, a method to progress each dimension independently is defined, and the progression functions derived from our framework are evaluated against state-of-the-art Curriculum Learning methods. Finally, a novel variation of the Multi-Armed Bandit problem is defined, where a target value is observed at each round, and the arm with the closest expected value to the target is chosen. Based on this framework, we define an algorithm to automate the generation of a mapping function. The end result of this thesis is a method that is learning algorithm agnostic, is able to translate domain knowledge into an increase in performance (providing similar benefits if such domain knowledge was not available), and can create a fully automated curriculum tailored to each learning agent

Independent - Oct. 15, 2013

https://neiudc.neiu.edu/independent/1469/thumbnail.jp

Regulation and deregulation in industrial countries : some lessons for LDCs

The United States'experience with antitrust and with directive regulation in the rail, trucking, airline, and telephone sectors offers useful lessons for developing countries. The experience highlights the realities both of market failure and of the difficulties of implementing regulation to control it - and reveals that imperfect regulation may be no better than imperfect competition. Antitrust measures to regulate price fixing and to require approval for mergers above some threshold level of industrial concentration are straightforward to implement and have provided some gains in economic welfare. The regulation of price discrimination, restrictive vertical practices, and predatory pricing is administratively more difficult, and the potential gains are less clearly evident. In many situations, import competition can be an efficient alternative. Direct regulation of rail, trucking, airline, and telephone was frequently inefficient, the regulatory apparatus often lost sight of its original objectives, and the regulators were captured by the regulated. For rail and trucking regulation, the regulatory outcome probably was worse than it would have been under laissez-faire.Administrative&Regulatory Law,Economic Theory&Research,National Governance,Knowledge Economy,Environmental Economics&Policies

On Building Generalizable Learning Agents

It has been a long-standing goal in Artificial Intelligence (AI) to build machines that can solve tasks that humans can. Thanks to the recent rapid progress in data-driven methods, which train agents to solve tasks by learning from massive training data, there have been many successes in applying such learning approaches to handle and even solve a number of extremely challenging tasks, including image classification, language generation, robotics control, and several multi-player games. The key factor for all these data-driven successes is that the trained agents can generalize to test scenarios that are unseen during training. This generalization capability is the foundation for building any practical AI system. This thesis studies generalization, the fundamental challenge in AI, and proposes solutions to improve the generalization performances of learning agents in a variety of problems. We start by providing a formal formulation of the generalization problem in the context of reinforcement learning and proposing 4 principles within this formulation to guide the design of training techniques for improved generalization. We validate the effectiveness of our proposed principles by considering 4 different domains, from simple to complex, and developing domain-specific techniques following these principles. Particularly, we begin with the simplest domain, i.e., path-finding on graphs (Part I), and then consider visual navigation in a 3D world (Part II) and competition in complex multi-agent games (Part III), and lastly tackle some natural language processing tasks (Part IV). Empirical evidences demonstrate that the proposed principles can generally lead to much improved generalization performances in a wide range of problems

Engineering evolutionary control for real-world robotic systems

Evolutionary Robotics (ER) is the field of study concerned with the application of evolutionary computation to the design of robotic systems. Two main issues have prevented ER from being applied to real-world tasks, namely scaling to complex tasks and the transfer of control to real-robot systems. Finding solutions to complex tasks is challenging for evolutionary approaches due to the bootstrap problem and deception. When the task goal is too difficult, the evolutionary process will drift in regions of the search space with equally low levels of performance and therefore fail to bootstrap. Furthermore, the search space tends to get rugged (deceptive) as task complexity increases, which can lead to premature convergence. Another prominent issue in ER is the reality gap. Behavioral control is typically evolved in simulation and then only transferred to the real robotic hardware when a good solution has been found. Since simulation is an abstraction of the real world, the accuracy of the robot model and its interactions with the environment is limited. As a result, control evolved in a simulator tends to display a lower performance in reality than in simulation. In this thesis, we present a hierarchical control synthesis approach that enables the use of ER techniques for complex tasks in real robotic hardware by mitigating the bootstrap problem, deception, and the reality gap. We recursively decompose a task into sub-tasks, and synthesize control for each sub-task. The individual behaviors are then composed hierarchically. The possibility of incrementally transferring control as the controller is composed allows transferability issues to be addressed locally in the controller hierarchy. Our approach features hybridity, allowing different control synthesis techniques to be combined. We demonstrate our approach in a series of tasks that go beyond the complexity of tasks where ER has been successfully applied. We further show that hierarchical control can be applied in single-robot systems and in multirobot systems. Given our long-term goal of enabling the application of ER techniques to real-world tasks, we systematically validate our approach in real robotic hardware. For one of the demonstrations in this thesis, we have designed and built a swarm robotic platform, and we show the first successful transfer of evolved and hierarchical control to a swarm of robots outside of controlled laboratory conditions.A Robótica Evolutiva (RE) é a área de investigação que estuda a aplicação de computação evolutiva na conceção de sistemas robóticos. Dois principais desafios têm impedido a aplicação da RE em tarefas do mundo real: a dificuldade em solucionar tarefas complexas e a transferência de controladores evoluídos para sistemas robóticos reais. Encontrar soluções para tarefas complexas é desafiante para as técnicas evolutivas devido ao bootstrap problem e à deception. Quando o objetivo é demasiado difícil, o processo evolutivo tende a permanecer em regiões do espaço de procura com níveis de desempenho igualmente baixos, e consequentemente não consegue inicializar. Por outro lado, o espaço de procura tende a enrugar à medida que a complexidade da tarefa aumenta, o que pode resultar numa convergência prematura. Outro desafio na RE é a reality gap. O controlo robótico é tipicamente evoluído em simulação, e só é transferido para o sistema robótico real quando uma boa solução tiver sido encontrada. Como a simulação é uma abstração da realidade, a precisão do modelo do robô e das suas interações com o ambiente é limitada, podendo resultar em controladores com um menor desempenho no mundo real. Nesta tese, apresentamos uma abordagem de síntese de controlo hierárquica que permite o uso de técnicas de RE em tarefas complexas com hardware robótico real, mitigando o bootstrap problem, a deception e a reality gap. Decompomos recursivamente uma tarefa em sub-tarefas, e sintetizamos controlo para cada subtarefa. Os comportamentos individuais são então compostos hierarquicamente. A possibilidade de transferir o controlo incrementalmente à medida que o controlador é composto permite que problemas de transferibilidade possam ser endereçados localmente na hierarquia do controlador. A nossa abordagem permite o uso de diferentes técnicas de síntese de controlo, resultando em controladores híbridos. Demonstramos a nossa abordagem em várias tarefas que vão para além da complexidade das tarefas onde a RE foi aplicada. Também mostramos que o controlo hierárquico pode ser aplicado em sistemas de um robô ou sistemas multirobô. Dado o nosso objetivo de longo prazo de permitir o uso de técnicas de RE em tarefas no mundo real, concebemos e desenvolvemos uma plataforma de robótica de enxame, e mostramos a primeira transferência de controlo evoluído e hierárquico para um exame de robôs fora de condições controladas de laboratório.This work has been supported by the Portuguese Foundation for Science and Technology (Fundação para a Ciência e Tecnologia) under the grants SFRH/BD/76438/2011, EXPL/EEI-AUT/0329/2013, and by Instituto de Telecomunicações under the grant UID/EEA/50008/2013

Magneto-capillary dynamics of particles at curved liquid interfaces

The ability to manipulate colloidal particles with magnetic fields has profound applications both in industry and academic research ranging from automobile shock absorbers to robotic micro-surgery. Many of these applications use field gradients to generate forces on magnetic objects. Such methods are limited by the complexity of the required fields and by the magnitude of the forces generated. Spatially uniform fields only apply torques, but no forces, on magnetic particles. However, by coupling the particles' orientation and location, even static uniform fields can drive particle motion. We demonstrate this idea using particles adsorbed at curved liquid interfaces. We first review the intersection between active colloidal particles and (passive) particles at the fluid-fluid interface (chapter 1), followed by the introduction of magnetism, magnetic manipulation, and magnetic Janus particle fabrication techniques (chapter 2). In chapter 3, we use magnetic Janus particles with amphiphilic surface chemistry adsorbed at the spherical interface of water drop in decane as a model system to study particle response to a uniform field. Owing to capillary constraints, Janus particles adsorbed at curved interfaces will move in a uniform magnetic field to align their magnetic moment parallel to the applied field. This phenomenon is labeled as the magneto-capillary effect in this thesis. As explained quantitatively by a simple model, the effective magnetic force on the particle induced by static uniform field scales linearly with the curvature of the interface. For particles adsorbed on small droplets such as those found in emulsions, these magneto-capillary forces can far exceed those due to magnetic field gradients in both magnitude and range. The time-varying fields induce more complex particle motions that persist as long as the field is applied (chapter 4). Depending on the angle and frequency of a precessing field, particles orbit the drop poles or zig-zag around the drop equator. Magneto-capillary effects are not limited to Janus particles. Similar behaviors are observed in commercially available carbonyl iron particles. Periodic particle motion at the liquid interface can drive fluid flows inside the droplets, which may be useful for enhancing mass transport in droplet micro-reactors. The magneto-capillary effect at curved liquid interfaces offers new capabilities in magnetic manipulation: even static uniform fields can propel magnetic particles and the use of time-varying fields leads to steady particle motions of increasing complexity. These experimental demonstrations and the quantitative models that accompany them should both inspire and enable continued innovations in the use of magnetic fields to drive active processes in colloid and interface science. The final chapter highlights some specific directions for future work in this area