Hybridization of Nonlinear and Mixed-Integer Linear Programming for Aircraft Separation With Trajectory Recovery

International audienceThe approach presented in this article aims at finding a solution to the problem of conflict-free motion planning for multiple aircraft on the same flight level with trajectory recovery. One contribution of this work is to develop three consistent models, from a continuous-time representation to a discrete-time linear approximation. Each of these models guarantees separation at all times as well as trajectory recovery, but they are not equally difficult to solve. A new hybrid algorithm is thus developed in order to use the optimal solution of a mixed integer linear program as a starting point when solving a nonlinear formulation of the problem. The significance of this process is that it always finds a solution when the linear model is feasible while still taking into account the nonlinear nature of the problem. A test bed containing numerous data sets is then generated from three virtual scenarios. A comparative analysis with three different initialisations of the nonlinear optimisation validates the efficiency of the hybrid method

Genetic algorithms for automatic regroupment of Air Traffic Control sectors

http://dl.acm.org/citation.cfm?id=546164International audienceIn this paper, we show how genetic algorithms can be used to compute automatically a balanced regroupement of Air Traffic Control sectors to optimally reduce the number of controller teams during daily low flow periods

LSTM Path-Maker : une nouvelle stratégie pour la patrouille multiagent basée sur l'architecture LSTM

National audienceAbstract For over a decade, the multi-agent patrol task has received a growing attention from the multi-agent community due to its wide range of potential applications. However, the existing patrolling-specific algorithms based on deep learning algorithms are still in preliminary stages. In this paper, we propose to integrate a recurrent neural network as part of * Paper presented at the 52nd Hawaii International Conference on System Sciences (HICSS52 2019), titre, résumé et mots-clés en français ajou-tés. a multi-agent patrolling strategy. Hence we proposed a formal model of an LSTM-based agent strategy named LSTM Path Maker. The LSTM network is trained over simulation traces of a coordinated strategy, then embedded on each agent of the new strategy to patrol efficiently without communicating. Finally this new LSTM-based strategy is evaluated in simulation and compared with two representative strategies : a coordinated one and a reactive one. Preliminary results indicate that the proposed strategy is better than the reactive.Depuis plus d'une décennie, la tâche de la patrouille mul-tiagent a attiré l'attention de la communauté multiagent de manière croissante, en raison de son grand nombre d'applications potentielles. Cependant, les algorithmes ba-sés sur des méthodes d'apprentissage profond pour traiter cette tâche sont à ce jour peu développés. Dans cet article, nous proposons d'intégrer un réseau de neurone récurrent à une stratégie de patrouille multiagent. Ce faisant, nous avons proposé un modèle formel de stratégie d'agent basée sur l'architecture LSTM, que nous avons nommé LSTM-Path-Maker. Le réseau LSTM est entraîné sur des traces de simulation d'une stratégie coordonnée et centralisée, puis embarqué dans chaque agent en vue de patrouiller effica-cement sans communication. Enfin, cette nouvelle stratégie basée sur l'architecture LSTM est évaluée en simulation et comparée d'une part à une stratégie coordonnée et d'autre part à une stratégie réactive. Les résultats préliminaires in-diquent que la stratégie proposée est meilleure que la stra-tégie réactive

Decentralized multi-agent patrolling strategies using global idleness estimation

International audienceThis paper presents preliminary results in the challenge of developing decentralised strategies approaching the performances of centralised ones. Indeed, the latter are better than the former due to centralisation of information. The approach studied here involves the estimation of node idlenesses derived from the paths of all agents, also known as real idlenesses, on the basis of those derived from the path of each agent considered alone, also known as individual idlenesses. This relation between real and individual idlenesses is learnt using traces of execution of a centralised strategy by optimising an error criterion. The strategy thereupon, uses online the learnt relation and is assessed according to certain evaluation criteria. The results indicate that such a relation between perceived and real idlenesses is not a function, leading to large values of the fitting criterion. Finally, the assessment of the strategy shows that performances are good in terms of mean interval but unsatisfactory in terms of quadratic mean interval

Qualitative Possibilistic Mixed-Observable MDPs

National audiencePossibilistic and qualitative POMDPs (pi-POMDPs) are counterparts of POMDPs used to model situations where the agent's initial belief or observation probabilities are imprecise due to lack of past experiences or insufficient data collection. However, like probabilistic POMDPs, optimally solving pi-POMDPs is intractable: the finite belief state space exponentially grows with the number of system's states. In this paper, a possibilistic version of Mixed-Observable MDPs is presented to get around this issue: the complexity of solving pi-POMDPs, some state variables of which are fully observable, can be then dramatically reduced. A value iteration algorithm for this new formulation under infinite horizon is next proposed and the optimality of the returned policy (for a specified criterion) is shown assuming the existence of a "stay" action in some goal states. Experimental work finally shows that this possibilistic model outperforms probabilistic POMDPs commonly used in robotics, for a target recognition problem where the agent's observations are imprecise

Structured Possibilistic Planning Using Decision Diagrams

National audienceQualitative Possibilistic Mixed-Observable MDPs (π-MOMDPs), generalizing π-MDPs and π-POMDPs, are well-suited models to planning under uncertainty with mixed-observability when transition, observation and reward functions are not precisely known and can be qualitatively described. Functions defining the model as well as intermediate calculations are valued in a finite possibilistic scale L, which induces a finite belief state space under partial observability contrary to its probabilistic counterpart. In this paper, we propose the first study of factored π-MOMDP models in order to solve large structured planning problems under qualitative uncertainty, or considered as qualitative approximations of probabilistic problems. Building upon the SPUDD algorithm for solving factored (probabilistic) MDPs, we conceived a symbolic algorithm named PPUDD for solving factored π-MOMDPs. Whereas SPUDD’s decision diagrams’ leaves may be as large as the state space since their values are real numbers aggregated through additions and multiplications, PPUDD’s ones always remain in the finite scale L via min and max operations only. Our experiments show that PPUDD’s computation time is much lower than SPUDD, Symbolic-HSVI and APPL for possibilistic and probabilistic versions of the same benchmarks under either total or mixed observability, while still providing high-quality policies

Planning in Partially Observable Domains with Fuzzy Epistemic States and Probabilistic Dynamics

International audienceA new translation from Partially Observable MDP into Fully Observable MDP is described here. Unlike the classical translation, the resulting problem state space is finite, making MDP solvers able to solve this simplified version of the initial partially observable problem: this approach encodes agent beliefs with possibility distributions over states, leading to an MDP whose state space is a finite set of epistemic states. After a short description of the POMDP framework as well as notions of Possibility Theory, the translation is described in a formal manner with semantic arguments. Then actual computations of this transformation are detailed, in order to highly benefit from the factored structure of the initial POMDP in the final MDP size reduction and structure. Finally size reduction and tractability of the resulting MDP is illustrated on a simple POMDP problem

Optimisation de POMDP : quelles récompenses sont réellement attendues à l'exécution de la politique ?

Les Processus Décisionnels Markoviens Partiellement Observables sont actuellement un sujet d'intérêt dans la communauté scientifique grâce aux progrès observés dans des algorithmes de résolution et dans les capacités numériques de calcul. La plupart de ces algorithmes sont focalisés sur la résolution d'un critère de performance, qui a pour ambition de caractériser les politiques qui permettront de générer les séquences de récompenses le plus importantes possibles. Dans la planification en Intelligence Artificielle, l'attention est tournée vers un critère qui optimise une somme pondérée des récompenses, et, pour des applications en perception active d'autre part, le critère est souvent défini en termes de gain d'information (entropie de Shannon). Aucun de ces critères ne prend en compte les récompenses réellement acquises lors de l'exécution de la politique. En effet, le premier critère est une moyenne linéaire sur l'espace d'états de croyance, de sorte que l'agent ne tend pas à obtenir une meilleure information des différentes observations, alors que le second critère ne prend pas en compte les récompenses. Ainsi, motivés par des exemples démonstratifs, nous étudions deux combinaisons, additive et multiplicative, de ces critères afin d'obtenir une meilleur séquence de récompenses et de gain d'information lors de l'exécution de la politique. Nous comparons nos critères avec le critère classique optimisé (y-pondéré) dans le cadre POMDP et nous soulignons l'intérêt de considérer un nouveau critère hybride non-linéaire pour des applications réalistes de reconnaissance et pistage multi-cibles

Une approche du traitement du temps dans le cadre MDP: trois méthodes de découpage de la droite temporelle

De nombreux problèmes de planification s'inscrivent dans un environnement instationnaire. Dans le cadre de la décision dans l'incertain sur horizon infini, pour les problèmes stationnaires à l'infini, on se propose de définir un cadre de modélisation dérivé du modèle SMDP dans lequel la variable temporelle est observable par l'agent. Dans ce cadre, nous développons trois approches de résolution différentes afin de générer des politiques qui en tout état discret du système spécifient l'action optimale à entreprendre en fonction de la date courante

POMDP solving: what rewards do you really expect at execution?

Partially Observable Markov Decision Processes have gained an increasing interest in many research communities, due to sensible improvements of their optimization algorithms and of computers capabilities. Yet, most research focus on optimizing either average accumulated rewards (AI planning) or direct entropy (active perception), whereas none of them matches the rewards actually gathered at execution. Indeed, the first optimization criterion linearly averages over all belief states, so that it does not gain best information from different observations, while the second one totally discards rewards. Thus, motivated by simple demonstrative examples, we study an additive combination of these two criteria to get the best of reward gathering and information acquisition at execution. We then compare our criterion with classical ones, and highlight the need to consider new hybrid non-linear criteria, on a realistic multi-target recognition and tracking mission