28 research outputs found

    Reinforcement Learning

    Get PDF
    Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

    Design Space Exploration and Resource Management of Multi/Many-Core Systems

    Get PDF
    The increasing demand of processing a higher number of applications and related data on computing platforms has resulted in reliance on multi-/many-core chips as they facilitate parallel processing. However, there is a desire for these platforms to be energy-efficient and reliable, and they need to perform secure computations for the interest of the whole community. This book provides perspectives on the aforementioned aspects from leading researchers in terms of state-of-the-art contributions and upcoming trends

    An Artificial Immune System-Inspired Multiobjective Evolutionary Algorithm with Application to the Detection of Distributed Computer Network Intrusions

    Get PDF
    Today\u27s predominantly-employed signature-based intrusion detection systems are reactive in nature and storage-limited. Their operation depends upon catching an instance of an intrusion or virus after a potentially successful attack, performing post-mortem analysis on that instance and encoding it into a signature that is stored in its anomaly database. The time required to perform these tasks provides a window of vulnerability to DoD computer systems. Further, because of the current maximum size of an Internet Protocol-based message, the database would have to be able to maintain 25665535 possible signature combinations. In order to tighten this response cycle within storage constraints, this thesis presents an Artificial Immune System-inspired Multiobjective Evolutionary Algorithm intended to measure the vector of trade-off solutions among detectors with regard to two independent objectives: best classification fitness and optimal hypervolume size. Modeled in the spirit of the human biological immune system and intended to augment DoD network defense systems, our algorithm generates network traffic detectors that are dispersed throughout the network. These detectors promiscuously monitor network traffic for exact and variant abnormal system events, based on only the detector\u27s own data structure and the ID domain truth set, and respond heuristically. The application domain employed for testing was the MIT-DARPA 1999 intrusion detection data set, composed of 7.2 million packets of notional Air Force Base network traffic. Results show our proof-of-concept algorithm correctly classifies at best 86.48% of the normal and 99.9% of the abnormal events, attributed to a detector affinity threshold typically between 39-44%. Further, four of the 16 intrusion sequences were classified with a 0% false positive rate

    The 11th Conference of PhD Students in Computer Science

    Get PDF

    Advanced meta-heuristic approaches and their application to operational optimization in forest wildfire management

    Get PDF
    La última década ha sido testigo de un aumento vertiginoso de la cantidad y frecuencia de desastres a gran escala, principalmente debido a los fenómenos devastadores derivados de paradigmas climatológicos y ambientales a gran escala como el calentamiento global. De entre ellos son las inundaciones, huracanes y terremotos los desastres de mayor frecuencia de aparición y fatales consecuencias durante este período, tal como certifican los más de 20.000 muertos a consecuencia de un terremoto en la región de Gujarat (India) en 2001, o las 230.000 y 316.000 pérdidas humanas de los terremotos de Indonesia y Haití en 2004 y 2010, respectivamente. En este contexto, el enfoque de esta tesis se centra en una casuística concreta de desastre a media-gran escala cuya frecuencia y severidad han crecido de manera igualmente preocupante en los últimos tiempos: los incendios, definidos como un fuego de grandes dimensiones no voluntariamente iniciado por el ser humano, y que afecta a aquello que no está destinado a quemarse. Pese a la diversidad de iniciativas, campañas y procedimientos orientados a la minimización del impacto y las consecuencias de los incendios, varios sucesos fatales acontecidos en los últimos años han puesto en duda la efectividad de las políticas actuales de gestión de recursos contra incendios como aeronaves, vehículos terrestres, equipamiento de comunicaciones radio, logística de abastecimiento y las brigadas desplegadas en el área afectada. Un ejemplo manifiesto de esta falta de eficacia es la muerte de once bomberos ocurrida en un incendio de 130 kilómetros cuadrados en la zona de Guadalajara (España) en 2005, oficialmente atribuida a una deficiente coordinación entre el puesto de mando y los equipos de extinción debida, fundamentalmente, a problemas de cobertura en los sistemas de radiocomunicación. Aunque la causa de esta falta de coordinación ha sido cuestionada por las autoridades y los agentes involucrados desde entonces, lo cierto es que este suceso supone un ejemplo evidente de la necesidad de estudiar y desarrollar herramientas algorítmicas que ayuden al personal de comandancia a ejecutar óptimamente sus tareas de coordinación y control. Desafortunadamente la coyuntura de crisis económica mundial que azota con especial fuerza los países del Sur de Europa ha mermado dramáticamente las partidas presupuestarias para la prevención y extinción de incendios en beneficio de programas nacionales de reducción de déficit. A consecuencia de estos recortes, el coste ha irrumpido con fuerza como un criterio de extrema relevancia en la planificación operativa de este tipo de desastres: desde la perspectiva de un problema de optimización, los recursos contra incendios son actualmente gestionados con el objetivo fundamental de maximizar su efectividad contra incendios, sujeto a la restricción de que el coste agregado asociado a las decisiones tomadas no supere un determinado umbral presupuestario. Pese a que estas restricciones de coste están bien acotadas, en la práctica la mayoría de los procedimientos de gestión de recursos contra incendios están fuertemente determinados por la capacidad limitada del ser humano para tomar decisiones ágiles en escenarios de elevada complejidad y heterogeneidad. Por los motivos anteriormente expuestos, la presente Tesis doctoral propone la adopción de algoritmos meta-heurísticos para solventar eficientemente problemas de optimización que modelan procesos de gestión de recursos contra incendios. Esta familia de algoritmos de optimización es capaz de explorar el espacio solución de un problema dado merced a la aplicación iterativa de mecanismos inteligentes de búsqueda explorativa y explotativa, produciendo soluciones que sacrifican calidad por una complejidad computacional menor en comparación con la resultante de procesos determinísticos de búsqueda exhaustiva. En particular la Tesis plantea la búsqueda por harmonía (del inglés Harmony Search) como la técnica meta-heurística de optimización común a las herramientas diseñadas para la gestión de recursos en dos escenarios diferentes: ? El primer escenario analizado contempla el despliegue óptimo de redes de comunicación inalámbrica para la coordinación de equipos de extinción en incendios forestales de gran escala. Desde el punto de vista formal, el problema del despliegue dinámico de retransmisores que caracteriza matemáticamente este escenario consiste en estimar el número y localización de los retransmisores radio que deben ser desplegados en el área afectada por el incendio, de tal modo que el número de nodos móviles (i.e. recursos) con cobertura radio es maximizado a un coste del despliegue mínimo. A fin de reflejar la diversidad de equipamiento de retransmisión radio existente en la realidad, este problema es reformulado para considerar modelos de retransmisor con diferentes características de cobertura y coste. El problema resultante es resuelto de manera eficiente mediante sendos algoritmos mono- y bi-objetivo que conjugan 1) la Búsqueda por Harmonía como método de búsqueda global; y 2) una versión modificada del algoritmo de agrupación K-means como técnica de búsqueda local. El desempeño de los métodos propuestos es evaluado mediante experimentos numéricos basados en datos estadísticos reales de la Comunidad de Castilla la Mancha (España), merced a cuyos resultados queda certificada su practicidad a la hora de desplegar infraestructura de comunicación en este tipo de desastres. ? El segundo escenario bajo estudio se concentra en el despliegue y planificación óptima de vehículos aéreos de extinción de incendios basados en estimaciones predictivas del riesgo de incendio de una cierta área geográfica. De manera enunciativa, el problema subyacente busca la asignación de recursos a aeródromos y aeropuertos con restricciones de capacidad que maximice la utilidad de dichos recursos en relación al riesgo de incendio y minimice, a su vez, el coste de ejecutar dicha asignación. La formulación de este problema también considera, dentro de la definición de dicha función de utilidad, la distancia relativa entre aeropuerto, punto de potencial riesgo de incendio y el recurso acuífero (lago, río o mar) más cercano. Para su resolución eficiente se propone el uso de algoritmos de optimización basados, de nuevo, en la Búsqueda por Harmonía, incorporando además métodos voraces de reparación capacitiva. La aplicabilidad práctica de estos métodos es validada mediante experimentos numéricos en escenarios sintéticos y un caso práctico que incluye valores reales del riesgo de incendio, posiciones de recursos acuíferos e instalaciones aeroportuarias. En resumen, esta Tesis evidencia, desde un punto de vista práctico, que la meta-heurística moderna supone una solución algorítmica computacionalmente eficiente para tratar problemas de gestión de recursos contra incendios sujetos a restricciones de coste

    Closing Loops in Supply Chain Management: Designing Reverse Supply Chains for End-of-Life Vehicles.

    Get PDF
    In this thesis, the focus is on the design of reverse supply chains for end-of-life products, in particular end-of-life vehicles. For long-term success of end-of-life management, more economic stimuli are needed than is currently the case. Legislation as a single driving force is insufficient for companies to achieve closed loop supply chains. The key issue is to find eco-efficient solutions, i.e. design and operate an economically low cost network without violating applicable targets imposed by environmental legislation. In this thesis, a case study research methodology is adopted to develop design principles for network design and assess the consequences on the operations research models. Three case studies, which stem from the network of Auto Recycling Nederland, are described in detail.

    Design and Development of an Automated Mobile Manipulator for Industrial Applications

    Get PDF
    This thesis presents the modeling, control and coordination of an automated mobile manipulator. A mobile manipulator in this investigation consists of a robotic manipulator and a mobile platform resulting in a hybrid mechanism that includes a mobile platform for locomotion and a manipulator arm for manipulation. The structural complexity of a mobile manipulator is the main challenging issue because it includes several problems like adapting a manipulator and a redundancy mobile platform at non-holonomic constraints. The objective of the thesis is to fabricate an automated mobile manipulator and develop control algorithms that effectively coordinate the arm manipulation and mobility of mobile platform. The research work starts with deriving the motion equations of mobile manipulators. The derivation introduced here makes use of motion equations of robot manipulators and mobile platforms separately, and then integrated them as one entity. The kinematic analysis is performed in two ways namely forward & inverse kinematics. The motion analysis is performed for various WMPs such as, Omnidirectional WMP, Differential three WMP, Three wheeled omni-steer WMP, Tricycle WMP and Two steer WMP. From the obtained motion analysis results, Differential three WMP is chosen as the mobile platform for the developed mobile manipulator. Later motion analysis is carried out for 4-axis articulated arm. Danvit-Hartenberg representation is implemented to perform forward kinematic analysis. Because of this representation, one can easily understand the kinematic equation for a robotic arm. From the obtained arm equation, Inverse kinematic model for the 4-axis robotic manipulator is developed. Motion planning of an intelligent mobile robot is one of the most vital issues in the field of robotics, which includes the generation of optimal collision free trajectories within its work space and finally reaches its target position. For solving this problem, two evolutionary algorithms namely Particle Swarm Optimization (PSO) and Artificial Immune System (AIS) are introduced to move the mobile platform in intelligent manner. The developed algorithms are effective in avoiding obstacles, trap situations and generating optimal paths within its unknown environments. Once the robot reaches its goal (within the work space of the manipulator), the manipulator will generate its trajectories according to task assigned by the user. Simulation analyses are performed using MATLAB-2010 in order to validate the feasibility of the developed methodologies in various unknown environments. Additionally, experiments are carried out on an automated mobile manipulator. ATmega16 Microcontrollers are used to enable the entire robot system movement in desired trajectories by means of robot interface application program. The control program is developed in robot software (Keil) to control the mobile manipulator servomotors via a serial connection through a personal computer. To support the proposed control algorithms both simulation and experimental results are presented. Moreover, validation of the developed methodologies has been made with the ER-400 mobile platform

    Evolutionary Computation

    Get PDF
    This book presents several recent advances on Evolutionary Computation, specially evolution-based optimization methods and hybrid algorithms for several applications, from optimization and learning to pattern recognition and bioinformatics. This book also presents new algorithms based on several analogies and metafores, where one of them is based on philosophy, specifically on the philosophy of praxis and dialectics. In this book it is also presented interesting applications on bioinformatics, specially the use of particle swarms to discover gene expression patterns in DNA microarrays. Therefore, this book features representative work on the field of evolutionary computation and applied sciences. The intended audience is graduate, undergraduate, researchers, and anyone who wishes to become familiar with the latest research work on this field

    Seasat. Volume 2: Flight systems

    Get PDF
    Flight systems used in the Seasat Project are described. Included are (1) launch operation; (2) satellite performance after launch; (3) sensors that collected data; and (4) the launch vehicle that placed the satellite into Earth orbit. Techniques for sensor management are explained

    Advances in Robotics, Automation and Control

    Get PDF
    The book presents an excellent overview of the recent developments in the different areas of Robotics, Automation and Control. Through its 24 chapters, this book presents topics related to control and robot design; it also introduces new mathematical tools and techniques devoted to improve the system modeling and control. An important point is the use of rational agents and heuristic techniques to cope with the computational complexity required for controlling complex systems. Through this book, we also find navigation and vision algorithms, automatic handwritten comprehension and speech recognition systems that will be included in the next generation of productive systems developed by man
    corecore