Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning

Recent advances in combining deep neural network architectures with reinforcement learning techniques have shown promising potential results in solving complex control problems with high dimensional state and action spaces. Inspired by these successes, in this paper, we build two kinds of reinforcement learning algorithms: deep policy-gradient and value-function based agents which can predict the best possible traffic signal for a traffic intersection. At each time step, these adaptive traffic light control agents receive a snapshot of the current state of a graphical traffic simulator and produce control signals. The policy-gradient based agent maps its observation directly to the control signal, however the value-function based agent first estimates values for all legal control signals. The agent then selects the optimal control action with the highest value. Our methods show promising results in a traffic network simulated in the SUMO traffic simulator, without suffering from instability issues during the training process

Desarrollo de un 'sniffer' para la generación de listas blancas para Snort

El objetivo de nuestro proyecto es el de contribuir con una nueva herramienta en el campo de la "Detección y Prevención" de ataques a la seguridad de Sistemas de Información en el entorno industrial. Para esto, el software sobre el que he estado trabajando, realizará un estudio estadístico de las tramas que transmiten información y señales de control entre dispositivos electrónicos que conforman segmentos de red en sistemas de entorno industrial. El "sniffer" que aquí presento, va algo mas allá con respecto a los analizadores de paquetes que ya conocemos (Ethereal, Wireshark...). Es capaz de extraer los campos de interés que caracterizan un conexión entre dichos dispositivos, almacenar estos datos en estructuras de almacenamiento dínámicas para datos adaptadas para este propósito, llegando a realizar una completa descripción del tráfico observado; para. Para posteriormente, con dicha información, realizar "listas blancas" (comportamiento permitido), las cuales serán utlizadas por el Detector de Intrusiones de software libre conocido como Snort. Además, nuestra herramienta, será capaz de interactuar con el sistema de ficheros de Snort. Utilizando las alternativas que los Sistemas Operativos Linux nos brindan a través de script. Somos de este modo capaces de automatizar el intercambio de archivos tanto de información como de configuración entre componentes software, facilitando en definitiva la labor de un admisnistrador de red

High-Accuracy Facial Depth Models derived from 3D Synthetic Data

In this paper, we explore how synthetically generated 3D face models can be used to construct a high accuracy ground truth for depth. This allows us to train the Convolutional Neural Networks (CNN) to solve facial depth estimation problems. These models provide sophisticated controls over image variations including pose, illumination, facial expressions and camera position. 2D training samples can be rendered from these models, typically in RGB format, together with depth information. Using synthetic facial animations, a dynamic facial expression or facial action data can be rendered for a sequence of image frames together with ground truth depth and additional metadata such as head pose, light direction, etc. The synthetic data is used to train a CNN based facial depth estimation system which is validated on both synthetic and real images. Potential fields of application include 3D reconstruction, driver monitoring systems, robotic vision systems, and advanced scene understanding

Optimization approach for the combined planning and control of an agile assembly system for electric vehicles

For some years now, the automotive industry has been challenged by growing market dynamics, shorter product lifecycles and customers' increasing demands for individualization. In order to cope with this development, the automotive assembly needs to adapt quickly to changing demands with a low level of investment in the future. Under the current circumstances, the traditional line assembly for high volume production is reaching its limits in terms of adaptability and scalability. A promising solution to address the current challenges is the concept of the agile assembly. The concept of agile assembly breaks up the rigid linkage of assembly stations and, thus, enables full flexibility in the sequence of assembly operations only limited by the precedence graph. Therefore, the routing of electric vehicles in the agile assembly is based on the availability of resources such as assembly stations and automated guided vehicles that handle the material supply. Further, by transferring the transport function to the vehicle itself, investments for convey or systems are eliminated. This research work presents an optimization approach for the machine scheduling and transportation planning, which derives instructions for electric vehicles, assembly stations as well as automated guided vehicles. For each electric vehicle, an optimized route is calculated, taking into account product-specific precedence graphs and minimizing the overall makespan. In addition, the machine scheduling and transportation planning is integrated into a combined planning and control concept which covers the allocation of resources and the assignment of capabilities of the entire assembly system. The approach is implemented and applied to a practical case of a compact electric vehicle. Thus, the work contributes to the evaluation of agile assembly systems in automotive production

Path Prediction For Efficient Order Release In Matrix-Structured Assembly Systems

Numerous research papers have already demonstrated the theoretical benefits of matrix-structured assembly systems. Nevertheless, such assembly systems have hardly been used in practice so far. The main reason for this, apart from the technical integration, is the complexity of controlling matrix-structured assembly systems. In theory, decentralized, agent-based control architectures have proven to be particularly suitable. However, order release has been largely neglected so far. Accordingly, the authors' previous work includes a conceptual approach for capacity-oriented order release in matrix-structured assembly systems. This previous approach calculates possible paths of an order and their capacity requirements considering both routing and sequence flexibility. Furthermore, by combining the possible paths of released orders with orders to be released and comparing them with the available capacity, the previously suggested approach can systematically carry out capacity-oriented release decisions. However, the NP-hard (NP: non-deterministic polynomial-time) problem arising from the consideration of all possible paths has a negative impact on the scalability and real-time capability of order release. Therefore, the present paper aims to extend the previously developed approach. By determining the most likely paths that a given order will take through the assembly system, the combination possibilities are limited in such a way that the total amount of calculations required to find a suitable order for release is reduced. Doing so, the NP-hardness of the previously developed approach can be circumvented. This work contributes to the practical realization and economic operation of matrix-structured assembly systems. The paper describes the logic of path prediction in detail and evaluates its impact on order release

Evaluation Of An Capacity-oriented, Agent-based Order Release For Matrix-structured Assembly Systems

To address growing challenges in automotive assembly with ever shorter innovation cycles, increasing variant diversity and uncertain market development, innovative concepts for assembly systems are needed. As a response, the concept of matrix-structured assembly system was introduced. Matrix-structured assembly systems break up with the rigid line structure of assembly stations and replace the cycle time-bound and product-specific station assignment of rigid line structure. A major challenge in the design of matrix-structured assembly systems is the assembly control. While certain approaches, mostly decentral and agent-based, are already capable to assign orders to assembly stations based on the availability of production resources, order release as part of the assembly control has been largely neglected. This is because routing and sequence flexibility lead to temporal uncertainty in the prediction of station-specific capacity requirements. Accordingly, the authors' previous work includes a conceptual methodology for capacity-oriented order release in matrix-structured assembly systems. After implementing this methodology, the actual benefit needs to be determined. For this purpose, the present paper suggests and applies a testing strategy based on the fundamentals of successful testing in software development domain. The testing aims to demonstrate the basic functionality of the implemented methodology as well as to compare it with other order release procedures that have been used for simulations in the context of matrix-structured assembly systems so far. It can be shown that the methodology for capacity-oriented order release in matrix-structured assembly systems achieves better adherence to delivery dates and lead times by anticipating bottlenecks compared to ConWIP control with a random order release. The knowledge gained from the testing strategy contributes to the improvement of order release procedures in matrix-structured assembly systems

From Complexity To Clarity In Sustainable Factory Planning: A Conceptual Approach For Data-driven Integration Of Green Factory KPIs In Manufacturing Site Selection

The selection of manufacturing facility locations entails high costs and long-term consequences. This necessitates an objective approach to mitigate uncertainties associated with subjective decision-making. Our paper builds upon previous research on data-driven location selection and conceptually extends it to integrate sustainability potential evaluation. By combining Green Factory Key Performance Indicators (KPIs), the authors aim to facilitate and standardize long-term decision-making in sustainable factory planning. After outlining the requirements, current state of the art, and limitations of location selection, we emphasize the need for integrating region-specific Green Factory KPIs with new data sources for site selection. Therefore, we propose a methodology involving a review of scientific literature and other sources to identify data sources for site selection, establishing research criteria for determining data suitability. The results include suitable subsets for location selection and future steps such as criteria application and target data determination. This paper contributes to paving the way for implementing sustainability-driven location selection strategies in factory planning. In conclusion, we outline a roadmap for further development and suggest two areas for future research: data collection and integration, as well as developing and validating a location selection app

Prosodic scoring of word hypotheses graphs

Prosodic boundary detection is important to disambiguate parsing, especially in spontaneous speech, where elliptic sentences occur frequently. Word graphs are an efficient interface between word recognition and parser. Prosodic classification of word chains has been published earlier. The adjustments necessary for applying these classification techniques to word graphs are discussed in this paper. When classifying a word hypothesis a set of context words has to be determined appropriately. A method has been developed to use stochastic language models for prosodic classification. This as well has been adopted for the use on word graphs. We also improved the set of acoustic-prosodic features with which the recognition errors were reduced by about 60% on the read speech we were working on previously, now achieving 10% error rate for 3 boundary classes and 3% for 2 accent classes. Moving to spontaneous speech the recognition error increases significantly (e.g. 16% for a 2-class boundary task). We show that even on word graphs the combination of language models which model a larger context with acoustic-prosodic classifiers reduces the recognition error by up to 50 %