Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning

Recent advances in combining deep neural network architectures with reinforcement learning techniques have shown promising potential results in solving complex control problems with high dimensional state and action spaces. Inspired by these successes, in this paper, we build two kinds of reinforcement learning algorithms: deep policy-gradient and value-function based agents which can predict the best possible traffic signal for a traffic intersection. At each time step, these adaptive traffic light control agents receive a snapshot of the current state of a graphical traffic simulator and produce control signals. The policy-gradient based agent maps its observation directly to the control signal, however the value-function based agent first estimates values for all legal control signals. The agent then selects the optimal control action with the highest value. Our methods show promising results in a traffic network simulated in the SUMO traffic simulator, without suffering from instability issues during the training process

Desarrollo de un 'sniffer' para la generación de listas blancas para Snort

El objetivo de nuestro proyecto es el de contribuir con una nueva herramienta en el campo de la "Detección y Prevención" de ataques a la seguridad de Sistemas de Información en el entorno industrial. Para esto, el software sobre el que he estado trabajando, realizará un estudio estadístico de las tramas que transmiten información y señales de control entre dispositivos electrónicos que conforman segmentos de red en sistemas de entorno industrial. El "sniffer" que aquí presento, va algo mas allá con respecto a los analizadores de paquetes que ya conocemos (Ethereal, Wireshark...). Es capaz de extraer los campos de interés que caracterizan un conexión entre dichos dispositivos, almacenar estos datos en estructuras de almacenamiento dínámicas para datos adaptadas para este propósito, llegando a realizar una completa descripción del tráfico observado; para. Para posteriormente, con dicha información, realizar "listas blancas" (comportamiento permitido), las cuales serán utlizadas por el Detector de Intrusiones de software libre conocido como Snort. Además, nuestra herramienta, será capaz de interactuar con el sistema de ficheros de Snort. Utilizando las alternativas que los Sistemas Operativos Linux nos brindan a través de script. Somos de este modo capaces de automatizar el intercambio de archivos tanto de información como de configuración entre componentes software, facilitando en definitiva la labor de un admisnistrador de red

High-Accuracy Facial Depth Models derived from 3D Synthetic Data

In this paper, we explore how synthetically generated 3D face models can be used to construct a high accuracy ground truth for depth. This allows us to train the Convolutional Neural Networks (CNN) to solve facial depth estimation problems. These models provide sophisticated controls over image variations including pose, illumination, facial expressions and camera position. 2D training samples can be rendered from these models, typically in RGB format, together with depth information. Using synthetic facial animations, a dynamic facial expression or facial action data can be rendered for a sequence of image frames together with ground truth depth and additional metadata such as head pose, light direction, etc. The synthetic data is used to train a CNN based facial depth estimation system which is validated on both synthetic and real images. Potential fields of application include 3D reconstruction, driver monitoring systems, robotic vision systems, and advanced scene understanding

From Complexity To Clarity In Sustainable Factory Planning: A Conceptual Approach For Data-driven Integration Of Green Factory KPIs In Manufacturing Site Selection

The selection of manufacturing facility locations entails high costs and long-term consequences. This necessitates an objective approach to mitigate uncertainties associated with subjective decision-making. Our paper builds upon previous research on data-driven location selection and conceptually extends it to integrate sustainability potential evaluation. By combining Green Factory Key Performance Indicators (KPIs), the authors aim to facilitate and standardize long-term decision-making in sustainable factory planning. After outlining the requirements, current state of the art, and limitations of location selection, we emphasize the need for integrating region-specific Green Factory KPIs with new data sources for site selection. Therefore, we propose a methodology involving a review of scientific literature and other sources to identify data sources for site selection, establishing research criteria for determining data suitability. The results include suitable subsets for location selection and future steps such as criteria application and target data determination. This paper contributes to paving the way for implementing sustainability-driven location selection strategies in factory planning. In conclusion, we outline a roadmap for further development and suggest two areas for future research: data collection and integration, as well as developing and validating a location selection app

A Security Enhancement of the Precision Time Protocol Using a Trusted Supervisor Node

The Precision Time Protocol (PTP) as described in IEEE 1588–2019 provides a sophisticated mechanism to achieve microsecond or even sub-microsecond synchronization of computer clocks in a well-designed and managed network, therefore meeting the needs of even the most time-sensitive industrial and financial applications. However, PTP is prone to many security threats that impact on a correct clock synchronization, leading to potentially devastating consequences. Here, the most vicious attacks are internal attacks, where a threat actor has full access to the infrastructure including any cryptographic keys used. This paper builds on existing research on the impact of internal attack strategies on PTP networks. It shows limitations of existing security approaches to tackle internal attacks and proposes a new security approach using a trusted supervisor node (TSN), in line with prong D as specified in IEEE 1588–2019. A TSN collects and analyzes delay and offset outputs of monitored slaves, as well as timestamps embedded in PTP synchronization messages, allowing it to detect abnormal patterns that point to an attack. The paper distinguishes between two types of TSN with different capabilities and proposes two different detection algorithms. Experiments show the ability of the proposed method to detect all internal PTP attacks, while outlining its limitations

An enhanced sum rate in the cluster based cognitive radio relay network using the sequential approach for the future Internet of Things

Abstract The cognitive radio relay plays a vital role in cognitive radio networking (CRN), as it can improve the cognitive sum rate, extend the coverage, and improve the spectral efficiency. However, cognitive relay aided CRNs cannot obtain a maximal sum rate, when the existing sensing approach is applied to a CRN. In this paper, we present an enhanced sum rate in the cluster based cognitive radio relay network utilizing a reporting framework in the sequential approach. In this approach a secondary user (SU) extends its sensing time until right before the beginning of its reporting time slot by utilizing the reporting framework. Secondly all the individual measurement results from each relay aided SU are passed on to the corresponding cluster head (CH) through a noisy reporting channel, while the CH with a soft-fusion report is forwarded to the fusion center that provides the final decision using the n-out-of-k-rule. With such extended sensing intervals and amplified reporting, a better sensing performance can be obtained than with a conventional non-sequential approach, therefore making it applicable for the future Internet of Things. In addition, the sum rate of the primary network and CCRRN are also investigated for the utilization reporting framework in the sequential approach with a relay using the n-out-of-k rule. By simulation, we show that the proposed sequential approach with a relay (Lemma 2) provides a significant sum rate gain compared to the conventional non-sequential approach with no relay (Lemma 1) under any condition

An enhanced sum rate in the cluster based cognitive radio relay network using the sequential approach for the future internet of things

The cognitive radio relay plays a vital role in cognitive radio networking (CRN), as it can improve the cognitive sum rate, extend the coverage, and improve the spectral efficiency. However, cognitive relay aided CRNs cannot obtain a maximal sum rate, when the existing sensing approach is applied to a CRN. In this paper, we present an enhanced sum rate in the cluster based cognitive radio relay network utilizing a reporting framework in the sequential approach. In this approach a secondary user (SU) extends its sensing time until right before the beginning of its reporting time slot by utilizing the reporting framework. Secondly all the individual measurement results from each relay aided SU are passed on to the corresponding cluster head (CH) through a noisy reporting channel, while the CH with a soft-fusion report is forwarded to the fusion center that provides the final decision using the n-out-of-k-rule. With such extended sensing intervals and amplified reporting, a better sensing performance can be obtained than with a conventional non-sequential approach, therefore making it applicable for the future Internet of Things. In addition, the sum rate of the primary network and CCRRN are also investigated for the utilization reporting framework in the sequential approach with a relay using the n-out-of-k rule. By simulation, we show that the proposed sequential approach with a relay (Lemma 2) provides a significant sum rate gain compared to the conventional non-sequential approach with no relay (Lemma 1) under any condition

Arrhythmia Identification from ECG Signals with a Neural Network Classifier Based on a Bayesian Framework

This paper presents an ANN-based diagnostic system for arrhythmia using Neural Network Classifier with Bayesian framework by time series biosignals. The Neural Network Classifier is built by the use of logistic regression model and back propagation algorithm. The prediction per-formance in training and test phases is evaluated by the False Rate. The dual threshold method is applied to determine diagnosis strategy and suppress false alarm signals. The results show that more than 90% prediction accuracy could be obtained using the improved methods in the study. Hopefully, the system can be further developed and fine-tuned for practical application