Reinforcement machine learning for predictive analytics in smart cities

The digitization of our lives cause a shift in the data production as well as in the required data management. Numerous nodes are capable of producing huge volumes of data in our everyday activities. Sensors, personal smart devices as well as the Internet of Things (IoT) paradigm lead to a vast infrastructure that covers all the aspects of activities in modern societies. In the most of the cases, the critical issue for public authorities (usually, local, like municipalities) is the efficient management of data towards the support of novel services. The reason is that analytics provided on top of the collected data could help in the delivery of new applications that will facilitate citizens’ lives. However, the provision of analytics demands intelligent techniques for the underlying data management. The most known technique is the separation of huge volumes of data into a number of parts and their parallel management to limit the required time for the delivery of analytics. Afterwards, analytics requests in the form of queries could be realized and derive the necessary knowledge for supporting intelligent applications. In this paper, we define the concept of a Query Controller ( QC ) that receives queries for analytics and assigns each of them to a processor placed in front of each data partition. We discuss an intelligent process for query assignments that adopts Machine Learning (ML). We adopt two learning schemes, i.e., Reinforcement Learning (RL) and clustering. We report on the comparison of the two schemes and elaborate on their combination. Our aim is to provide an efficient framework to support the decision making of the QC that should swiftly select the appropriate processor for each query. We provide mathematical formulations for the discussed problem and present simulation results. Through a comprehensive experimental evaluation, we reveal the advantages of the proposed models and describe the outcomes results while comparing them with a deterministic framework

Dynamic distributed clustering in wireless sensor networks via Voronoi tessellation control

This paper presents two dynamic and distributed clustering algorithms for Wireless Sensor Networks (WSNs). Clustering approaches are used in WSNs to improve the network lifetime and scalability by balancing the workload among the clusters. Each cluster is managed by a cluster head (CH) node. The first algorithm requires the CH nodes to be mobile: by dynamically varying the CH node positions, the algorithm is proved to converge to a specific partition of the mission area, the generalised Voronoi tessellation, in which the loads of the CH nodes are balanced. Conversely, if the CH nodes are fixed, a weighted Voronoi clustering approach is proposed with the same load-balancing objective: a reinforcement learning approach is used to dynamically vary the mission space partition by controlling the weights of the Voronoi regions. Numerical simulations are provided to validate the approaches

An Overview on Application of Machine Learning Techniques in Optical Networks

Today's telecommunication networks have become sources of enormous amounts of widely heterogeneous data. This information can be retrieved from network traffic traces, network alarms, signal quality indicators, users' behavioral data, etc. Advanced mathematical tools are required to extract meaningful information from these data and take decisions pertaining to the proper functioning of the networks from the network-generated data. Among these mathematical tools, Machine Learning (ML) is regarded as one of the most promising methodological approaches to perform network-data analysis and enable automated network self-configuration and fault management. The adoption of ML techniques in the field of optical communication networks is motivated by the unprecedented growth of network complexity faced by optical networks in the last few years. Such complexity increase is due to the introduction of a huge number of adjustable and interdependent system parameters (e.g., routing configurations, modulation format, symbol rate, coding schemes, etc.) that are enabled by the usage of coherent transmission/reception technologies, advanced digital signal processing and compensation of nonlinear effects in optical fiber propagation. In this paper we provide an overview of the application of ML to optical communications and networking. We classify and survey relevant literature dealing with the topic, and we also provide an introductory tutorial on ML for researchers and practitioners interested in this field. Although a good number of research papers have recently appeared, the application of ML to optical networks is still in its infancy: to stimulate further work in this area, we conclude the paper proposing new possible research directions

A system for crack pattern detection, characterization and diagnosis in concrete structures by means of image processing and machine learning techniques

A system that attempts to find cracks in a RGB picture of a concrete beam, measure the cracks angles and widths; and classify crack patterns in 3 pathologies has been designed and implemented in the MATLAB programming language. The system is divided in three parts: Crack Detection, Crack Clustering and Crack Pattern Classification. The Crack Detection algorithm attempts to detect pixels depicting cracks in a region of interest (ROI) and measure the crack angles and widths. The input ROI is segmented several times: First with an artificial Neural Network (NN) that classifies image patches in "Crack" or "Not Crack", then with the Canny Edge detector and finally with the local Mean and Standard deviation of the intensities. Then all neighborhoods in the mask are passed through special modified line kernels called "orientation kernels" designed to detect cracks and measure their angles; in order to obtain the width measurement, a line of pixels perpendicular to the crack is extracted and with an approximation of the intensity gradient of that line the width is measured. This algorithm outputs a mask the same size as the input picture with the measured angles and widths. The Crack Clustering algorithm groups up all the crack image patches recognized from the Crack Detection to approximate clusters that match the quantity of cracks in the image. To achieve this a special distance metric has been designed to group up aligned crack image patches; then with an algorithm based on the connectivity among the crack patches the clusters are obtained. The Crack Pattern Classification takes the mask outputs from the Crack Detection step as input for a Neural Network (NN) designed to classify crack patterns in concrete beams in 3 classes: Flexion, Shear and Corrosion-Bond cracks. The width and angles masks are first transformed into a Feature matrix to reduce the degrees of freedom of the input for the NN. To achieve a desirable classification in cases when more than 1 pathology is present, every angle and width mask is separated in as many Features matrices as clusters found with the Clustering algorithm; then separately classified with the NN designed. Several photos depicting concrete surfaces are presented as examples to check the accuracy of the width and angle measurements from the Crack Detection step. Other photos showing concrete beams with crack patterns are used to check the classification prowess of the Crack Pattern Classification step. The most important conclusion of this work is the transference of empirical knowledge from rehabilitation of structures to a machine learning model in order to diagnose the damage on an element. This opens possibilities for new lines of research to make a larger system with wider utilities, more pathologies and elements to classify.Se ha diseñado un sistema que a partir de una foto a color de una superficie de hormigón realiza las siguientes tareas: Detectar fisuras, medir su ángulo y ancho, clasificar los patrones de fisuración asociados a tres patologías del hormigón; el cual ha sido implementado en el lenguaje de programación MATLAB. El sistema se divide en tres partes: Detección y medición de fisuras; algoritmo de análisis de grupos de fisuras y clasificación de patrones de fisuración. El algoritmo de detección de fisuras detecta los pixeles en donde hay fisuras dentro de una región de interés y mide el ancho y ángulos de dichas fisuras. La región de interés es segmentada varias veces: Primero con una red neuronal artificial que clasifica teselas de la imagen en dos categorías "Fisura" y "No fisura"; después se hace otra segmentación con un filtro Canny de detección de bordes y finalmente se segmenta con la media y desviaciones intensidades en teselas de la imagen. Entonces todas las localidades de la máscara de imagen obtenida con las segmentaciones anteriores se las pasa por varios filtros de detección de líneas diseñados para detectar y medir las fisuras. Este algoritmo resulta en dos máscaras de imagen con los anchos y ángulos de todas las fisuras encontradas en la región de interés. El algoritmo de análisis de grupos de teselas reconocidas como fisuras se hace para intentar reconocer y contar cuantas fisuras aparecen en la región de interés. Para lograr esto se diseñó una función de distancia para que teselas de fisura alineadas se junten; después con un algoritmo basado en la conectividad entre estas teselas o vectores fisura se obtienen los grupos de fisura. La clasificación de patrones de fisuración toma las máscaras de imagen del paso de detección de fisuras y lo toma como dato de entrada para una red neuronal diseñada para clasificar patrones de fisuración en tres categorías seleccionadas: Flexión, Cortante y Corrosión-Adherencia. Las máscaras de imagen de ancho y ángulo se transforman en una matriz de características para reducir los grados de libertad del problema, estandarizar un tamaño para la entrada al modelo de red neuronal. Para lograr clasificaciones correctas cuando más de 1 patología está presente en las vigas, cada máscara de imagen de ángulos y anchos de fisura se divide en cuantos cuantos grupos de teselas de fisuras haya en la imagen, y para cada uno se obtienen una matriz de características. Entonces se clasifican separadamente dichas matrices con la red neuronal artificial diseñada. Varias fotos con superficies de hormigón se presentan como ejemplos para evaluar la precisión de las mediciones de ancho y ángulo del paso de detección de fisuras. Otras fotos mostrando patrones de fisuración en vigas de hormigón se muestran para revisar las capacidades de diagnóstico del paso de clasificación de patrones de fisuración. La conclusión más importante de este trabajo es la transferencia del conocimiento empírico de la rehabilitación de estructuras hacia un modelo de inteligencia artificial para diagnosticar el daño en un elemento de la estructura. Esto abre un campo grande de líneas de investigación hacia el diseño e implementación de sistemas automatizados con más utilidades, más patologías y elementos para clasificar.Postprint (published version

A survey of machine learning techniques applied to self organizing cellular networks

In this paper, a survey of the literature of the past fifteen years involving Machine Learning (ML) algorithms applied to self organizing cellular networks is performed. In order for future networks to overcome the current limitations and address the issues of current cellular systems, it is clear that more intelligence needs to be deployed, so that a fully autonomous and flexible network can be enabled. This paper focuses on the learning perspective of Self Organizing Networks (SON) solutions and provides, not only an overview of the most common ML techniques encountered in cellular networks, but also manages to classify each paper in terms of its learning solution, while also giving some examples. The authors also classify each paper in terms of its self-organizing use-case and discuss how each proposed solution performed. In addition, a comparison between the most commonly found ML algorithms in terms of certain SON metrics is performed and general guidelines on when to choose each ML algorithm for each SON function are proposed. Lastly, this work also provides future research directions and new paradigms that the use of more robust and intelligent algorithms, together with data gathered by operators, can bring to the cellular networks domain and fully enable the concept of SON in the near future