Structural Return Maximization for Reinforcement Learning

Batch Reinforcement Learning (RL) algorithms attempt to choose a policy from a designer-provided class of policies given a fixed set of training data. Choosing the policy which maximizes an estimate of return often leads to over-fitting when only limited data is available, due to the size of the policy class in relation to the amount of data available. In this work, we focus on learning policy classes that are appropriately sized to the amount of data available. We accomplish this by using the principle of Structural Risk Minimization, from Statistical Learning Theory, which uses Rademacher complexity to identify a policy class that maximizes a bound on the return of the best policy in the chosen policy class, given the available data. Unlike similar batch RL approaches, our bound on return requires only extremely weak assumptions on the true system

Security in Computer and Information Sciences

This open access book constitutes the thoroughly refereed proceedings of the Second International Symposium on Computer and Information Sciences, EuroCybersec 2021, held in Nice, France, in October 2021. The 9 papers presented together with 1 invited paper were carefully reviewed and selected from 21 submissions. The papers focus on topics of security of distributed interconnected systems, software systems, Internet of Things, health informatics systems, energy systems, digital cities, digital economy, mobile networks, and the underlying physical and network infrastructures. This is an open access book

A review of cyber threats and defence approaches in emergency management

Emergency planners, first responders and relief workers increasingly rely on computational and communication systems that support all aspects of emergency management, from mitigation and preparedness to response and recovery. Failure of these systems, whether accidental or because of malicious action, can have severe implications for emergency management. Accidental failures have been extensively documented in the past and significant effort has been put into the development and introduction of more resilient technologies. At the same time researchers have been raising concerns about the potential of cyber attacks to cause physical disasters or to maximise the impact of one by intentionally impeding the work of the emergency services. Here, we provide a review of current research on the cyber threats to communication, sensing, information management and vehicular technologies used in emergency management. We emphasise on open issues for research, which are the cyber threats that have the potential to affect emergency management severely and for which solutions have not yet been proposed in the literature

Random Neural Networks and Optimisation

In this thesis we introduce new models and learning algorithms for the Random Neural Network (RNN), and we develop RNN-based and other approaches for the solution of emergency management optimisation problems. With respect to RNN developments, two novel supervised learning algorithms are proposed. The first, is a gradient descent algorithm for an RNN extension model that we have introduced, the RNN with synchronised interactions (RNNSI), which was inspired from the synchronised firing activity observed in brain neural circuits. The second algorithm is based on modelling the signal-flow equations in RNN as a nonnegative least squares (NNLS) problem. NNLS is solved using a limited-memory quasi-Newton algorithm specifically designed for the RNN case. Regarding the investigation of emergency management optimisation problems, we examine combinatorial assignment problems that require fast, distributed and close to optimal solution, under information uncertainty. We consider three different problems with the above characteristics associated with the assignment of emergency units to incidents with injured civilians (AEUI), the assignment of assets to tasks under execution uncertainty (ATAU), and the deployment of a robotic network to establish communication with trapped civilians (DRNCTC). AEUI is solved by training an RNN tool with instances of the optimisation problem and then using the trained RNN for decision making; training is achieved using the developed learning algorithms. For the solution of ATAU problem, we introduce two different approaches. The first is based on mapping parameters of the optimisation problem to RNN parameters, and the second on solving a sequence of minimum cost flow problems on appropriately constructed networks with estimated arc costs. For the exact solution of DRNCTC problem, we develop a mixed-integer linear programming formulation, which is based on network flows. Finally, we design and implement distributed heuristic algorithms for the deployment of robots when the civilian locations are known or uncertain

Navigability and synchronization in complex networks: a computational approach

Les xarxes complexes han demostrat ser una eina molt valuosa per estudiar sistemes reals, en part, gràcies a la creixent capacitat de computació. En aquesta tesi abordem computacionalment diversos problemes dividits en dos blocs. El primer bloc està motivat pels problemes que planteja la ràpida evolució de la Internet. D’una banda, el creixement exponencial de la xarxa està comprometent la seva escalabilitat per les dependències a les taules d’enrutament globals. Al Capítol 4 proposem un esquema d’enrutament descentralitzat que fa servir la projecció TSVD de l’estructura mesoscòpica de la xarxa com a mapa. Els resultats mostren que fent servir informació local podem guiar amb èxit en l’enrutament. Al Capítol 3 també avaluem la fiabilitat d’aquesta projecció davant el creixement de la xarxa. Els resultats indiquen que aquest mapa és robust i no necessita actualitzacions contínues. D’altra banda, la creixent demanda d’ample de banda és un factor potencial per produir congestió. Al Capítol 5 estenem un esquema d’enrutament dinàmic en el context de les xarxes multiplex, i l’analitzem amb xarxes sintètiques amb diferents assortativitats d’acoblament. Els resultats mostren que tenir en compte el volum de trànsit en l’enrutament retarda l’inici de la congestió. Tot i això, la distribució uniforme del trànsit produeix una transició de fase abrupta. Amb tot, l’acoblament assortatiu es presenta com la millor opció per a dissenys de xarxes òptimes. El segon bloc ve motivat per l’actual crisi financera mundial. Al Capítol 6 proposem estudiar la propagació de les crisis econòmiques utilitzant un model simple de xarxa formada per oscil·ladors integrate-and-fire, i caracteritzar la seva sincronització durant l’evolució de la xarxa de comerç. Els resultats mostren l’aparició d’un procés de globalització que dilueix les fronteres topològiques i accelera la propagació de les crisis financeres.Las redes complejas han demostrado ser una herramienta muy valiosa para estudiar sistemas reales, en parte, gracias a la creciente capacidad de computación. En esta tesis abordamos computacionalmente varios problemas divididos en dos bloques. El primer bloque está motivado por los problemas que plantea la rápida evolución de Internet. Por un lado, el crecimiento exponencial de la red está comprometiendo su escalabilidad por las dependencias a las tablas de enrutado globales. En el Capítulo 4 proponemos un esquema de enrutamiento descentralizado que utiliza la proyección TSVD de la estructura mesoscópica de la red como mapa. Los resultados muestran que utilizando información local podemos guiar con éxito el enrutado. En el Calítulo 3 también evaluamos la fiabilidad de esta proyección bajo cambios en la topología de la red. Los resultados indican que este mapa es robusto y no necesita actualizaciones continuas. Por otra parte, la creciente demanda de ancho de banda es un factor potencial de congestión. En el Capítulo 5 extendemos un esquema de enrutamiento dinámico en el marco de las redes multiplex, y lo analizamos en redes sintéticas con distintas asortatividades de acoplamiento. Los resultados muestran que tener en cuenta el volumen de tráfico en el enrutado retrasa la congestión. Sin embargo, la distribución uniforme del tráfico produce una transición de fase abrupta. Además, el acoplamiento asortativo se presenta como la mejor opción para diseños de redes óptimas. El segundo bloque viene motivado por la actual crisis financiera mundial. En el Capítulo 6 proponemos estudiar la propagación de las crisis económicas utilizando un modelo simple de red formada por osciladores integrate-and-fire, y caracterizar su sincronización durante la evolución de la red de comercio. Los resultados muestran la aparición de un proceso de globalización que diluye las fronteras topológicas y acelera la propagación de las crisis financieras.Complex networks are a powerful tool to study many real systems, partly thanks to the increasing capacity of computational resources. In this dissertation we address computationally a broad scope of problems that are framed in two parts. The first part is motivated by the issues posed by the rapid evolution of the Internet. On one side, the exponential growth of the network is compromising its scalability due to dependencies on global routing tables. In Chapter 4 we propose a decentralized routing scheme that exploits the TSVD projection of the mesoscopic structure of the network as a map. The results show that, using only local information, we can achieve good success rates in the routing process. Additionally, Chapter 3 evaluates the reliability of this projection when network topology changes. The results indicate that this map is very robust and does not need continual updates. On the other side, the increasing bandwidth demand is a potential trigger for congestion episodes. In Chapter 5 we extend a dynamic traffic-aware routing scheme to the context of multiplex networks, and we conduct the analysis on synthetic networks with different coupling assortativity. The results show that considering the traffic load in the transmission process delays the congestion onset. However, the uniform distribution of traffic produces an abrupt phase transition from free-flow to congested state. Withal, assortative coupling is depicted as the best consideration for optimal network designs. The second part is motivated by the current global financial crises. Chapter 6 presents a study on the spreading of economic crises using a simple model of networked integrate-and-fire oscillators and we characterize synchronization process on the evolving trade network. The results show the emergence of a globalization process that dilutes the topological borders and accelerates the spreading of financial crashes

Queuing analysis and optimization techniques for energy efficiency in packet networks

Energy efficiency in all aspects of human life has become a major concern, due to its significant environmental impact as well as its economic importance. Information and Communication Technology (ICT) plays a dual role in this; not only does it constitute a major consumer itself (estimated 2-10% of the global consumption), but is also expected to enable global energy efficiency through new technologies tightly dependent on networks (smart grid, smart homes, cloud computing etc.). To this purpose, this work studies the problem of energy efficiency in wired networks. As this subject has recently become very active in the research community, there is parallel research towards several research directions. In this work, the problem is being examined from its foundations and a solid analytical approach is presented. Specifically, a network model based on G-network queuing theory is built, which can incorporate all the important parameters of power consumption together with traditional performance metrics and routing control capability. This generalized model can be applied for any network case to build optimization algorithms and estimate the performance of different policies and network designs. Composite optimization goals functions are proposed, comprising both power consumption and performance metrics. A gradient descent optimization algorithm that can run in O(N3) time complexity is built thereof. Using power consumption characteristics of current and future equipment, several case studies are presented and the optimization results are evaluated. Moreover, a faster gradient-descent based heuristic and a decentralized algorithm are proposed. Apart from the routing control analysis, the case of a harsher energy saving solution, namely turning o the networking equipment, is also experimentally explored. Applying a tradeoff study on a laboratory testbed, implementation challenges are identified and conclusions significant for future work are drawn. Finally, a novel admission control mechanism is proposed and experimentally evaluated, which can monitor and manage the power consumption and performance of a network.Open Acces

Development Of Mobile Search Applications Over Structured Web Data Through Domain-specific Modeling Languages

Tez (Yüksek Lisans) -- İstanbul Teknik Üniversitesi, Fen Bilimleri Enstitüsü, 2012Thesis (M.Sc.) -- İstanbul Technical University, Institute of Science and Technology, 2012Kişisel bilgisayarlar için geliştirilen metin-tabanlı web arama gezgin cihazlara geçişte geliştirilip iyileştirilebilir. Web aramadaki yeni yöntemler kullanıcının gezgin cihazın kısıtlarına takılmadan arama yapmasını sağlayabilir. Hatta, uygun çözümler bu tür cihazların güçlü yanlarından da yararlanmayı sağlayabilir. Bu tezde arama alanındaki yeni eğilim ve teknolojilerin, özellikle de çok alanlı ve keşifçi aramanın gezgin ortamlara nasıl uygulanabileceğini göstermektedir.Text-based web search that is primarily designed for personal computers, can be enhanced and optimized while moving to mobile devices. New methods on web search may let user conduct the search without being hampered by the limitations of the device. Moreover, appropriate solutions may also exploit the advantages of such devices. This thesis summarizes new trends and technologies of searching, especially multi-domain and exploratory search, as well as demonstrating how they can be best applied to mobile environments.Yüksek LisansM.Sc