funcX: A Federated Function Serving Fabric for Science

Exploding data volumes and velocities, new computational methods and platforms, and ubiquitous connectivity demand new approaches to computation in the sciences. These new approaches must enable computation to be mobile, so that, for example, it can occur near data, be triggered by events (e.g., arrival of new data), be offloaded to specialized accelerators, or run remotely where resources are available. They also require new design approaches in which monolithic applications can be decomposed into smaller components, that may in turn be executed separately and on the most suitable resources. To address these needs we present funcX---a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. funcX's endpoint software can transform existing clouds, clusters, and supercomputers into function serving systems, while funcX's cloud-hosted service provides transparent, secure, and reliable function execution across a federated ecosystem of endpoints. We motivate the need for funcX with several scientific case studies, present our prototype design and implementation, show optimizations that deliver throughput in excess of 1 million functions per second, and demonstrate, via experiments on two supercomputers, that funcX can scale to more than more than 130000 concurrent workers.Comment: Accepted to ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC 2020). arXiv admin note: substantial text overlap with arXiv:1908.0490

Secure Data Management and Transmission Infrastructure for the Future Smart Grid

Power grid has played a crucial role since its inception in the Industrial Age. It has evolved from a wide network supplying energy for incorporated multiple areas to the largest cyber-physical system. Its security and reliability are crucial to any country’s economy and stability [1]. With the emergence of the new technologies and the growing pressure of the global warming, the aging power grid can no longer meet the requirements of the modern industry, which leads to the proposal of ‘smart grid’. In smart grid, both electricity and control information communicate in a massively distributed power network. It is essential for smart grid to deliver real-time data by communication network. By using smart meter, AMI can measure energy consumption, monitor loads, collect data and forward information to collectors. Smart grid is an intelligent network consists of many technologies in not only power but also information, telecommunications and control. The most famous structure of smart grid is the three-layer structure. It divides smart grid into three different layers, each layer has its own duty. All these three layers work together, providing us a smart grid that monitor and optimize the operations of all functional units from power generation to all the end-customers [2]. To enhance the security level of future smart grid, deploying a high secure level data transmission scheme on critical nodes is an effective and practical approach. A critical node is a communication node in a cyber-physical network which can be developed to meet certain requirements. It also has firewalls and capability of intrusion detection, so it is useful for a time-critical network system, in other words, it is suitable for future smart grid. The deployment of such a scheme can be tricky regarding to different network topologies. A simple and general way is to install it on every node in the network, that is to say all nodes in this network are critical nodes, but this way takes time, energy and money. Obviously, it is not the best way to do so. Thus, we propose a multi-objective evolutionary algorithm for the searching of critical nodes. A new scheme should be proposed for smart grid. Also, an optimal planning in power grid for embedding large system can effectively ensure every power station and substation to operate safely and detect anomalies in time. Using such a new method is a reliable method to meet increasing security challenges. The evolutionary frame helps in getting optimum without calculating the gradient of the objective function. In the meanwhile, a means of decomposition is useful for exploring solutions evenly in decision space. Furthermore, constraints handling technologies can place critical nodes on optimal locations so as to enhance system security even with several constraints of limited resources and/or hardware. The high-quality experimental results have validated the efficiency and applicability of the proposed approach. It has good reason to believe that the new algorithm has a promising space over the real-world multi-objective optimization problems extracted from power grid security domain. In this thesis, a cloud-based information infrastructure is proposed to deal with the big data storage and computation problems for the future smart grid, some challenges and limitations are addressed, and a new secure data management and transmission strategy regarding increasing security challenges of future smart grid are given as well

Design patterns for multi-agent simulations

The advent of mobile agent technology has brought along a few difficulties in designing a stable, efficient and scalable system for a certain problem. Agent-based simulations prove to be powerful tools for economic analyses. In this paper we aim at describing a set of design patterns which were specifically built for agents and multi-agent systems. The details of each design pattern discussed are presented and the possible applications and known issues are noted. In order to aid the software designers, we provide some examples of the basic implementation of these patterns using the JADE multi-agent framework.intelligent agent, multi-agent design, multi-agent simulation.

Secure Data Management and Transmission Infrastructure for the Future Smart Grid

Power grid has played a crucial role since its inception in the Industrial Age. It has evolved from a wide network supplying energy for incorporated multiple areas to the largest cyber-physical system. Its security and reliability are crucial to any country’s economy and stability [1]. With the emergence of the new technologies and the growing pressure of the global warming, the aging power grid can no longer meet the requirements of the modern industry, which leads to the proposal of ‘smart grid’. In smart grid, both electricity and control information communicate in a massively distributed power network. It is essential for smart grid to deliver real-time data by communication network. By using smart meter, AMI can measure energy consumption, monitor loads, collect data and forward information to collectors. Smart grid is an intelligent network consists of many technologies in not only power but also information, telecommunications and control. The most famous structure of smart grid is the three-layer structure. It divides smart grid into three different layers, each layer has its own duty. All these three layers work together, providing us a smart grid that monitor and optimize the operations of all functional units from power generation to all the end-customers [2]. To enhance the security level of future smart grid, deploying a high secure level data transmission scheme on critical nodes is an effective and practical approach. A critical node is a communication node in a cyber-physical network which can be developed to meet certain requirements. It also has firewalls and capability of intrusion detection, so it is useful for a time-critical network system, in other words, it is suitable for future smart grid. The deployment of such a scheme can be tricky regarding to different network topologies. A simple and general way is to install it on every node in the network, that is to say all nodes in this network are critical nodes, but this way takes time, energy and money. Obviously, it is not the best way to do so. Thus, we propose a multi-objective evolutionary algorithm for the searching of critical nodes. A new scheme should be proposed for smart grid. Also, an optimal planning in power grid for embedding large system can effectively ensure every power station and substation to operate safely and detect anomalies in time. Using such a new method is a reliable method to meet increasing security challenges. The evolutionary frame helps in getting optimum without calculating the gradient of the objective function. In the meanwhile, a means of decomposition is useful for exploring solutions evenly in decision space. Furthermore, constraints handling technologies can place critical nodes on optimal locations so as to enhance system security even with several constraints of limited resources and/or hardware. The high-quality experimental results have validated the efficiency and applicability of the proposed approach. It has good reason to believe that the new algorithm has a promising space over the real-world multi-objective optimization problems extracted from power grid security domain. In this thesis, a cloud-based information infrastructure is proposed to deal with the big data storage and computation problems for the future smart grid, some challenges and limitations are addressed, and a new secure data management and transmission strategy regarding increasing security challenges of future smart grid are given as well

DISCO: Distributed Multi-domain SDN Controllers

Modern multi-domain networks now span over datacenter networks, enterprise networks, customer sites and mobile entities. Such networks are critical and, thus, must be resilient, scalable and easily extensible. The emergence of Software-Defined Networking (SDN) protocols, which enables to decouple the data plane from the control plane and dynamically program the network, opens up new ways to architect such networks. In this paper, we propose DISCO, an open and extensible DIstributed SDN COntrol plane able to cope with the distributed and heterogeneous nature of modern overlay networks and wide area networks. DISCO controllers manage their own network domain and communicate with each others to provide end-to-end network services. This communication is based on a unique lightweight and highly manageable control channel used by agents to self-adaptively share aggregated network-wide information. We implemented DISCO on top of the Floodlight OpenFlow controller and the AMQP protocol. We demonstrated how DISCO's control plane dynamically adapts to heterogeneous network topologies while being resilient enough to survive to disruptions and attacks and providing classic functionalities such as end-point migration and network-wide traffic engineering. The experimentation results we present are organized around three use cases: inter-domain topology disruption, end-to-end priority service request and virtual machine migration

Service-oriented mobility of java code in web services-based architectures

Dissertação apresentada na Faculdade de Ciências e Tecnologias da Universidade Nova de Lisboa para a obtenção do Grau de Mestre em Engenharia InformáticaSoftware mobility consists of providing software components, the ability to migrate to a remote host with the purpose of interacting locally. In other words, this technology enables computations to be transferred from the current machine to a remote one. This powerful enhancement embodied in a traditional network fairly raises security concerns. For now, we believe that software mobility paradigm is confined to environments with bases of trust such as local area networks or middleware layers where security issues can be better controlled. Service-oriented computations reorganize the network architecture in the form of services, where components are more easily integrated, modified and removed. They have the ability to cooperate between them regardless the programming language used in their development. In addition, service-oriented computing is a widely accepted technology for the implementation of distributed applications, namely middleware. The work developed in this thesis consists of instantiating a model which combines software mobility and service-oriented paradigms as proposed by Paulino [20]. In this model, migrating sessions take advantage of the resources of a service-oriented network, creating thus an environment where the migration is modeled in terms of services instead of network nodes abstractions. In the instantiated model, we aim to apply the migration of Java programs in a context of a service-oriented architecture developed with Web services. This application comprises of a middleware layer that runs between the source program and the Web services technologies, and whose interface is the result of the mapping of the operations defined in the model. The evaluation performed to the instantiated model allows us to identify situations in which component migration to the server to interact locally is more advantageous in comparison to remote interacting with it

A NetLogo simulation tool for UAV-based secure location verification in crowd sensing

In the last decade, Unmanned Aerial Vehicle (UAV) production and interest is observing a continuous growth that appears not to decline. Meanwhile, thanks to the increase in the use of personal mobile devices and their onboard sensors, which is becoming more and more widespread, a new data collection technique, named crowd sensing, has emerged. Unfortunately, security remains a relevant issue, chiefly the integrity, i.e. the assurance that the information reported is trustworthy and accurate, still remains unsolved. The information the participant declares could be inaccurate or even counterfeit, due to flaws or fraud. Current literature shows no efficient solutions to the security problem, hence the arising need to point in this direction. The idea of this thesis came from the merging of the aforementioned mobile technologies. The aim is to fill the security gap in the crowd sensing process through UAVs employment, to prove trustworthiness and accuracy of sensorial data. The project presumes UAVs expedition in swarms where the data is originated, the authenticity of which could be promptly and directly verified thanks to the onboard sensors and, possibly, through interaction with other close sensors. Through the deployment of a simulator, written in the NetLogo language, it has been possible to reproduce a crowd sensing system and investigate the trustworthiness gap. We proposed and compared two different decision criteria to reveal attacks, named Dictatorship and Majority, both based on distance evaluation through radio frequency communication with the participant. In Dictatorship, it is sufficient that one UAV detects an inconsistency to warn an attack. In Majority, the half plus one of UAVs must detect an inconsistency in order to warn an attack. With regard to that, Dictatorship criterion showed certainly a better performance than Majority one. We further focused on participants' waiting time reduction acting on the algorithms to schedule swarms missions. A First Come First Served (FCFS)-like routine and an Insertion heuristic have been deployed. Since there are no statistical differences between the two for the tests we performed, the former scheduling algorithm is preferable due to its deterministic nature