Middleware and Architecture for Advanced Applications of Cyber-physical Systems

In this thesis, we address issues related to middleware, architecture and applications of cyber-physical systems. The first problem we address is the cross-layer design of cyber-physical systems to cope with interactions between the cyber layer and the physical layer in a dynamic environment. We propose a bi-directional middleware that allows the optimal utilization of the common resources for the benefit of either or both the layers in order to obtain overall system performance. The case study of network connectivity preservation in a vehicular formation illustrates how this approach can be applied to a particular situation where the network connectivity drives the application layer. Next we address another aspect of cross-layer impact: the problem that arises when network performance, in this case delay performance, affects control system performance. We propose a two-pronged approach involving a flexible adaptive model identification algorithm with outlier rejection, which in turn uses an adaptive system model to detect and reject outliers, thus shielding the estimation algorithm and thereby improving reliability. We experimentally demonstrate that the outlier rejection approach which intercepts and filters the data, combined with simultaneous model adaptation, can result in improved performance of Model Predictive Control in the vehicular testbed. Then we turn to two advanced applications of cyber-physical systems. First, we address the problem of security of cyber-physical systems. We consider the context of an intelligent transportation system in which a malicious sensor node manipulates the position data of one of the autonomous cars to deviate from a safe trajectory and collide with other cars. In order to secure the safety of such systems where sensor measurements are compromised, we employ the procedure of “dynamic watermarking”. This procedure enables an honest node in the control loop to detect the existence of a malicious node within the feedback loop. We demonstrate in the testbed that dynamic watermarking can indeed protect cars against collisions even in the presence of sensor attacks. The second application of cyber-physical systems that we consider is cyber-manufacturing which is an origami-type laser-based custom manufacturing machine employing folding and cutting of sheet material to manufacture 3D objects. We have developed such a system for use in a laser-based autonomous custom manufacturing machine equipped with real-time sensing and control. The basic elements in the architecture are a laser processing machine, a sensing system to estimate the state of the workpiece, a control system determining control inputs for a laser system based on the estimated data, a robotic arm manipulating the workpiece in the work space, and middleware supporting the communication among the systems. We demonstrate automated 3D laser cutting and bending to fabricate a 3D product as an experimental result. Lastly, we address the problem of traffic management of an unmanned aerial system. In an effort to improve the performance of the traffic management for unmanned aircrafts, we propose a probability-based collision resolution algorithm. The proposed algorithm analyzes the planned trajectories to calculate their collision probabilities, and modifies individual drone starting times to reduce the probability of collision, while attempting to preserve high performance. Our simulation results demonstrate that the proposed algorithm improves the performance of the drone traffic management by guaranteeing high safety with low modification of the starting times

Middleware and Architecture for Advanced Applications of Cyber-physical Systems

In this thesis, we address issues related to middleware, architecture and applications of cyber-physical systems. The first problem we address is the cross-layer design of cyber-physical systems to cope with interactions between the cyber layer and the physical layer in a dynamic environment. We propose a bi-directional middleware that allows the optimal utilization of the common resources for the benefit of either or both the layers in order to obtain overall system performance. The case study of network connectivity preservation in a vehicular formation illustrates how this approach can be applied to a particular situation where the network connectivity drives the application layer. Next we address another aspect of cross-layer impact: the problem that arises when network performance, in this case delay performance, affects control system performance. We propose a two-pronged approach involving a flexible adaptive model identification algorithm with outlier rejection, which in turn uses an adaptive system model to detect and reject outliers, thus shielding the estimation algorithm and thereby improving reliability. We experimentally demonstrate that the outlier rejection approach which intercepts and filters the data, combined with simultaneous model adaptation, can result in improved performance of Model Predictive Control in the vehicular testbed. Then we turn to two advanced applications of cyber-physical systems. First, we address the problem of security of cyber-physical systems. We consider the context of an intelligent transportation system in which a malicious sensor node manipulates the position data of one of the autonomous cars to deviate from a safe trajectory and collide with other cars. In order to secure the safety of such systems where sensor measurements are compromised, we employ the procedure of “dynamic watermarking”. This procedure enables an honest node in the control loop to detect the existence of a malicious node within the feedback loop. We demonstrate in the testbed that dynamic watermarking can indeed protect cars against collisions even in the presence of sensor attacks. The second application of cyber-physical systems that we consider is cyber-manufacturing which is an origami-type laser-based custom manufacturing machine employing folding and cutting of sheet material to manufacture 3D objects. We have developed such a system for use in a laser-based autonomous custom manufacturing machine equipped with real-time sensing and control. The basic elements in the architecture are a laser processing machine, a sensing system to estimate the state of the workpiece, a control system determining control inputs for a laser system based on the estimated data, a robotic arm manipulating the workpiece in the work space, and middleware supporting the communication among the systems. We demonstrate automated 3D laser cutting and bending to fabricate a 3D product as an experimental result. Lastly, we address the problem of traffic management of an unmanned aerial system. In an effort to improve the performance of the traffic management for unmanned aircrafts, we propose a probability-based collision resolution algorithm. The proposed algorithm analyzes the planned trajectories to calculate their collision probabilities, and modifies individual drone starting times to reduce the probability of collision, while attempting to preserve high performance. Our simulation results demonstrate that the proposed algorithm improves the performance of the drone traffic management by guaranteeing high safety with low modification of the starting times

RAHIM: Robust Adaptive Approach Based on Hierarchical Monitoring Providing Trust Aggregation for Wireless Sensor Networks

In-network data aggregation has a great impact on the energy consumption in large-scale wireless sensor networks. However, the resource constraints and vulnerable deployment environments challenge the application of this technique in terms of security and efficiency. A compromised node may forge arbitrary aggregation value and mislead the base station into trusting a false reading. In this paper, we present RAHIM, a reactive defense to secure data aggregation scheme in cluster-based wireless sensor networks. The proposed scheme is based on a novel application of adaptive hierarchical level of monitoring providing accuracy of data aggregation result in lightweight manner, even if all aggregator nodes and a part of sensors are compromised in the network

Shooter localization and weapon classification with soldier-wearable networked sensors

The paper presents a wireless sensor network-based mobile countersniper system. A sensor node consists of a helmetmounted microphone array, a COTS MICAz mote for internode communication and a custom sensorboard that implements the acoustic detection and Time of Arrival (ToA) estimation algorithms on an FPGA. A 3-axis compass provides self orientation and Bluetooth is used for communication with the soldier’s PDA running the data fusion and the user interface. The heterogeneous sensor fusion algorithm can work with data from a single sensor or it can fuse ToA or Angle of Arrival (AoA) observations of muzzle blasts and ballistic shockwaves from multiple sensors. The system estimates the trajectory, the range, the caliber and the weapon type. The paper presents the system design and the results from an independent evaluation at the US Army Aberdeen Test Center. The system performance is characterized by 1-degree trajectory precision and over 95 % caliber estimation accuracy for all shots, and close to 100 % weapon estimation accuracy for 4 out of 6 guns tested

An integrative framework for cooperative production resources in smart manufacturing

Under the push of Industry 4.0 paradigm modern manufacturing companies are dealing with a significant digital transition, with the aim to better address the challenges posed by the growing complexity of globalized businesses (Hermann, Pentek, & Otto, Design principles for industrie 4.0 scenarios, 2016). One basic principle of this paradigm is that products, machines, systems and business are always connected to create an intelligent network along the entire factory\u2019s value chain. According to this vision, manufacturing resources are being transformed from monolithic entities into distributed components, which are loosely coupled and autonomous but nevertheless provided of the networking and connectivity capabilities enabled by the increasingly widespread Industrial Internet of Things technology. Under these conditions, they become capable of working together in a reliable and predictable manner, collaborating among themselves in a highly efficient way. Such a mechanism of synergistic collaboration is crucial for the correct evolution of any organization ranging from a multi-cellular organism to a complex modern manufacturing system (Moghaddam & Nof, 2017). Specifically of the last scenario, which is the field of our study, collaboration enables involved resources to exchange relevant information about the evolution of their context. These information can be in turn elaborated to make some decisions, and trigger some actions. In this way connected resources can modify their structure and configuration in response to specific business or operational variations (Alexopoulos, Makris, Xanthakis, Sipsas, & Chryssolouris, 2016). Such a model of \u201csocial\u201d and context-aware resources can contribute to the realization of a highly flexible, robust and responsive manufacturing system, which is an objective particularly relevant in the modern factories, as its inclusion in the scope of the priority research lines for the H2020 three-year period 2018-2020 can demonstrate (EFFRA, 2016). Interesting examples of these resources are self-organized logistics which can react to unexpected changes occurred in production or machines capable to predict failures on the basis of the contextual information and then trigger adjustments processes autonomously. This vision of collaborative and cooperative resources can be realized with the support of several studies in various fields ranging from information and communication technologies to artificial intelligence. An update state of the art highlights significant recent achievements that have been making these resources more intelligent and closer to the user needs. However, we are still far from an overall implementation of the vision, which is hindered by three major issues. The first one is the limited capability of a large part of the resources distributed within the shop floor to automatically interpret the exchanged information in a meaningful manner (semantic interoperability) (Atzori, Iera, & Morabito, 2010). This issue is mainly due to the high heterogeneity of data model formats adopted by the different resources used within the shop floor (Modoni, Doukas, Terkaj, Sacco, & Mourtzis, 2016). Another open issue is the lack of efficient methods to fully virtualize the physical resources (Rosen, von Wichert, Lo, & Bettenhausen, 2015), since only pairing physical resource with its digital counterpart that abstracts the complexity of the real world, it is possible to augment communication and collaboration capabilities of the physical component. The third issue is a side effect of the ongoing technological ICT evolutions affecting all the manufacturing companies and consists in the continuous growth of the number of threats and vulnerabilities, which can both jeopardize the cybersecurity of the overall manufacturing system (Wells, Camelio, Williams, & White, 2014). For this reason, aspects related with cyber-security should be considered at the early stage of the design of any ICT solution, in order to prevent potential threats and vulnerabilities. All three of the above mentioned open issues have been addressed in this research work with the aim to explore and identify a precise, secure and efficient model of collaboration among the production resources distributed within the shop floor. This document illustrates main outcomes of the research, focusing mainly on the Virtual Integrative Manufacturing Framework for resources Interaction (VICKI), a potential reference architecture for a middleware application enabling semantic-based cooperation among manufacturing resources. Specifically, this framework provides a technological and service-oriented infrastructure offering an event-driven mechanism that dynamically propagates the changing factors to the interested devices. The proposed system supports the coexistence and combination of physical components and their virtual counterparts in a network of interacting collaborative elements in constant connection, thus allowing to bring back the manufacturing system to a cooperative Cyber-physical Production System (CPPS) (Monostori, 2014). Within this network, the information coming from the productive chain can be promptly and seamlessly shared, distributed and understood by any actor operating in such a context. In order to overcome the problem of the limited interoperability among the connected resources, the framework leverages a common data model based on the Semantic Web technologies (SWT) (Berners-Lee, Hendler, & Lassila, 2001). The model provides a shared understanding on the vocabulary adopted by the distributed resources during their knowledge exchange. In this way, this model allows to integrate heterogeneous data streams into a coherent semantically enriched scheme that represents the evolution of the factory objects, their context and their smart reactions to all kind of situations. The semantic model is also machine-interpretable and re-usable. In addition to modeling, the virtualization of the overall manufacturing system is empowered by the adoption of an agent-based modeling, which contributes to hide and abstract the control functions complexity of the cooperating entities, thus providing the foundations to achieve a flexible and reconfigurable system. Finally, in order to mitigate the risk of internal and external attacks against the proposed infrastructure, it is explored the potential of a strategy based on the analysis and assessment of the manufacturing systems cyber-security aspects integrated into the context of the organization\u2019s business model. To test and validate the proposed framework, a demonstration scenarios has been identified, which are thought to represent different significant case studies of the factory\u2019s life cycle. To prove the correctness of the approach, the validation of an instance of the framework is carried out within a real case study. Moreover, as for data intensive systems such as the manufacturing system, the quality of service (QoS) requirements in terms of latency, efficiency, and scalability are stringent, an evaluation of these requirements is needed in a real case study by means of a defined benchmark, thus showing the impact of the data storage, of the connected resources and of their requests

Adaptive Middleware for Resource-Constrained Mobile Ad Hoc and Wireless Sensor Networks

Mobile ad hoc networks: MANETs) and wireless sensor networks: WSNs) are two recently-developed technologies that uniquely function without fixed infrastructure support, and sense at scales, resolutions, and durations previously not possible. While both offer great potential in many applications, developing software for these types of networks is extremely difficult, preventing their wide-spread use. Three primary challenges are: 1) the high level of dynamics within the network in terms of changing wireless links and node hardware configurations,: 2) the wide variety of hardware present in these networks, and: 3) the extremely limited computational and energy resources available. Until now, the burden of handling these issues was put on the software application developer. This dissertation presents three novel programming models and middleware systems that address these challenges: Limone, Agilla, and Servilla. Limone reliably handles high levels of dynamics within MANETs. It does this through lightweight coordination primitives that make minimal assumptions about network connectivity. Agilla enables self-adaptive WSN applications via the integration of mobile agent and tuple space programming models, which is critical given the continuously changing network. It is the first system to successfully demonstrate the feasibility of using mobile agents and tuple spaces within WSNs. Servilla addresses the challenges that arise from WSN hardware heterogeneity using principles of Service-Oriented Computing: SOC). It is the first system to successfully implement the entire SOC model within WSNs and uniquely tailors it to the WSN domain by making it energy-aware and adaptive. The efficacies of the above three systems are demonstrated through implementation, micro-benchmarks, and the evaluation of several real-world applications including Universal Remote, Fire Detection and Tracking, Structural Health Monitoring, and Medical Patient Monitoring

A distributed architecture for unmanned aerial systems based on publish/subscribe messaging and simultaneous localisation and mapping (SLAM) testbed

A dissertation submitted in fulfilment for the degree of Master of Science. School of Computational and Applied Mathematics, University of the Witwatersrand, Johannesburg, South Africa, November 2017The increased capabilities and lower cost of Micro Aerial Vehicles (MAVs) unveil big opportunities for a rapidly growing number of civilian and commercial applications. Some missions require direct control using a receiver in a point-to-point connection, involving one or very few MAVs. An alternative class of mission is remotely controlled, with the control of the drone automated to a certain extent using mission planning software and autopilot systems. For most emerging missions, there is a need for more autonomous, cooperative control of MAVs, as well as more complex data processing from sensors like cameras and laser scanners. In the last decade, this has given rise to an extensive research from both academia and industry. This research direction applies robotics and computer vision concepts to Unmanned Aerial Systems (UASs). However, UASs are often designed for specific hardware and software, thus providing limited integration, interoperability and re-usability across different missions. In addition, there are numerous open issues related to UAS command, control and communication(C3), and multi-MAVs. We argue and elaborate throughout this dissertation that some of the recent standardbased publish/subscribe communication protocols can solve many of these challenges and meet the non-functional requirements of MAV robotics applications. This dissertation assesses the MQTT, DDS and TCPROS protocols in a distributed architecture of a UAS control system and Ground Control Station software. While TCPROS has been the leading robotics communication transport for ROS applications, MQTT and DDS are lightweight enough to be used for data exchange between distributed systems of aerial robots. Furthermore, MQTT and DDS are based on industry standards to foster communication interoperability of “things”. Both protocols have been extensively presented to address many of today’s needs related to networks based on the internet of things (IoT). For example, MQTT has been used to exchange data with space probes, whereas DDS was employed for aerospace defence and applications of smart cities. We designed and implemented a distributed UAS architecture based on each publish/subscribe protocol TCPROS, MQTT and DDS. The proposed communication systems were tested with a vision-based Simultaneous Localisation and Mapping (SLAM) system involving three Parrot AR Drone2 MAVs. Within the context of this study, MQTT and DDS messaging frameworks serve the purpose of abstracting UAS complexity and heterogeneity. Additionally, these protocols are expected to provide low-latency communication and scale up to meet the requirements of real-time remote sensing applications. The most important contribution of this work is the implementation of a complete distributed communication architecture for multi-MAVs. Furthermore, we assess the viability of this architecture and benchmark the performance of the protocols in relation to an autonomous quadcopter navigation testbed composed of a SLAM algorithm, an extended Kalman filter and a PID controller.XL201

NeBula: TEAM CoSTAR’s robotic autonomy solution that won phase II of DARPA subterranean challenge

This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge. Specifically, it presents the techniques utilized within the Tunnel (2019) and Urban (2020) competitions, where CoSTAR achieved second and first place, respectively. We also discuss CoSTAR’s demonstrations in Martian-analog surface and subsurface (lava tubes) exploration. The paper introduces our autonomy solution, referred to as NeBula (Networked Belief-aware Perceptual Autonomy). NeBula is an uncertainty-aware framework that aims at enabling resilient and modular autonomy solutions by performing reasoning and decision making in the belief space (space of probability distributions over the robot and world states). We discuss various components of the NeBula framework, including (i) geometric and semantic environment mapping, (ii) a multi-modal positioning system, (iii) traversability analysis and local planning, (iv) global motion planning and exploration behavior, (v) risk-aware mission planning, (vi) networking and decentralized reasoning, and (vii) learning-enabled adaptation. We discuss the performance of NeBula on several robot types (e.g., wheeled, legged, flying), in various environments. We discuss the specific results and lessons learned from fielding this solution in the challenging courses of the DARPA Subterranean Challenge competition.Peer ReviewedAgha, A., Otsu, K., Morrell, B., Fan, D. D., Thakker, R., Santamaria-Navarro, A., Kim, S.-K., Bouman, A., Lei, X., Edlund, J., Ginting, M. F., Ebadi, K., Anderson, M., Pailevanian, T., Terry, E., Wolf, M., Tagliabue, A., Vaquero, T. S., Palieri, M., Tepsuporn, S., Chang, Y., Kalantari, A., Chavez, F., Lopez, B., Funabiki, N., Miles, G., Touma, T., Buscicchio, A., Tordesillas, J., Alatur, N., Nash, J., Walsh, W., Jung, S., Lee, H., Kanellakis, C., Mayo, J., Harper, S., Kaufmann, M., Dixit, A., Correa, G. J., Lee, C., Gao, J., Merewether, G., Maldonado-Contreras, J., Salhotra, G., Da Silva, M. S., Ramtoula, B., Fakoorian, S., Hatteland, A., Kim, T., Bartlett, T., Stephens, A., Kim, L., Bergh, C., Heiden, E., Lew, T., Cauligi, A., Heywood, T., Kramer, A., Leopold, H. A., Melikyan, H., Choi, H. C., Daftry, S., Toupet, O., Wee, I., Thakur, A., Feras, M., Beltrame, G., Nikolakopoulos, G., Shim, D., Carlone, L., & Burdick, JPostprint (published version