SCALING REINFORCEMENT LEARNING THROUGH FEUDAL MULTI-AGENT HIERARCHY

Militaries conduct wargames for training, planning, and research purposes. Artificial intelligence (AI) can improve military wargaming by reducing costs, speeding up the decision-making process, and offering new insights. Previous researchers explored using reinforcement learning (RL) for wargaming based on the successful use of RL for other human competitive games. While previous research has demonstrated that an RL agent can generate combat behavior, those experiments have been limited to small-scale wargames. This thesis investigates the feasibility and acceptability of -scaling hierarchical reinforcement learning (HRL) to support integrating AI into large military wargames. Additionally, this thesis also investigates potential complications that arise when replacing the opposing force with an intelligent agent by exploring the ways in which an intelligent agent can cause a wargame to fail. The resources required to train a feudal multi-agent hierarchy (FMH) and a standard RL agent and their effectiveness are compared in increasingly complicated wargames. While FMH fails to demonstrate the performance required for large wargames, it offers insight for future HRL research. Finally, the Department of Defense verification, validation, and accreditation process is proposed as a method to ensure that any future AI application applied to wargames are suitable.Lieutenant Colonel, United States ArmyApproved for public release. Distribution is unlimited

Integrating Blockchain and Fog Computing Technologies for Efficient Privacy-preserving Systems

This PhD dissertation concludes a three-year long research journey on the integration of Fog Computing and Blockchain technologies. The main aim of such integration is to address the challenges of each of these technologies, by integrating it with the other. Blockchain technology (BC) is a distributed ledger technology in the form of a distributed transactional database, secured by cryptography, and governed by a consensus mechanism. It was initially proposed for decentralized cryptocurrency applications with practically proven high robustness. Fog Computing (FC) is a geographically distributed computing architecture, in which various heterogeneous devices at the edge of network are ubiquitously connected to collaboratively provide elastic computation services. FC provides enhanced services closer to end-users in terms of time, energy, and network load. The integration of FC with BC can result in more efficient services, in terms of latency and privacy, mostly required by Internet of Things systems

Accurate,robust and harmonized implementation of morpho-functional imaging in treatment planning for personalized radiotherapy

In this work we present a methodology able to use harmonized PET/CT imaging in dose painting by number (DPBN) approach by means of a robust and accurate treatment planning system. Image processing and treatment planning were performed by using a Matlab-based platform, called CARMEN, in which a full Monte Carlo simulation is included. Linear programming formulation was developed for a voxel-by-voxel robust optimization and a specific direct aperture optimization was designed for an efficient adaptive radiotherapy implementation. DPBN approach with our methodology was tested to reduce the uncertainties associated with both, the absolute value and the relative value of the information in the functional image. For the same H&N case, a single robust treatment was planned for dose prescription maps corresponding to standardized uptake value distributions from two different image reconstruction protocols: One to fulfill EARL accreditation for harmonization of [18F]FDG PET/CT image, and the other one to use the highest available spatial resolution. Also, a robust treatment was planned to fulfill dose prescription maps corresponding to both approaches, the dose painting by contour based on volumes and our voxel-by-voxel DPBN. Adaptive planning was also carried out to check the suitability of our proposal. Different plans showed robustness to cover a range of scenarios for implementation of harmonizing strategies by using the highest available resolution. Also, robustness associated to discretization level of dose prescription according to the use of contours or numbers was achieved. All plans showed excellent quality index histogram and quality factors below 2%. Efficient solution for adaptive radiotherapy based directly on changes in functional image was obtained. We proved that by using voxel-by-voxel DPBN approach it is possible to overcome typical drawbacks linked to PET/CT images, providing to the clinical specialist confidence enough for routinely implementation of functional imaging for personalized radiotherapy.Junta de Andalucía (FISEVI, reference project CTS 2482)European Regional Development Fund (FEDER

Context-aware multi-attribute decision multi - attribute decision making for radio access technology selection in ultra dense network

Ultra Dense Network (UDN) is the extreme densification of heterogeneous Radio Access Technology (RAT) that is deployed closely in coordinated or uncoordinated manner. The densification of RAT forms an overlapping zone of signal coverage leading to the frequent service handovers among the RAT, thus degrading overall system performance. The current RAT selection approach is biased towards network-centric criteria pertaining to signal strength. However, the paradigm shift from network-centric to user-centric approach necessitates a multi-criteria selection process, with methodology relating to both network and user preferences in the context of future generation networks. Hence, an effective selection approach is required to avoid unnecessary handovers in RAT. The main aim of this study is to propose the Context-aware Multiattribute decision making for RAT (CMRAT) selection for investigating the need to choose a new RAT and further determine the best amongst the available methods. The CMRAT consists of two mechanisms, namely the Context-aware Analytical Hierarchy Process (CAHP) and Context-aware Technique for Order Preference by Similarity to an Ideal Solution (CTOPSIS). The CAHP mechanism measures the need to switch from the current RAT, while CTOPSIS aids in decision making to choose the best target RAT. A series of experimental studies were conducted to validate the effectiveness of CMRAT for achieving improved system performance. The investigation utilises shopping mall and urban dense network scenarios to evaluate the performance of RAT selection through simulation. The findings demonstrated that the CMRAT approach reduces delay and the number of handovers leading to an improvement of throughput and packet delivery ratio when compared to that of the commonly used A2A4-RSRQ approach. The CMRAT approach is effective in the RAT selection within UDN environment, thus supporting heterogeneous RAT deployment in future 5G networks. With context-aware selection, the user-centric feature is also emphasized

The need of diagrams based on Toulmin schema application: an aeronautical case study

In this article, Justification Diagrams are introduced for structuring evidence to support conclusions that are reached from results of simulation studies. An industrial application is used to illustrate the use of the Justification Diagrams. Adapted from the Toulmin schema, the aim of Justification Diagram is to define a comprehensive, auditable and shareable notation to explain the results, the input data, the assumptions made and the techniques applied, to construct a cogent conclusion. Further, the Justification Diagrams provide a visual representation of the argument that aims to corroborate the specified claims, or conclusions. A large part of this work is based on the application of the Justification Diagrams in the context of the European project, TOICA. The Justification Diagrams were used to structure all justifications that would be needed to convince an authority that a simulation process, and the associated results, upheld a particular conclusion. These diagrams are built concurrently in a product development process that accompanies the various stages of Verification and Validation (V&V) and where, for each design stage of V&V, argumentation is constructed by aggregating evidence and documents produced at this design stage

Flow Cytometric Analyses of Lymphocyte Markers in Immune Oncology: A Comprehensive Guidance for Validation Practice According to Laws and Standards

Many anticancer therapies such as antibody-based therapies, cellular therapeutics (e.g., genetically modified cells, regulators of cytokine signaling, and signal transduction), and other biologically tailored interventions strongly influence the immune system and require tools for research, diagnosis, and monitoring. In flow cytometry, in vitro diagnostic (IVD) test kits that have been compiled and validated by the manufacturer are not available for all requirements. Laboratories are therefore usually dependent onmodifying commercially available assays or, most often, developing them to meet clinical needs. However, both variants must then undergo full validation to fulfill the IVD regulatory requirements. Flow cytometric immunophenotyping is a multiparametric analysis of parameters, some of which have to be repeatedly adjusted; that must be considered when developing specific antibody panels. Careful adjustments of general rules are required to meet legal and regulatory requirements in the analysis of these assays. Here, we describe the relevant regulatory framework for flow cytometry-based assays and describe methods for the introduction of new antibody combinations into routine work including development of performance specifications, validation, and statistical methodology for design and analysis of the experiments. The aim is to increase reliability, efficiency, and auditability after the introduction of in-house-developed flow cytometry assays