Softmax exploration strategies for multiobjective reinforcement learning

Despite growing interest over recent years in applying reinforcement learning to multiobjective problems, there has been little research into the applicability and effectiveness of exploration strategies within the multiobjective context. This work considers several widely-used approaches to exploration from the single-objective reinforcement learning literature, and examines their incorporation into multiobjective Q-learning. In particular this paper proposes two novel approaches which extend the softmax operator to work with vector-valued rewards. The performance of these exploration strategies is evaluated across a set of benchmark environments. Issues arising from the multiobjective formulation of these benchmarks which impact on the performance of the exploration strategies are identified. It is shown that of the techniques considered, the combination of the novel softmax–epsilon exploration with optimistic initialisation provides the most effective trade-off between exploration and exploitation

Using MapReduce Streaming for Distributed Life Simulation on the Cloud

Distributed software simulations are indispensable in the study of large-scale life models but often require the use of technically complex lower-level distributed computing frameworks, such as MPI. We propose to overcome the complexity challenge by applying the emerging MapReduce (MR) model to distributed life simulations and by running such simulations on the cloud. Technically, we design optimized MR streaming algorithms for discrete and continuous versions of Conway’s life according to a general MR streaming pattern. We chose life because it is simple enough as a testbed for MR’s applicability to a-life simulations and general enough to make our results applicable to various lattice-based a-life models. We implement and empirically evaluate our algorithms’ performance on Amazon’s Elastic MR cloud. Our experiments demonstrate that a single MR optimization technique called strip partitioning can reduce the execution time of continuous life simulations by 64%. To the best of our knowledge, we are the first to propose and evaluate MR streaming algorithms for lattice-based simulations. Our algorithms can serve as prototypes in the development of novel MR simulation algorithms for large-scale lattice-based a-life models.https://digitalcommons.chapman.edu/scs_books/1014/thumbnail.jp

Algorithms for Geometric Optimization and Enrichment in Industrialized Building Construction

The burgeoning use of industrialized building construction, coupled with advances in digital technologies, is unlocking new opportunities to improve the status quo of construction projects being over-budget, delayed and having undesirable quality. Yet there are still several objective barriers that need to be overcome in order to fully realize the full potential of these innovations. Analysis of literature and examples from industry reveal the following notable barriers: (1) geometric optimization methods need to be developed for the stricter dimensional requirements in industrialized construction, (2) methods are needed to preserve model semantics during the process of generating an updated as-built model, (3) semantic enrichment methods are required for the end-of-life stage of industrialized buildings, and (4) there is a need to develop pragmatic approaches for algorithms to ensure they achieve required computational efficiency. The common thread across these examples is the need for developing algorithms to optimize and enrich geometric models. To date, a comprehensive approach paired with pragmatic solutions remains elusive. This research fills this gap by presenting a new approach for algorithm development along with pragmatic implementations for the industrialized building construction sector. Computational algorithms are effective for driving the design, analysis, and optimization of geometric models. As such, this thesis develops new computational algorithms for design, fabrication and assembly, onsite construction, and end-of-life stages of industrialized buildings. A common theme throughout this work is the development and comparison of varied algorithmic approaches (i.e., exact vs. approximate solutions) to see which is optimal for a given process. This is implemented in the following ways. First, a probabilistic method is used to simulate the accumulation of dimensional tolerances in order to optimize geometric models during design. Second, a series of exact and approximate algorithms are used to optimize the topology of 2D panelized assemblies to minimize material use during fabrication and assembly. Third, a new approach to automatically update geometric models is developed whereby initial model semantics are preserved during the process of generating an as-built model. Finally, a series of algorithms are developed to semantically enrich geometric models to enable industrialized buildings to be disassembled and reused. The developments made in this research form a rational and pragmatic approach to addressing the existing challenges faced in industrialized building construction. Such developments are shown not only to be effective in improving the status quo in the industry (i.e., improving cost, reducing project duration, and improving quality), but also for facilitating continuous innovation in construction. By way of assessing the potential impact of this work, the proposed algorithms can reduce rework risk during fabrication and assembly (65% rework reduction in the case study for the new tolerance simulation algorithm), reduce waste during manufacturing (11% waste reduction in the case study for the new panel unfolding and nesting algorithms), improve accuracy and automation of as-built model generation (model error reduction from 50.4 mm to 5.7 mm in the case study for the new parametric BIM updating algorithms), reduce lifecycle cost for adapting industrialized buildings (15% reduction in capital costs in the computational building configurator) and reducing lifecycle impacts for reusing structural systems from industrialized buildings (between 54% to 95% reduction in average lifecycle impacts for the approach illustrated in Appendix B). From a computational standpoint, the novelty of the algorithms developed in this research can be described as follows. Complex geometric processes can be codified solely on the innate properties of geometry – that is, by parameterizing geometry and using methods such as combinatorial optimization, topology can be optimized and semantics can be automatically enriched for building assemblies. Employing the use of functional discretization (whereby continuous variable domains are converted into discrete variable domains) is shown to be highly effective for complex geometric optimization approaches. Finally, the algorithms encapsulate and balance the benefits posed by both parametric and non-parametric schemas, resulting in the ability to achieve both high representational accuracy and semantically rich information (which has previously not been achieved or demonstrated). In summary, this thesis makes several key improvements to industrialized building construction. One of the key findings is that rather than pre-emptively determining the best suited algorithm for a given process or problem, it is often more pragmatic to derive both an exact and approximate solution and then decide which is optimal to use for a given process. Generally, most tasks related to optimizing or enriching geometric models is best solved using approximate methods. To this end, this research presents a series of key techniques that can be followed to improve the temporal performance of algorithms. The new approach for developing computational algorithms and the pragmatic demonstrations for geometric optimization and enrichment are expected to bring the industry forward and solve many of the current barriers it faces

The 1995 Goddard Conference on Space Applications of Artificial Intelligence and Emerging Information Technologies

This publication comprises the papers presented at the 1995 Goddard Conference on Space Applications of Artificial Intelligence and Emerging Information Technologies held at the NASA/Goddard Space Flight Center, Greenbelt, Maryland, on May 9-11, 1995. The purpose of this annual conference is to provide a forum in which current research and development directed at space applications of artificial intelligence can be presented and discussed

Resource Allocation and Service Management in Next Generation 5G Wireless Networks

The accelerated evolution towards next generation networks is expected to dramatically increase mobile data traffic, posing challenging requirements for future radio cellular communications. User connections are multiplying, whilst data hungry content is dominating wireless services putting significant pressure on network's available spectrum. Ensuring energy-efficient and low latency transmissions, while maintaining advanced Quality of Service (QoS) and high standards of user experience are of profound importance in order to address diversifying user prerequisites and ensure superior and sustainable network performance. At the same time, the rise of 5G networks and the Internet of Things (IoT) evolution is transforming wireless infrastructure towards enhanced heterogeneity, multi-tier architectures and standards, as well as new disruptive telecommunication technologies. The above developments require a rethinking of how wireless networks are designed and operate, in conjunction with the need to understand more holistically how users interact with the network and with each other. In this dissertation, we tackle the problem of efficient resource allocation and service management in various network topologies under a user-centric approach. In the direction of ad-hoc and self-organizing networks where the decision making process lies at the user level, we develop a novel and generic enough framework capable of solving a wide array of problems with regards to resource distribution in an adaptable and multi-disciplinary manner. Aiming at maximizing user satisfaction and also achieve high performance - low power resource utilization, the theory of network utility maximization is adopted, with the examined problems being formulated as non-cooperative games. The considered games are solved via the principles of Game Theory and Optimization, while iterative and low complexity algorithms establish their convergence to steady operational outcomes, i.e., Nash Equilibrium points. This thesis consists a meaningful contribution to the current state of the art research in the field of wireless network optimization, by allowing users to control multiple degrees of freedom with regards to their transmission, considering mobile customers and their strategies as the key elements for the amelioration of network's performance, while also adopting novel technologies in the resource management problems. First, multi-variable resource allocation problems are studied for multi-tier architectures with the use of femtocells, addressing the topic of efficient power and/or rate control, while also the topic is examined in Visible Light Communication (VLC) networks under various access technologies. Next, the problem of customized resource pricing is considered as a separate and bounded resource to be optimized under distinct scenarios, which expresses users' willingness to pay instead of being commonly implemented by a central administrator in the form of penalties. The investigation is further expanded by examining the case of service provider selection in competitive telecommunication markets which aim to increase their market share by applying different pricing policies, while the users model the selection process by behaving as learning automata under a Machine Learning framework. Additionally, the problem of resource allocation is examined for heterogeneous services where users are enabled to dynamically pick the modules needed for their transmission based on their preferences, via the concept of Service Bundling. Moreover, in this thesis we examine the correlation of users' energy requirements with their transmission needs, by allowing the adaptive energy harvesting to reflect the consumed power in the subsequent information transmission in Wireless Powered Communication Networks (WPCNs). Furthermore, in this thesis a fresh perspective with respect to resource allocation is provided assuming real life conditions, by modeling user behavior under Prospect Theory. Subjectivity in decisions of users is introduced in situations of high uncertainty in a more pragmatic manner compared to the literature, where they behave as blind utility maximizers. In addition, network spectrum is considered as a fragile resource which might collapse if over-exploited under the principles of the Tragedy of the Commons, allowing hence users to sense risk and redefine their strategies accordingly. The above framework is applied in different cases where users have to select between a safe and a common pool of resources (CPR) i.e., licensed and unlicensed bands, different access technologies, etc., while also the impact of pricing in protecting resource fragility is studied. Additionally, the above resource allocation problems are expanded in Public Safety Networks (PSNs) assisted by Unmanned Aerial Vehicles (UAVs), while also aspects related to network security against malign user behaviors are examined. Finally, all the above problems are thoroughly evaluated and tested via a series of arithmetic simulations with regards to the main characteristics of their operation, as well as against other approaches from the literature. In each case, important performance gains are identified with respect to the overall energy savings and increased spectrum utilization, while also the advantages of the proposed framework are mirrored in the improvement of the satisfaction and the superior Quality of Service of each user within the network. Lastly, the flexibility and scalability of this work allow for interesting applications in other domains related to resource allocation in wireless networks and beyond

PSA 2018

These preprints were automatically compiled into a PDF from the collection of papers deposited in PhilSci-Archive in conjunction with the PSA 2018