Evolutionary Multiagent Transfer Learning With Model-Based Opponent Behavior Prediction

This article embarks a study on multiagent transfer learning (TL) for addressing the specific challenges that arise in complex multiagent systems where agents have different or even competing objectives. Specifically, beyond the essential backbone of a state-of-the-art evolutionary TL framework (eTL), this article presents the novel TL framework with prediction (eTL-P) as an upgrade over existing eTL to endow agents with abilities to interact with their opponents effectively by building candidate models and accordingly predicting their behavioral strategies. To reduce the complexity of candidate models, eTL-P constructs a monotone submodular function, which facilitates to select Top-K models from all available candidate models based on their representativeness in terms of behavioral coverage as well as reward diversity. eTL-P also integrates social selection mechanisms for agents to identify their better-performing partners, thus improving their learning performance and reducing the complexity of behavior prediction by reusing useful knowledge with respect to their partners' mind universes. Experiments based on a partner-opponent minefield navigation task (PO-MNT) have shown that eTL-P exhibits the superiority in achieving higher learning capability and efficiency of multiple agents when compared to the state-of-the-art multiagent TL approaches

Variants of CTGF are associated with hepatic fibrosis in Chinese, Sudanese, and Brazilians infected with Schistosomes

Abnormal fibrosis occurs during chronic hepatic inflammations and is the principal cause of death in hepatitis C virus and schistosome infections. Hepatic fibrosis (HF) may develop either slowly or rapidly in schistosome-infected subjects. This depends, in part, on a major genetic control exerted by genes of chromosome 6q23. A gene (connective tissue growth factor [CTGF]) is located in that region that encodes a strongly fibrogenic molecule. We show that the single nucleotide polymorphism (SNP) rs9402373 that lies close to CTGF is associated with severe HF (P = 2 × 10−6; odds ratio [OR] = 2.01; confidence interval of OR [CI] = 1.51–2.7) in two Chinese samples, in Sudanese, and in Brazilians infected with either Schistosoma japonicum or S. mansoni. Furthermore, SNP rs12526196, also located close to CTGF, is independently associated with severe fibrosis (P = 6 × 10−4; OR = 1.94; CI = 1.32–2.82) in the Chinese and Sudanese subjects. Both variants affect nuclear factor binding and may alter gene transcription or transcript stability. The identified variants may be valuable markers for the prediction of disease progression, and identify a critical step in the development of HF that could be a target for chemotherapy

Identification of avian polyomavirus and its pathogenicity to SPF chickens

The research aimed to study an Avian polyomavirus strain that was isolated in Shandong, China. To study the pathogenicity of APV in SPF chickens, and provide references for epidemiological research and disease prevention and control of APV. The genetic characterization of APV strain (termed APV-20) was analyzed and the pathogenicity of APV was investigated from two aspects: different age SPF chickens, and different infection doses. The results revealed that the APV-20 exhibits a nucleotide homology of 99% with the other three APV strains, and the evolution of APV In China was slow. In addition, the APV-20 infection in chickens caused depression, drowsiness, clustering, and fluffy feathers, but no deaths occurred in the infected chickens. The main manifestations of necropsy, and Hematoxylin and Eosin staining (HE) showed that one-day-old SPF chickens were the most susceptible, and there was a positive correlation between viral load and infection dose in the same tissue. This study showed that SPF chickens were susceptible to APV, and an experimental animal model was established. This study can provide a reference for the pathogenic mechanism of immune prevention and control of APV

Evolutionary transfer learning for complex multi-agent reinforcement learning systems

Multi-agent systems (MAS) are computerized systems composing of multiple interacting and autonomous agents within a common environment of interest for problem-solving. Since the behavioral strategies of agents in conventional MAS are usually manually defined in advance, the development of intelligent agents that are capable of adapting to the dynamic environment has attracted increasing attention in the past decades. In the past decades, reinforcement learning (RL) has been introduced into MAS as the learning paradigm of individual agents through trial-and-error interactions in a dynamic environment. Due to its generality and simplicity of use, the study of RL is rapidly expanding and a wide variety of approaches has already been proposed to exploit its benefits and applicabilities. A more recent machine learning paradigm of transfer learning (TL) has been proposed as an approach of leveraging valuable knowledge from related and well studied problem domains to enhance problem-solving in the target domains of interest. TL has been successfully used for enhancing RL tasks and many TL methodologies, such as instance or feature transfer, have been explored. Recently, the research on TL has been considered for enhancing multi-agent RL methods. In the context of computational intelligence, the science of memetics, especially memetic computation, has become an increasing topic of research. Many of the existing work in memetic computation has been established as an extension of the classical evolutionary algorithms where a meme is perceived as a form of individual learning procedure or local search operator in population based search algorithms. However, recent research shows memetic computation can become more meme-centric wherein memes transpire as units of domain information or knowledge building blocks useful for problem-solving. The intrinsic parallelism of natural evolution in such meme-centric computing may derive its strength from the simultaneous explorations of differing regions of a common problem domain and exploitative social interactions between the multiple agent learners. This dissertation hence seeks for the new study on a meme centric evolutionary knowledge transfer paradigm for problem-solving of multi-agent reinforcement learning systems. Specifically, this thesis presents an evolutionary transfer learning framework (eTL) which comprises a series of meme-inspired evolutionary knowledge representation and transfer mechanisms. The proposed framework serves to enhance the learning capabilities while addresses the limitations (e.g. blind reliance) of existing knowledge transfer frameworks. Subsequently, a novel transfer learning framework with predictive capabilities (eTL-P) is proposed to cope with the challenges arising in complex multi-agent systems where agents have differing or even competitive objectives. eTL-P endows agents with abilities to interact with competitive opponents, modeling their opponents, and hence predicting their behaviors accordingly. Further, to reduce the complexity of the opponent candidate models, a Top-K model selection method is proposed for selecting a smaller yet remarkably representative candidate model set from the entire model space. Last but not least, a summary of the future study on the evolutionary knowledge transfer paradigm is presented.Doctor of Philosophy (IGS

Fabricating Homogeneous FeCoCrNi High-Entropy Alloys via SLM In Situ Alloying

Selective laser melting (SLM) in situ alloying is an effective way to design and fabricate novel materials in which the elemental powder is adopted as the raw material and micro-areas of elemental powder blend are alloyed synchronously in the forming process of selective laser melting (SLM). The pre-alloying process of preparation of raw material powder can be left out, and a batch of bulk samples can be prepared via the technology combined with quantitative powder mixing and feeding. The technique can be applied to high-throughput sample preparation to efficiently obtain a microstructure and performance data for material design. In the present work, bulk equiatomic FeCoCrNi high-entropy alloys with different processing parameters were fabricated via laser in situ alloying. Finite element simulation and CALPHAD calculation were used to determine the appropriate SLM and post-heating parameters. SEM (scanning electron microscope), EDS (energy dispersive spectroscopy), XRD (X-ray diffraction), and mechanical testing were used to characterize the composition, microstructure, and mechanical properties of as-printed and post-heat-treated samples. The experimental results show that the composition deviation of laser in situ alloying samples could be controlled within 20 wt %. The crystal structure of as-printed samples is a single-phase face-centered cubic (FCC), which is the same as those prepared by the traditional method. The mechanical properties of the samples prepared by laser in situ alloying with elemental powder blend are comparable to those prepared by pre-alloying powder and much higher than those prepared by the traditional method (arc melting). As-printed samples can get a homogeneous microstructure under the optimal laser in situ alloying process combined with post-heat treatment at 1200 °C for 20 h

Budgeted Sequence Submodular Maximization

The problem of selecting a sequence of items that maximizes a given submodular function appears in many real-world applications. Existing study on the problem only considers uniform costs over items, but non-uniform costs on items are more general. Taking this cue, we study the problem of budgeted sequence submodular maximization (BSSM), which introduces non-uniform costs of items into the sequence selection. This problem can be found in a number of applications such as movie recommendation, course sequence design and so on. Non-uniform costs on items significantly increase the solution complexity and we prove that BSSM is NP-hard. To solve the problem, we first propose a greedy algorithm GBM with an error bound. We also design an anytime algorithm POBM based on Pareto optimization to improve the quality of solutions. Moreover, we prove that POBM can obtain approximate solutions in expected polynomial running time, and converges faster than a state-of-the-art algorithm POSEQSEL for sequence submodular maximization with cardinality constraints. We further introduce optimizations to speed up POBM. Experimental results on both synthetic and real-world datasets demonstrate the performance of our new algorithms

Half a dozen real-world applications of evolutionary multitasking, and more

Until recently, the potential to transfer evolved skills across distinct optimization problem instances (or tasks) was seldom explored in evolutionary computation. The concept of evolutionary multitasking (EMT) fills this gap. It unlocks a population's implicit parallelism to jointly solve a set of tasks, hence creating avenues for skills transfer between them. Despite it being early days, the idea of EMT has begun to show promise in a range of real-world applications. In the backdrop of recent advances, the contribution of this paper is twofold. First, a review of several application-oriented explorations of EMT in the literature is presented; the works are assimilated into half a dozen broad categories according to their respective application domains. Each of these six categories elaborates fundamental motivations to multitask, and contains a representative experimental study (referred from the literature). Second, a set of recipes is provided showing how problem formulations of general interest, those that cut across different disciplines, could be transformed in the new light of EMT. Our discussions emphasize the many practical use-cases of EMT, and are intended to spark future research towards crafting novel algorithms for real-world deployment.Agency for Science, Technology and Research (A*STAR)Abhishek Gupta was supported by the A*STAR AI3 HTPO seed grant C211118016 on Upside-Down Multi-Objective Bayesian Optimization for Few-Shot Design. The work was also supported in part by the Cyber-Physical Production System Research Program, under the IAF-PP Grant A19C1a0018. Yaqing Hou was supported by the National Natural Science Foundation of China under Grant 61906032