Learning Meta Representations for Agents in Multi-Agent Reinforcement Learning

In multi-agent reinforcement learning, the behaviors that agents learn in a single Markov Game (MG) are typically confined to the given agent number (i.e., population size). Every single MG induced by varying population sizes may possess distinct optimal joint strategies and game-specific knowledge, which are modeled independently in modern multi-agent algorithms. In this work, we focus on creating agents that generalize across population-varying MGs. Instead of learning a unimodal policy, each agent learns a policy set that is formed by effective strategies across a variety of games. We propose Meta Representations for Agents (MRA) that explicitly models the game-common and game-specific strategic knowledge. By representing the policy sets with multi-modal latent policies, the common strategic knowledge and diverse strategic modes are discovered with an iterative optimization procedure. We prove that as an approximation to a constrained mutual information maximization objective, the learned policies can reach Nash Equilibrium in every evaluation MG under the assumption of Lipschitz game on a sufficiently large latent space. When deploying it at practical latent models with limited size, fast adaptation can be achieved by leveraging the first-order gradient information. Extensive experiments show the effectiveness of MRA on both training performance and generalization ability in hard and unseen games

Effects of KN-42 on Growth Performance, Diarrhea and Faecal Bacterial Flora of Weaned Piglets

This research focused on the effects of different doses of Bacillus subtilis KN-42 on the growth performance, diarrhea incidence, faecal bacterial flora, and the relative number of Lactobacillus and Escherichia coli in faeces of weaned piglets to determine whether the strain can serve as a candidate antimicrobial growth promoter. A total of 360 piglets (initial body weight 7.14±0.63 kg) weaned at 26±2 days of age were randomly allotted to 5 treatment groups (4 pens per treatment with 18 pigs per pen) for a 28-day trial. Dietary treatments were basal diet without any antimicrobial (negative control; NC), basal diet supplemented with 120 mg/kg feed of neomycin sulfate (positive control; PC) and basal diet supplemented with 2×109 (L), 4×109 (M) and 20×109 (H) CFU/kg feed of B. subtilis KN-42. During the overall period, average daily gain and feed efficiency of piglets were higher in groups PC, M, and H than those in group NC (p<0.05), and all probiotics and antibiotics groups had a lower diarrhea index than group NC (p<0.05). The 16S rDNA gene-based methods were used to analyze faecal bacterial flora on day 28 of experiment. The result of denaturing gradient gel electrophoresis analysis showed that supplementation of B. subtilis KN-42 to the diet changed the bacterial communities, with a higher bacterial diversity and band number in group M than in the other four groups. Real-time polymerase chain reaction analysis showed that the relative number of Lactobacillus were higher in groups PC and H than in group NC (p<0.05), and the supplemented B. subtilis KN-42 to the diet also reduced the relative number of E. coli (p<0.05). These results suggest that dietary addition of B. subtilis KN-42 can improve the growth performance and gastrointestinal health of piglets

Isolation and characterization of bacterial cellulose produced from soybean whey and soybean hydrolyzate

Abstract Soybean whey and soybean hydrolyzate can be used for the biotechnological production of high-value products. Herein, we isolate soybean whey (SW)-and soybean hydrolyzate (SH)-derived bacterial cellulose (BC, produced by kombucha) and characterize it by a range of instrumental techniques to reveal differences in micromorphology, crystallinity, and themal behavior. Studies have shown that the amounts of wet state BC produced from HS, SW and SH was 181 g/L, 47 g/L and 83 g/L, respectively. The instrumental analysis of BC, included SEM, AFM, FT-IR, XRD and TGA. It is shown that the FT-IR spectra of BC have a similar character, but we found differences in the micromorphology,crystallinity and thermal temperature of BC. The minimum average widths of the fibers produced from SH medium was 100 ± 29 nm. The CrI values of BC produced from SH medium was 61.8%. The maximum thermal degradation rate temperature of BC produced from SW medium was 355.73 °C. The combined results demonstrate that soybean industrial waste can be used as a cost-effective raw material for BC production

Frost resistance and life prediction of recycled brick aggregate concrete with waste polypropylene fiber

Due to recycled aggregate concrete technology, sustainable resource utilization can be achieved, but the weak frost resistance of this type of concrete affects its application in cold regions. Using waste polypropylene fibers as reinforcing materials can improve the mechanical properties and durability of concrete. This study explores the influence of waste polypropylene fiber on the frost resistance durability and microstructure of recycled brick aggregate (RA) concrete. The results show that with the increase in freeze–thaw cycles, the mass of the concrete first increases and then decreases, while its relative dynamic elastic modulus and compressive strength gradually decrease. After 60 freeze–thaw cycles, the maximum mass loss, maximum relative dynamic elastic modulus loss, and maximum compressive strength loss of the RA concrete are 1.73, 45.1, and 73.7%, respectively. Waste fiber (WF) can improve the frost resistance of concrete, as demonstrated by the obvious reduction in mass loss, relative dynamic elasticity modulus loss, and compressive strength loss, which are 0.11, 33.0, and 64.0%, respectively, after 60 freeze–thaw cycles. The action mechanism of WF on the frost resistance of RA concrete is revealed, and the life prediction model of RA concrete with WF under freeze–thaw conditions is established

Identification of band fragments in DGGE gels (Fig. 1A).

<p>* Bands are numbered according to <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0116635#pone.0116635.g001" target="_blank">Fig. 1A</a>.</p><p>^◆Identity represents the sequence identity (%) compared with that in the GenBank database.</p><p>Identification of band fragments in DGGE gels (<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0116635#pone.0116635.g001" target="_blank">Fig. 1A</a>).</p

<i>Lactobacillus</i> community of weaned piglets fed with neomycin or <i>E. faecalis</i>.

<p>(A) DGGE profiles of V3 region of the 16S rDNA gene fragments with the primes Lac1 and Lac2-GC. The denaturant gradient range is from 41% to 60%. Lanes N (negative control, basal diet); P (positive control, diet supplemented with neomycin); L, M, H (diets supplemented with probiotics 0.5×10⁹, 1.0×10⁹ and 2.5×10⁹ CFU/kg feed, respectively); (B) UPGMA cluster analysis of Dice similarity indices from DGGE profiles.</p

Bacterial diversity index calculated from the DGGE banding patterns (Fig. 1A).

<p>N (negative control, basal diet); P (positive control, diet supplemented with neomycin); L, M, H (diets supplemented with probiotics 0.5×10⁹, 1.0×10⁹ and 2.5×10⁹ CFU/kg feed, respectively);</p><p>*1/D, the reciprocal of Simpson diversity index.</p><p>Bacterial diversity index calculated from the DGGE banding patterns (<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0116635#pone.0116635.g001" target="_blank">Fig. 1A</a>).</p

Dietary <i>Enterococcus faecalis</i> LAB31 Improves Growth Performance, Reduces Diarrhea, and Increases Fecal <i>Lactobacillus</i> Number of Weaned Piglets

<div><p>Lactic acid bacteria (LAB) have been shown to enhance performance of weaned piglets. However, few studies have reported the addition of LAB <i>Enterococcus faecalis</i> as alternatives to growth promoting antibiotics for weaned piglets. This study evaluated the effects of dietary <i>E. faecalis</i> LAB31 on the growth performance, diarrhea incidence, blood parameters, fecal bacterial and <i>Lactobacillus</i> communities in weaned piglets. A total of 360 piglets weaned at 26 ± 2 days of age were randomly allotted to 5 groups (20 pens, with 4 pens for each group) for a trial of 28 days: group N (negative control, without antibiotics or probiotics); group P (Neomycin sulfate, 100 mg/kg feed); groups L, M and H (supplemented with <i>E. faecalis</i> LAB31 0.5×10⁹, 1.0×10⁹, and 2.5×10⁹ CFU/kg feed, respectively). Average daily gain and feed conversion efficiency were found to be higher in group H than in group N, and showed significant differences between group H and group P (<i>P₀</i> < 0.05). Furthermore, groups H and P had a lower diarrhea index than the other three groups (<i>P₀</i> < 0.05). Denaturing gradient gel electrophoresis (DGGE) showed that the application of probiotics to the diet changed the bacterial community, with a higher bacterial diversity in group M than in the other four groups. Real-time PCR revealed that the relative number of <i>Lactobacillus</i> increased by addition of probiotics, and was higher in group H than in group N (<i>P₀</i> < 0.05). However, group-specific PCR-DGGE showed no obvious difference among the five groups in <i>Lactobacillus</i> composition and diversity. Therefore, the dietary addition of <i>E. faecalis</i> LAB31 can improve growth performance, reduce diarrhea, and increase the relative number of <i>Lactobacillus</i> in feces of weaned piglets.</p></div

Bacterial community of weaned piglets fed with neomycin or <i>E. faecalis</i>.

<p>(A) DGGE profiles of the V6~V8 regions of the 16S rDNA gene fragments from the samples. The denaturant gradient range is from 42% to 58%. The major difference bands are numbered. Lane S (Standard ladder, which are PCR products generated from different bacterial 16S rDNA genes with primers 968F-GC and 1401R); N (negative control, basal diet); P (positive control, diet supplemented with neomycin); L, M, H (diets supplemented with probiotics 0.5×10⁹, 1.0×10⁹ and 2.5×10⁹ CFU/kg feed, respectively); (B) UPGMA cluster analysis of Dice similarity indices from DGGE profiles.</p