Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning

Data augmentation (DA) is a crucial technique for enhancing the sample efficiency of visual reinforcement learning (RL) algorithms. Notably, employing simple observation transformations alone can yield outstanding performance without extra auxiliary representation tasks or pre-trained encoders. However, it remains unclear which attributes of DA account for its effectiveness in achieving sample-efficient visual RL. To investigate this issue and further explore the potential of DA, this work conducts comprehensive experiments to assess the impact of DA's attributes on its efficacy and provides the following insights and improvements: (1) For individual DA operations, we reveal that both ample spatial diversity and slight hardness are indispensable. Building on this finding, we introduce Random PadResize (Rand PR), a new DA operation that offers abundant spatial diversity with minimal hardness. (2) For multi-type DA fusion schemes, the increased DA hardness and unstable data distribution result in the current fusion schemes being unable to achieve higher sample efficiency than their corresponding individual operations. Taking the non-stationary nature of RL into account, we propose a RL-tailored multi-type DA fusion scheme called Cycling Augmentation (CycAug), which performs periodic cycles of different DA operations to increase type diversity while maintaining data distribution consistency. Extensive evaluations on the DeepMind Control suite and CARLA driving simulator demonstrate that our methods achieve superior sample efficiency compared with the prior state-of-the-art methods.Comment: NeurIPS 2023 poste

Silkworm Coatomers and Their Role in Tube Expansion of Posterior Silkgland

Background: Coat protein complex I (COPI) vesicles, coated by seven coatomer subunits, are mainly responsible for Golgito-ER transport. Silkworm posterior silkgland (PSG), a highly differentiated secretory tissue, secretes fibroin for silk production, but many physiological processes in the PSG cells await further investigation. Methodology/Principal Findings: Here, to investigate the role of silkworm COPI, we cloned six silkworm COPI subunits (a,b,b9, d, e, and f-COP), determined their peak expression in day 2 in fifth-instar PSG, and visualized the localization of COPI, as a coat complex, with cis-Golgi. By dsRNA injection into silkworm larvae, we suppressed the expression of a-, b9- and c-COP, and demonstrated that COPI subunits were required for PSG tube expansion. Knockdown of a-COP disrupted the integrity of Golgi apparatus and led to a narrower glandular lumen of the PSG, suggesting that silkworm COPI is essential for PSG tube expansion. Conclusions/Significance: The initial characterization reveals the essential roles of silkworm COPI in PSG. Although silkworm COPI resembles the previously characterized coatomers in other organisms, some surprising findings require further investigation. Therefore, our results suggest the silkworm as a model for studying intracellular transport, and woul

Robust estimation of bacterial cell count from optical density

Optical density (OD) is widely used to estimate the density of cells in liquid culture, but cannot be compared between instruments without a standardized calibration protocol and is challenging to relate to actual cell count. We address this with an interlaboratory study comparing three simple, low-cost, and highly accessible OD calibration protocols across 244 laboratories, applied to eight strains of constitutive GFP-expressing E. coli. Based on our results, we recommend calibrating OD to estimated cell count using serial dilution of silica microspheres, which produces highly precise calibration (95.5% of residuals <1.2-fold), is easily assessed for quality control, also assesses instrument effective linear range, and can be combined with fluorescence calibration to obtain units of Molecules of Equivalent Fluorescein (MEFL) per cell, allowing direct comparison and data fusion with flow cytometry measurements: in our study, fluorescence per cell measurements showed only a 1.07-fold mean difference between plate reader and flow cytometry data

A study of miRNAs targets prediction and experimental validation

National Basic Research Program of China (973 Program) [2005CB121004]; National Programs for High Technology Research and Development Program of China (863 Program) [2006AA10A119]; Innovation Foundation for Graduate Students of Jiangsu Province; NationalmicroRNAs (miRNAs) are 20-24 nucleotide (nt) RNAs that regulate eukaryotic gene expression post-transcriptionally by the degradation or translational inhibition of their target messenger RNAs (mRNAs). To identify miRNA target genes will help a lot by understanding their biological functions. Sophisticated computational approaches for miRNA target prediction, and effective biological techniques for validating these targets now play a central role in elucidating their functions. Owing to the imperfect complementarity of animal miRNAs with their targets, it is difficult to judge the accuracy of the prediction. Complexity of regulation by miRNA-mediated targets at protein and mRNAs levels has made it more challenging to identify the targets. To date, only a few miRNAs targets are confirmed. In this article, we review the methods of miRNA target prediction and the experimental validation for their corresponding mRNA targets in animals

Mutation of a Cuticle Protein Gene, <i>BmCPG10</i>, Is Responsible for Silkworm Non-Moulting in the 2^nd Instar Mutant

<div><p>In the silkworm, metamorphosis and moulting are regulated by ecdysone hormone and juvenile hormone. The subject in the present study is a silkworm mutant that does not moult in the 2^nd instar (<i>nm2</i>). Genetic analysis indicated that the <i>nm2</i> mutation is controlled by a recessive gene and is homozygous lethal. Based on positional cloning, <i>nm2</i> was located in a region approximately 275 kb on the 5^th linkage group by eleven SSR polymorphism markers. In this specific range, according to the transcriptional expression of thirteen genes and cloning, the relative expression level of the <i>BmCPG10</i> gene that encodes a cuticle protein was lower than the expression level of the wild-type gene. Moreover, this gene’s structure differs from that of the wild-type gene: there is a deletion of 217 bp in its open reading frame, which resulted in a change in the protein it encoded. The <i>BmCPG10</i> mRNA was detectable throughout silkworm development from the egg to the moth. This mRNA was low in the pre-moulting and moulting stages of each instar but was high in the gluttonous stage and in newly exuviated larvae. The <i>BmCPG10</i> mRNA showed high expression levels in the epidermis, head and trachea, while the expression levels were low in the midgut, Malpighian tubule, prothoracic gland, haemolymph and ventral nerve cord. The ecdysone titre was determined by ELISA, and the results demonstrated that the ecdysone titre of <i>nm2</i> larvae was lower than that of the wild-type larvae. The <i>nm2</i> mutant could be rescued by feeding 20-hydroxyecdysone, cholesterol and 7—dehydrocholesterol (7dC), but the rescued <i>nm2</i> only developed to the 4^th instar and subsequently died. The moulting time of silkworms could be delayed by <i>BmCPG10</i> RNAi. Thus, we speculated that the mutation of <i>BmCPG10</i> was responsible for the silkworm mutant that did not moult in the 2^nd instar.</p></div

Results of feeding rescue experiments with 20E, cholesterol, and 7dC, using water as a control.

<p>Results of feeding rescue experiments with 20E, cholesterol, and 7dC, using water as a control.</p

Inhibiting weld cracking in high-strength aluminium alloys

Cracking from a fine equiaxed zone (FQZ), often just tens of microns across, plagues the welding of 7000 series aluminum alloys. Using a multiscale correlative methodology, from the millimeter scale to the nanoscale, we shed light on the strengthening mechanisms and the resulting intergranular failure at the FQZ. We show that intergranular AlCuMg phases give rise to cracking by micro-void nucleation and subsequent link-up due to the plastic incompatibility between the hard phases and soft (low precipitate density) grain interiors in the FQZ. To mitigate this, we propose a hybrid welding strategy exploiting laser beam oscillation and a pulsed magnetic field. This achieves a wavy and interrupted FQZ along with a higher precipitate density, thereby considerably increasing tensile strength over conventionally hybrid welded butt joints, and even friction stir welds

The expression and amino acid sequence of the <i>nm2</i> candidate gene.

<p>(A) Expression profiles of <i>nm2</i> candidate genes based on semi-quantitative RT-PCR. 1–13 representing 13 genes: <i>BMgn002598</i>, <i>BMgn002690</i>, <i>BMgn015078</i>, <i>BMgn002599</i>, <i>BMgn002689</i>, <i>BMgn002600</i>, <i>BMgn002688</i>, <i>BMgn002601</i>, <i>BmCPG10 (BMgn002602)</i>, <i>BMgn002603</i>, <i>BMgn002687</i>, <i>BMgn015079</i> and <i>BGIBMGA002685</i>. The <i>Bm-actin A3</i> gene was used as an internal control. Left lane of each map is wild-type and right lane of each map is <i>nm2</i> mutant. (B) Amino acid sequence alignments for wild-type and <i>nm2</i>. The boxes indicate the amino acid deletions in <i>nm2</i>. (C) The top gene structure is the wild-type and the bottom structure is the <i>nm2</i> mutant. The boxes indicate exons for the coding region and the line for the noncoding region. Start and stop codons are indicated by ATG and stop. The deletion of 217 bp in <i>nm2</i> is indicate by a red dotted line. (D) The top is the full -length cDNA of wild-type, and the bottom is the full -length cDNA of <i>nm2</i>.</p

Assay of the ecdysone titre in the wild type and <i>nm2</i>, as determined by ELISA.

<p>ELISA analysis was repeated at least three times for each set of protein samples. Each point represents the mean value ±SD. **indicates significant difference (p < 0.01) compared with the wild type.</p