Continuously Discovering Novel Strategies via Reward-Switching Policy Optimization

We present Reward-Switching Policy Optimization (RSPO), a paradigm to discover diverse strategies in complex RL environments by iteratively finding novel policies that are both locally optimal and sufficiently different from existing ones. To encourage the learning policy to consistently converge towards a previously undiscovered local optimum, RSPO switches between extrinsic and intrinsic rewards via a trajectory-based novelty measurement during the optimization process. When a sampled trajectory is sufficiently distinct, RSPO performs standard policy optimization with extrinsic rewards. For trajectories with high likelihood under existing policies, RSPO utilizes an intrinsic diversity reward to promote exploration. Experiments show that RSPO is able to discover a wide spectrum of strategies in a variety of domains, ranging from single-agent particle-world tasks and MuJoCo continuous control to multi-agent stag-hunt games and StarCraftII challenges.Comment: 30 pages, 15 figures, published as a conference paper at ICLR 202

Oxidative stress in NSC-741909-induced apoptosis of cancer cells

<p>Abstract</p> <p>Background</p> <p>NSC-741909 is a novel anticancer agent that can effectively suppress the growth of several cell lines derived from lung, colon, breast, ovarian, and kidney cancers. We recently showed that NSC-741909-induced antitumor activity is associated with sustained Jun N-terminal kinase (JNK) activation, resulting from suppression of JNK dephosphorylation associated with decreased protein levels of MAPK phosphatase-1. However, the mechanisms of NSC-741909-induced antitumor activity remain unclear. Because JNK is frequently activated by oxidative stress in cells, we hypothesized that reactive oxygen species (ROS) may be involved in the suppression of JNK dephosphorylation and the cytotoxicity of NSC-741909.</p> <p>Methods</p> <p>The generation of ROS was measured by using the cell-permeable nonfluorescent compound H₂DCF-DA and flow cytometry analysis. Cell viability was determined by sulforhodamine B assay. Western blot analysis, immunofluorescent staining and flow cytometry assays were used to determine apoptosis and molecular changes induced by NSC-741909.</p> <p>Results</p> <p>Treatment with NSC-741909 induced robust ROS generation and marked MAPK phosphatase-1 and -7 clustering in NSC-741909-sensitive, but not resistant cell lines, in a dose- and time-dependent manner. The generation of ROS was detectable as early as 30 min and ROS levels were as high as 6- to 8-fold above basal levels after treatment. Moreover, the NSC-741909-induced ROS generation could be blocked by pretreatment with antioxidants, such as nordihydroguaiaretic acid, aesculetin, baicalein, and caffeic acid, which in turn, inhibited the NSC-741909-induced JNK activation and apoptosis.</p> <p>Conclusion</p> <p>Our results demonstrate that the increased ROS production was associated with NSC-741909-induced antitumor activity and that ROS generation and subsequent JNK activation is one of the primary mechanisms of NSC-741909-mediated antitumor cell activity.</p

Apoptosis Induction by MEK Inhibition in Human Lung Cancer Cells Is Mediated by Bim

AZD6244 (ARRY-142886) is an inhibitor of MEK1/2 and can inhibit cell proliferation or induce apoptosis in a cell-type dependent manner. The precise molecular mechanism of AZD6244-induced apoptosis is not clear. To investigate mechanisms of AZD6244 induced apoptosis in human lung cancer, we determined the molecular changes of two subgroups of human lung cancer cell lines that are either sensitive or resistant to AZD6244 treatment. We found that AZD6244 elicited a large increase of Bim proteins and a smaller increase of PUMA and NOXA proteins, and induced cell death in sensitive lung cancer cell lines, but had no effect on other Bcl-2 related proteins in those cell lines. Knockdown of Bim by siRNA greatly increased the IC50 and reduced apoptosis for AZD6244 treated cells. We also found that levels of endogenous p-Thr32-FOXO3a and p-Ser253-FOXO3a were lower in AZD6244-sensitive cells than in AZD6244-resistant cells. In the sensitive cells, AZD6244 induced FOXO3a nuclear translocation required for Bim activation. Moreover, the silencing of FOXO3a by siRNA abrogated AZD6244-induced cell apoptosis. In addition, we found that transfection of constitutively active AKT up-regulated p-Thr32-FOXO3a and p-Ser253-FOXO3a expression and inhibited AZD6244-induced Bim expression in sensitive cells. These results show that Bim plays an important role in AZD6244-induced apoptosis in lung cancer cells and that the PI3K/AKT/FOXO3a pathway is involved in Bim regulation and susceptibility of lung cancer cells to AZD6244. These results have implications in the development of strategies to overcome resistance to MEK inhibitors

Antitumor Activity of a Novel Oncrasin Analogue Is Mediated by JNK Activation and STAT3 Inhibition

To optimize the antitumor activity of oncrasin-1, a small molecule identified through synthetic lethality screening on isogenic K-Ras mutant tumor cells, we developed several analogues and determined their antitumor activities. Here we investigated in vitro and in vivo antitumor activity of NSC-743380 (1-[(3-chlorophenyl) methyl]-1H-indole-3-methanol, oncrasin-72), one of most potent analogues of oncrasin-1.In vitro antitumor activity was determined in NCI-60 cancer cell line panel using cell viability assay. In vivo antitumor activity was determined in parallel with NSC-741909 (oncrasin-60) in xenograft tumors established in nude mice from A498, a human renal cancer cell line. Changes in gene expression levels and signaling pathway activities upon treatment with NSC-743380 were analyzed in breast and renal cancer cells by Western blot analysis. Apoptosis was demonstrated by Western blot analysis and flow cytometric analysis. NSC-743380 is highly active against a subset of cancer cell lines derived from human lung, colon, ovary, kidney, and breast cancers. The 50% growth-inhibitory concentration (GI(50)) for eight of the most sensitive cell lines was ≤ 10 nM. In vivo study showed that NSC-743380 has a better safety profile and greater antitumor activity than NSC-741909. Treatment with NSC-743380 caused complete regression of A498 xenograft tumors in nude mice at the tested doses ranging from 67 mg/kg to 150 mg/kg. Mechanistic characterization revealed that NSC-743380 suppressed the phosphorylation of C-terminal domain of RNA polymerase II, induced JNK activation, inhibited JAK2/STAT3 phosphorylation and suppressed cyclin D1 expression in sensitive human cancer cells. Blocking JNK activation or overexpression of constitutively active STAT3 partially blocked NSC-743380-induced antitumor activity.NSC-743380 induces antitumor activity through modulation of functions in multiple cancer related pathways and could be a potential anticancer agent for some solid tumors

Integrated transcriptomics and metabolomics analysis provide insight into the resistance response of rice against brown planthopper

IntroductionThe brown planthopper (Nilaparvata lugens Stål, BPH) is one of the most economically significant pests of rice. The Bph30 gene has been successfully cloned and conferred rice with broad-spectrum resistance to BPH. However, the molecular mechanisms by which Bph30 enhances resistance to BPH remain poorly understood.MethodsHere, we conducted a transcriptomic and metabolomic analysis of Bph30-transgenic (BPH30T) and BPH-susceptible Nipponbare plants to elucidate the response of Bph30 to BPH infestation.ResultsTranscriptomic analyses revealed that the pathway of plant hormone signal transduction enriched exclusively in Nipponbare, and the greatest number of differentially expressed genes (DEGs) were involved in indole 3-acetic acid (IAA) signal transduction. Analysis of differentially accumulated metabolites (DAMs) revealed that DAMs involved in the amino acids and derivatives category were down-regulated in BPH30T plants following BPH feeding, and the great majority of DAMs in flavonoids category displayed the trend of increasing in BPH30T plants; the opposite pattern was observed in Nipponbare plants. Combined transcriptomics and metabolomics analysis revealed that the pathways of amino acids biosynthesis, plant hormone signal transduction, phenylpropanoid biosynthesis and flavonoid biosynthesis were enriched. The content of IAA significantly decreased in BPH30T plants following BPH feeding, and the content of IAA remained unchanged in Nipponbare. The exogenous application of IAA weakened the BPH resistance conferred by Bph30.DiscussionOur results indicated that Bph30 might coordinate the movement of primary and secondary metabolites and hormones in plants via the shikimate pathway to enhance the resistance of rice to BPH. Our results have important reference significance for the resistance mechanisms analysis and the efficient utilization of major BPH-resistance genes

Interkingdom multi-omics analysis reveals the effects of nitrogen application on growth and rhizosphere microbial community of Tartary buckwheat

Tartary buckwheat (Fagopyrum tataricum Gaertn.) is an important pseudocereal crop with excellent edible, nutritional and medicinal values. However, the yield of Tartary buckwheat (TB) is very low due to old-fashioned cultivation techniques, particularly unreasonable application of nitrogen fertilizer. To improve the understanding on the theories of nitrogen use in TB, the effects of nitrogen application on growth, as well as chemical properties and microbial community of rhizosphere soil were investigated in this study. Nitrogen application could promote the plant height, stem diameter, nitrogen accumulation and yield of TB. The relative abundance and diversity of bacteria and fungi in the rhizosphere soil of TB were improved by nitrogen fertilizer. Nitrogen application increased the abundance of beneficial bacteria such as Lysobacter and Sphingomonas in rhizosphere soil, and decreased the abundance of pathogenic fungi such as Fusarium and Plectosphaerella. The results indicated that nitrogen application changed the distribution of microbial communities in TB rhizosphere soil. Furthermore, the specific enriched or depleted microorganisms in the rhizosphere soil of four TB varieties were analyzed at OTU level. 87 specific nitrogen-responsive genes with sequence variation were identified in four varieties by integrating genomic re-sequencing and transcriptome analysis, and these genes may involve in the recruitment of specific rhizosphere microorganisms in different TB varieties. This study provided new insights into the effects of nitrogen application on TB growth and rhizosphere microbial community, and improved the understanding on the mechanisms of TB root–microbe interactions

MEI Kodierung der frühesten Notation in linienlosen Neumen

Das Optical Neume Recognition Project (ONRP) hat die digitale Kodierung von musikalischen Notationszeichen aus dem Jahr um 1000 zum Ziel – ein ambitioniertes Vorhaben, das die Projektmitglieder veranlasste, verschiedenste methodische Ansätze zu evaluieren. Die Optical Music Recognition-Software soll eine linienlose Notation aus einem der ältesten erhaltenen Quellen mit Notationszeichen, dem Antiphonar Hartker aus der Benediktinerabtei St. Gallen (Schweiz), welches heute in zwei Bänden in der Stiftsbibliothek in St. Gallen aufbewahrt wird, erfassen. Aufgrund der handgeschriebenen, linienlosen Notation stellt dieser Gregorianische Gesang den Forscher vor viele Herausforderungen. Das Werk umfasst über 300 verschiedene Neumenzeichen und ihre Notation, die mit Hilfe der Music Encoding Initiative (MEI) erfasst und beschrieben werden sollen. Der folgende Artikel beschreibt den Prozess der Adaptierung, um die MEI auf die Notation von Neumen ohne Notenlinien anzuwenden. Beschrieben werden Eigenschaften der Neumennotation, um zu verdeutlichen, wo die Herausforderungen dieser Arbeit liegen sowie die Funktionsweise des Classifiers, einer Art digitalen Neumenwörterbuchs

Hyperspectral image classification based on adaptive segmentation

Object-based hyperspectral image classification (OBHIC) converts the basic unit from 'pixel' to 'object' by image segmentation, in order to take advantage of the spatial distribution law of geographical substances, as well as increase classification performances. However, it involves the problem of scale selection, i.e. the segmentation parameters are set manually by empirical values. In this paper, a novel OBHIC algorithm based on adaptive segmentation is proposed. Here, hyperspectral images (HSIs) are first segmented through a new segmentation scheme with constraint ability, and the thresholds for segmentation are calculated adaptively by utilizing training samples. And then K-nearest neighbor algorithm (KNN) is applied to classify the centers of each region after segmentation. In addition, based on the semisupervised idea, semi-known samples are obtained to further improve the classification performance. Experimental results are presented on two HSI datasets. For different HSIs, the adaptive thresholds calculated are consistent with empirical ones, and the developed classification algorithm has achieved good classification results, thus demonstrating strong robustness of the algorithm. For the HSI Indian Pines from AVIRIS sensor, the Overall Accuracy (OA) and kappa are 95.13% and 0.9444 respectively with 10% training samples, and for the HSI Pavia University from ROSIS sensor, the OA and kappa are 95.52% and 0.9416 respectively with 2% training samples. And good classification performance is still maintained for small number of training samples