Playing 20 Question Game with Policy-Based Reinforcement Learning

The 20 Questions (Q20) game is a well known game which encourages deductive reasoning and creativity. In the game, the answerer first thinks of an object such as a famous person or a kind of animal. Then the questioner tries to guess the object by asking 20 questions. In a Q20 game system, the user is considered as the answerer while the system itself acts as the questioner which requires a good strategy of question selection to figure out the correct object and win the game. However, the optimal policy of question selection is hard to be derived due to the complexity and volatility of the game environment. In this paper, we propose a novel policy-based Reinforcement Learning (RL) method, which enables the questioner agent to learn the optimal policy of question selection through continuous interactions with users. To facilitate training, we also propose to use a reward network to estimate the more informative reward. Compared to previous methods, our RL method is robust to noisy answers and does not rely on the Knowledge Base of objects. Experimental results show that our RL method clearly outperforms an entropy-based engineering system and has competitive performance in a noisy-free simulation environment.Comment: Accepted by EMNLP 201

Growth of High Quality Al-Doped CsLiB6O10 Crystals Using Cs2O-Li2O-MoO3 Fluxes

Abstract: High quality and large size Al-doped CsLiB6O10 (CLBO) single crystals have been successfully grown by top-seeded solution growth (TSSG) technique using Cs2O–Li2O–MoO3 fluxes. The advantages of this newly developed flux system were investigated by viscosity measurements and growth experiments. Al-doped CLBO presents a very high transmittance in the visible region and the weak absorption values at 1064 nm along a and c axes are only 140 and 50 ppm/cm, respectively. The measured LIDT of Al-doped CLBO at λ = 1064 nm and τ = 5.0 ns is 5.10 GW/cm2. Moreover, Al-doped CLBO exhibits an apparent enhancement of the hygroscopic nature in contrast with the undoped crystal as determined by the humidity experiments. Finally, a high fourth harmonic generation (FHG) conversion efficiency of 63% utilizing Al-doped CLBO has been achieved by a picosecond mode-locked Nd:YAG laser, the results also reveal that Al doping has no obvious impact on the FHG conversion efficiency

Computational Insights into Allosteric Conformational Modulation of P-Glycoprotein by Substrate and Inhibitor Binding

The ATP-binding cassette (ABC) transporter P-glycoprotein (P-gp) is a physiologically essential membrane protein that protects many tissues against xenobiotic molecules, but limits the access of chemotherapeutics into tumor cells, thus contributing to multidrug resistance. The atomic-level mechanism of how substrates and inhibitors differentially affect the ATP hydrolysis by P-gp remains to be elucidated. In this work, atomistic molecular dynamics simulations in an explicit membrane/water environment were performed to explore the effects of substrate and inhibitor binding on the conformational dynamics of P-gp. Distinct differences in conformational changes that mainly occurred in the nucleotide-binding domains (NBDs) were observed from the substrate- and inhibitor-bound simulations. The binding of rhodamine-123 can increase the probability of the formation of an intermediate conformation, in which the NBDs were closer and better aligned, suggesting that substrate binding may prime the transporter for ATP hydrolysis. By contrast, the inhibitor QZ-Leu stabilized NBDs in a much more separated and misaligned conformation, which may result in the deficiency of ATP hydrolysis. The significant differences in conformational modulation of P-gp by substrate and inhibitor binding provided a molecular explanation of how these small molecules exert opposite effects on the ATPase activity. A further structural analysis suggested that the allosteric communication between transmembrane domains (TMDs) and NBDs was primarily mediated by two intracellular coupling helices. Our computational simulations provide not only valuable insights into the transport mechanism of P-gp substrates, but also for the molecular design of P-gp inhibitors

Chemical Compositions, Extraction Optimizations, and In Vitro Bioactivities of Flavonoids from Perilla Leaves (<i>Perillae folium</i>) by Microwave-Assisted Natural Deep Eutectic Solvents

Natural deep eutectic solvents (NADESs) have been gradually applied to green extraction of active ingredients. In this study, microwave-assisted NADESs were applied to the extraction of flavonoid compounds from perilla leaves. Through comparative experiments, NADES-3 (choline chloride and malic acid at a molar ratio of 1:1) was found to have the highest extraction efficiency of total flavonoids, including apigenin 7-O-caffeoylglucoside, scutellarein 7-O-diglucuronide, luteolin 7-O-diglucuronide, and scutellarein 7-O-glucuronide by HPLC-MS. The following optimal extraction parameters were obtained based on response surface design: water content in NADES of 23%, extraction power of 410 W, extraction time of 31 min, and solid–liquid ratio of 75 mg/mL, leading to the extraction yield of total flavonoids of 72.54 mg/g. Additionally, the strong antimicrobial and antiallergic activity, inhibition of nitrosation, and antioxidant activity of total flavonoids by using NADESs were confirmed. This new extraction method provides a reference for the further exploration of NADES systems and may be widely used for the green extraction of natural active ingredients

The VHSE-based prediction of proteasomal cleavage sites.

Prediction of proteasomal cleavage sites has been a focus of computational biology. Up to date, the predictive methods are mostly based on nonlinear classifiers and variables with little physicochemical meanings. In this paper, the physicochemical properties of 14 residues both upstream and downstream of a cleavage site are characterized by VHSE (principal component score vector of hydrophobic, steric, and electronic properties) descriptors. Then, the resulting VHSE descriptors are employed to construct prediction models by support vector machine (SVM). For both in vivo and in vitro datasets, the performance of VHSE-based method is comparatively better than that of the well-known PAProC, MAPPP, and NetChop methods. The results reveal that the hydrophobic property of 10 residues both upstream and downstream of the cleavage site is a dominant factor affecting in vivo and in vitro cleavage specificities, followed by residue's electronic and steric properties. Furthermore, the difference in hydrophobic potential between residues flanking the cleavage site is proposed to favor substrate cleavages. Overall, the interpretable VHSE-based method provides a preferable way to predict proteasomal cleavage sites

Non-invasively predicting differentiation of pancreatic cancer through comparative serum metabonomic profiling

Abstract Background The differentiation of pancreatic ductal adenocarcinoma (PDAC) could be associated with prognosis and may influence the choices of clinical management. No applicable methods could reliably predict the tumor differentiation preoperatively. Thus, the aim of this study was to compare the metabonomic profiling of pancreatic ductal adenocarcinoma with different differentiations and assess the feasibility of predicting tumor differentiations through metabonomic strategy based on nuclear magnetic resonance spectroscopy. Methods By implanting pancreatic cancer cell strains Panc-1, Bxpc-3 and SW1990 in nude mice in situ, we successfully established the orthotopic xenograft models of PDAC with different differentiations. The metabonomic profiling of serum from different PDAC was achieved and analyzed by using 1H nuclear magnetic resonance (NMR) spectroscopy combined with the multivariate statistical analysis. Then, the differential metabolites acquired were used for enrichment analysis of metabolic pathways to get a deep insight. Results An obvious metabonomic difference was demonstrated between all groups and the pattern recognition models were established successfully. The higher concentrations of amino acids, glycolytic and glutaminolytic participators in SW1990 and choline-contain metabolites in Panc-1 relative to other PDAC cells were demonstrated, which may be served as potential indicators for tumor differentiation. The metabolic pathways and differential metabolites identified in current study may be associated with specific pathways such as serine-glycine-one-carbon and glutaminolytic pathways, which can regulate tumorous proliferation and epigenetic regulation. Conclusion The NMR-based metabonomic strategy may be served as a non-invasive detection method for predicting tumor differentiation preoperatively

Nonlinear expression and visualization of nonmetric relationships in genetic diseases and microbiome data

Abstract Background The traditional methods of visualizing high-dimensional data objects in low-dimensional metric spaces are subject to the basic limitations of metric space. These limitations result in multidimensional scaling that fails to faithfully represent non-metric similarity data. Results Multiple maps t-SNE (mm-tSNE) has drawn much attention due to the construction of multiple mappings in low-dimensional space to visualize the non-metric pairwise similarity to eliminate the limitations of a single metric map. mm-tSNE regularization combines the intrinsic geometry between data points in a high-dimensional space. The weight of data points on each map is used as the regularization parameter of the manifold, so the weights of similar data points on the same map are also as close as possible. However, these methods use standard momentum methods to calculate parameters of gradient at each iteration, which may lead to erroneous gradient search directions so that the target loss function fails to achieve a better local minimum. In this article, we use a Nesterov momentum method to learn the target loss function and correct each gradient update by looking back at the previous gradient in the candidate search direction. By using indirect second-order information, the algorithm obtains faster convergence than the original algorithm. To further evaluate our approach from a comparative perspective, we conducted experiments on several datasets including social network data, phenotype similarity data, and microbiomic data. Conclusions The experimental results show that the proposed method achieves better results than several versions of mm-tSNE based on three evaluation indicators including the neighborhood preservation ratio (NPR), error rate and time complexity

NS3 Protein from Rice stripe virus affects the expression of endogenous genes in Nicotiana benthamiana

Abstract Background Rice stripe virus (RSV) belongs to the genus Tenuivirus. It is transmitted by small brown planthoppers in a persistent and circulative-propagative manner and causes rice stripe disease (RSD). The NS3 protein of RSV, encoded by the viral strand of RNA3, is a viral suppressor of RNA silencing (VSR). NS3 plays a significant role in viral infection, and NS3-transgenic plants manifest resistance to the virus. Methods The stability and availability of NS3 produced by transgenic Nicotiana benthamiana was investigated by northern blot analysis. The accumulation of virus was detected by western blot analysis. Transcriptome sequencing was used to identify differentially expressed genes (DEGs) in NS3-transgenic N. benthamiana. Results When the host plants were inoculated with RSV, symptoms and viral accumulation in NS3-transgenic N. benthamiana were reduced compared with the wild type. Transcriptome analysis identified 2533 differentially expressed genes (DEGs) in the NS3-transgenic N. benthamiana, including 597 upregulated genes and 1936 downregulated genes. These DEGs were classified into three Gene Ontology (GO) categories and were associated with 43 GO terms. KEGG pathway analysis revealed that these DEGs were involved in pathways associated with ribosomes (ko03010), photosynthesis (ko00195), photosynthesis-antenna proteins (ko00196), and carbon metabolism (ko01200). More than 70 DEGs were in these four pathways. Twelve DEGs were selected for RT-qPCR verification and subsequent analysis. The results showed that NS3 induced host resistance by affecting host gene expression. Conclusion NS3, which plays dual roles in the process of infection, may act as a VSR during RSV infection, and enable viral resistance in transgenic host plants. NS3 from RSV affects the expression of genes associated with ribosomes, photosynthesis, and carbon metabolism in N. benthamiana. This study enhances our understanding of the interactions between VSRs and host plants