Optimal movement through a maze by an application of Q learning

Machine learning has become a very important aspect of the development of artificial intelligence (AI) in the world of computing. As a part of the machine learning techniques, reinforcement learning enables an agent to explore and study the unknown environment and through trial and error, the agent learns how to perform desired tasks in an optimum way. In this project, one of the methods to solve reinforcement problems, namely Q-learning, is studied and analysed. A maze problem is set up to count the learning time of an agent to travel from a fixed starting point A to destination B using Q-learning algorithm on Matlab platform. The learning time will be studied and then compared with different sets of maze of different size and complexity. Finally, a thorough discussion on the differences in the learning time and methods to further optimise the Q-learning algorithm will be provided. This document reports the outcome of this project and highlights the problems and challenges encountered and solutions devised

Data_Sheet_1_Urinary polycyclic aromatic hydrocarbon metabolites were associated with short sleep duration and self-reported trouble sleeping in US adults: data from NHANES 2005–2016 study population.pdf

BackgroundThe aim of the current study was to investigate the link between human exposure to PAHs with short sleep duration (SSD) and self-reported trouble sleeping.MethodsA total of 9,754 participants and 9,777 participants obtained from NHANES 2005–2016 were included in this cross-sectional study about SSD and self-reported trouble sleeping, respectively. The association between urinary PAHs metabolites with the prevalence of SSD and self-reported trouble sleeping by the weighted multivariate logistic regression model, restricted cubic spline (RCS) curves, and weighted quantile sum (WQS) regression.ResultsAfter adjusting for all covariates, 1-hydroxynapthalene, 2-hydroxynapthalene, 3-hydroxyfluorene, 2-hydroxyfluorene, 1-hydroxyphenanthrene, and 1-hydroxyphenanthrene demonstrated positive associations with SSD prevalence. Besides, 1-hydroxynapthalene, 2-hydroxynapthalene, 3-hydroxyfluorene, 2-hydroxyfluorene, 1-hydroxyphenanthrene, and 1-hydroxyphenanthrene exhibited positive associations with the prevalence of self-reported trouble sleeping following the adjustment for all covariates. RCS curves confirmed the non-linear associations between 1-hydroxynapthalene, 2-hydroxynapthalene, 3-hydroxyfluorene, 2-hydroxyfluorene, and 1-hydroxyphenanthrene with the prevalence of SSD, and 1-hydroxynapthalene, 3-hydroxyfluorene, and 2-hydroxyfluorene with the prevalence of self-reported trouble sleeping. The WQS results showed that mixed exposure to PAH metabolites had a significant positive association with the prevalence of SSD (OR: 1.087, 95% CI: 1.026, 1.152, p = 0.004) and self-reported trouble sleeping (OR: 1.190, 95% CI: 1.108, 1.278, p ConclusionUrinary concentrations of PAH metabolites exhibited a close association with the prevalence of SSD and self-reported trouble sleeping in US adults. More emphasis should be placed on the importance of environmental effects on sleep health.</p

pH-Responsive Core–Shell Structured Nanoparticles for Triple-Stage Targeted Delivery of Doxorubicin to Tumors

The application of cytotoxic chemotherapeutics in cancer therapy has been largely restricted by their lack of selectivity. Despite the existence of numerous targeted delivery systems, it is practically challenging to develop one single system to simultaneously cover tumor-targeted delivery of chemotherapeutics at the tissue, cellular, and subcellular levels. To solve this problem, pH-responsive core–shell structured nanoparticles (CSNPs) were self-assembled in this study to provide triple-stage targeted delivery of doxorubicin (DOX) from the injection site to the nuclei of cancer cells. Amino-functionalized mesoporous silica nanoparticles (MSN) were doubly modified with TAT peptide and acid-cleavable polyethylene glycol (PEG) as the DOX-loaded cationic core. The anionic shell was constituted by galactose-modified poly­(allylamine hydrochloride)-citraconic anhydride, a hepato-carcinoma-targeting polymer with charge-reversible property. In vitro results showed that PEG effectively reduced protein adsorption and phagocytic capture of CSNPs in the circulating blood (pH 7.4), thus facilitating passive accumulation in tumors (tissue level). Following PEG detachment via acidic hydrolysis in tumor microenvironment (pH 6.5), the exposed galactose ligands endowed CSNPs with active internalization into hepato-carcinoma cells (cellular level). Afterward, the acidity in endosomes and lysosomes (pH 5.0) triggered the conversion of anionic shell into positive charges, leading to core–shell disassembly and subsequent TAT-mediated delivery of DOX to the nuclei (subcellular level). Importantly, the efficiencies of each targeting moiety were nicely preserved when combining together in CSNPs. As a result, improved tumorous distribution and potent therapeutic efficacy of CSNPs were noted in tumor-bearing mice at a relatively low dose. CSNPs therefore provide an efficient and nontoxic platform for the targeted delivery of antitumor drugs

Table_1_Mendelian randomization analysis to elucidate the causal relationship between small molecule metabolites and ovarian cancer risk.docx

BackgroundSmall molecule metabolites are potential biomarkers for ovarian cancer. However, the causal relationship between small molecule metabolites and ovarian cancer remains unclear.MethodsSingle nucleotide polymorphisms (SNPs) correlated with 53 distinct small molecule metabolites were identified as instrumental variables (IVs) from comprehensive genome-wide association studies. Aggregate data encompassing 25,509 cases of ovarian cancer and 40,941 controls of European descent were procured from the Ovarian Cancer Association Consortium. To evaluate causative associations, four Mendelian randomization techniques—including inverse-variance weighted, weighted median, maximum likelihood, and MR-Egger regression—were employed.ResultsIn total, 242 SNPs were delineated as IVs for the small molecule metabolites under consideration. A significant association with the overarching risk of ovarian cancer was observed for six distinct metabolites. Hexadecenoylcarnitine and methioninesulfoxide were associated with a 32% and 31% reduced risk, respectively. Fifteen metabolites were linked to subtype ovarian cancers. For instance, both methionine sulfoxide and tetradecanoyl carnitine exhibited an inverse association with the risk of clear cell and high-grade serous ovarian cancers. Conversely, tryptophan demonstrated a 1.72-fold elevated risk for endometrioid ovarian cancer.ConclusionThis study identified several metabolites with putative causal effects on ovarian cancer risk using Mendelian randomization analysis. The findings provide insight into the etiological role of small molecule metabolites and highlight potential early detection biomarkers for ovarian cancer. Subsequent investigations are imperative to corroborate these findings and elucidate the underlying pathophysiological mechanisms.</p

Mulching films affecting soil bacterial and fungal communities in a drip-irrigated potato soil

ABSTRACT Film mulching is an effective water-saving and yield-increasing measure for potato production in Northwest China. However, the response mechanism of microbial communities to mulching films in the soil is still unclear. In this study, polyethylene film mulching (PM), biodegradable film mulching (BM), liquid film mulching (LM), and non-mulching (NM) were applied on the drip-irrigated soil to investigate the effects of mulching films on soil bacterial and fungal communities through DNA sequencing, Pearson correlation analysis, and redundancy analysis. The results showed that LM treatment significantly increased the contents of soil mineral N (SMN), dissolved organic carbon (DOC), and dissolved organic nitrogen (DON) (p</div

The Relative Orientation of the TM3 and TM4 Domains Varies between α1 and α3 Glycine Receptors

Glycine receptors (GlyRs) are anion-conducting members of the pentameric ligand-gated ion channel family. We previously showed that the dramatic difference in glycine efficacies of α1 and α3 GlyRs is largely attributable to their nonconserved TM4 domains. Because mutation of individual nonconserved TM4 residues had little effect, we concluded that the efficacy difference was a distributed effect of all nonconserved TM4 residues. We therefore hypothesized that the TM4 domains of α1 and α3 GlyRs differ in structure, membrane orientation, and/or molecular dynamic properties. Here we employed voltage-clamp fluorometry to test whether their TM4 domains interact differently with their respective TM3 domains. We found a rhodamine fluorophore covalently attached to a homologous TM4 residue in each receptor interacts differentially with a conserved TM3 residue. We conclude that the α1 and α3 GlyR TM4 domains are orientated differently relative to their TM3 domains. This may underlie their differential ability to influence glycine efficacy

Evolution of Packing Parameters in the Structural Changes of Silica Mesoporous Crystals: Cage-Type, 2D Cylindrical, Bicontinuous Diamond and Gyroid, and Lamellar

Cage-type, two-dimensional (2D) cylindrical hexagonal (C), bicontinuous diamond (D), bicontinuous gyroid (G), and one-dimensional (1D) lamellar (L) structures of silica mesoporous crystals (SMCs) were obtained by using the anionic surfactant N-stearoyl-l-glutamic acid (C18GluA) as a template in the presence of the nonionic surfactant C16(EO)10 (Brij-56). The mesostructures were controlled by the organic/inorganic interface curvature change induced by Brij-56. A synthesis-field diagram showed that the mesostructure changed in the sequence cage-type → C → intergrowth of C and D → intergrowth of C and G → D → G → L with increase of the amount of Brij-56. Mixed micelles were formed by the anionic and nonionic surfactants, the packing parameter g of which increased with increasing the addition amount of nonionic surfactant and the reaction temperature. The local g parameter was obtained from electron crystallography reconstruction results by calculating mean curvatures and Gaussian curvatures from the equi-electrostatic potential surface. The intergrowth of C and D and two kinds of intergrowth of C and G are also discussed

Structural Analyses of Intergrowth and Stacking Fault in Cage-Type Mesoporous Crystals

A mixture of Gemini cationic surfactant, costructure directing agent (CSDA) with carboxylate group, and silica oligomer leads to the formation of cage-type mesoporous crystals, and the system changes the structure from Fm3̅m (or P63/mmc) type to Fd3̅m type with an increasing amount of additive (HCl). Here we have investigated the Fm3̅m−Fd3̅m type intergrowth and new type stacking fault observed in the intermediate phase using transmission electron microscopy (TEM) and proposed a layer-by-layer growth mechanism. In order to describe the structures, four types of polyhedra for the tetrahedrally close-packed structures are introduced. In the intergrowth, the structure changes the packing of spherical micelles between the Fm3̅m (or P63/mmc) and Fd3̅m structures depending on the pH. The results can be explained by the change between a hard sphere characteristic (perfect sphere) in the Fm3̅m structure and a soft sphere characteristic (polyhedron) in the Fd3̅m structure. The stacking fault in the Fd3̅m structure can be explained by new arrangement of two polyhedra, the 14- and 15-hedron

An adaptive wavelet method for solving mixed-integer dynamic optimization problems with discontinuous controls and application to alkali–surfactant–polymer flooding

This article presents an adaptive rationalized Haar function approximation method to solve dynamic optimization with mixed-integer and discontinuous controls. Three measures are taken to deal with the discontinuity. First, the problem is converted into a multi-stage optimization problem by non-uniform control vector parameterization. Secondly, an adaptive strategy is proposed to regulate the interval division and the order of Haar function vectors. Thirdly, a structure detection method is presented to refine the subintervals, in which adjacent arcs with the same input type are merged into one to modify redundant subintervals. During this approximation solution, the mixed-integer restriction is realized by the integer truncation strategy. Combined with the Hamiltonian function, a validation principle is shown to verify the optimality of the solution. Finally, the proposed method is applied to solve the enhanced oil recovery for alkali–surfactant–polymer flooding. The effectiveness of the method is illustrated through simulation.</p

Effective Solid Electrolyte Based on Benzothiazolium for Dye-Sensitized Solar Cells

Thiaozole/benzothiaozole-based dicationic conductors were synthesized and applied as solid-state electrolyte in dye-sensitized solar cells (DSSCs). X-ray diffraction, scanning electron microscopy, thermal gravimetric analysis, steady-state voltammogram, photocurrent intensity–photovoltage test, and electrochemical impedance spectroscopy are used to characterize the materials and the mechanism of the cell performance. Compared to the traditional monocationic crystals, the dicationic crystals have a larger size and can provide more opportunities to fine-tune their physical/chemical properties. As a consequence, this solid-state electrolyte-based DSSC achieved photoelectric conversion efficiency of 7.90% under full air-mass (AM 1.5) sunlight (100 mW·cm^–2)