Using Reinforcement Learning to Solve a Dynamic Orienteering Problem with Random Rewards Affected by the Battery Status

This paper discusses an orienteering optimization problem where a vehicle using electric batteries must travel from an origin depot to a destination depot while maximizing the total reward collected along its route. The vehicle must cross several consecutive regions, with each region containing different types of charging nodes. A charging node has to be selected in each region, and the reward for visiting each node-in terms of a 'satisfactory' charging process-is a binary random variable that depends upon dynamic factors such as the type of charging node, weather conditions, congestion, battery status, etc. To learn how to efficiently operate in this dynamic environment, a hybrid methodology combining simulation with reinforcement learning is proposed. The reinforcement learning component is able to make informed decisions at each stage, while the simulation component is employed to validate the learning process. The computational experiments show how the proposed methodology is capable of design routing plans that are significantly better than non-informed decisions, thus allowing for an efficient management of the vehicle's battery under such dynamic conditions

Using Reinforcement Learning to Solve a Dynamic Orienteering Problem with Random Rewards Affected by the Battery Status

[EN] This paper discusses an orienteering optimization problem where a vehicle using electric batteries must travel from an origin depot to a destination depot while maximizing the total reward collected along its route. The vehicle must cross several consecutive regions, with each region containing different types of charging nodes. A charging node has to be selected in each region, and the reward for visiting each node¿in terms of a `satisfactory¿ charging process¿is a binary random variable that depends upon dynamic factors such as the type of charging node, weather conditions, congestion, battery status, etc. To learn how to efficiently operate in this dynamic environment, a hybrid methodology combining simulation with reinforcement learning is proposed. The reinforcement learning component is able to make informed decisions at each stage, while the simulation component is employed to validate the learning process. The computational experiments show how the proposed methodology is capable of design routing plans that are significantly better than non-informed decisions, thus allowing for an efficient management of the vehicle¿s battery under such dynamic conditions.This work was partially funded by the European Commission projects SUN (HORIZONCL4-2022-HUMAN-01-14-101092612), and AIDEAS (HORIZON-CL4-2021-TWIN-TRANSITION-01-07-101057294).Juan, AA.; Marugan, CA.; Ahsini, Y.; Fornes, R.; Panadero, J.; Martín, XA. (2023). Using Reinforcement Learning to Solve a Dynamic Orienteering Problem with Random Rewards Affected by the Battery Status. Batteries. 9(8). https://doi.org/10.3390/batteries90804169

Stabilization of p21 by mTORC1/4E-BP1 predicts clinical outcome of head and neck cancers

The levels, regulation and prognostic value of p21 in head and neck squamous cell carcinomas (HNSCC) has been puzzling for years. Here, we report a new mechanism of regulation of p21 by the mTORC1/4E-BP1 pathway. We find that non-phosphorylated 4E-BP1 interacts with p21 and induces its degradation. Accordingly, hyper-activation of mTORC1 results in phosphorylation of 4E-BP1 and stabilization of p21. In HNSCC, p21 levels strongly correlate with mTORC1 activity but not with p53 status. Finally, clinical data indicate that HNSCC patients with p21 and phospho-S6-double-positive tumours present a better disease-specific survival. We conclude that over-activation of the mTORC1/4E-BP1/p21 pathway is a frequent and clinically relevant alteration in HNSCC.We are grateful to Reidar Grenman, Silvio Gutkind, Nahum Sonenberg, Gordon Peters, David Sabatini and Mariano Barbacid for sharing critical reagents. We also thank Aurora Astudillo, Aitana Vallina, Laura Alonso-Dura ́n and Eva Allonca for excellent technical assistance. Work in the laboratory of M.S. is funded by the CNIO and by grants from the Spanish Ministry of Economy (SAF) co-funded by the European Regional Development Fund, the European Research Council (ERC Advanced Grant), the Regional Government of Madrid co-funded by the European Social Fund, the Botin Foundation and BancoSantander (Santander Universities Global Division), the Ramon Areces Foundation an the AXA Foundation. Work in the laboratory of J.M.G.-P. and J.P.R. was supported bygrants from Plan Nacional de DþI 2013–2016 ISCIII (CP13/00013 andPI13/00259),RD12/0036/0015 of Red Tematica de Investigacio ́n Cooperativa en Cancer (RTICC), Spain and the FEDER Funding Program from the European UnionS

High Throughput Screening for Inhibitors of Alpha-Galactosidase

Fabry disease is a rare X-linked lysosomal storage disorder caused by a deficiency in α-galactosidase A (GLA), which catalyzes the hydrolysis of terminal α-galactosyl groups from glycosphingolipids, such as globotriaosylceramide (Gb3). Many of the mutations in the GLA gene are missense alterations that cause misfolding, decreased stability, and/or mistrafficking of this protein. Small molecule compounds that correct the misfolding and mistrafficking, or activate the mutant enzyme, may be useful in the treatment of Fabry disease. We have screened a library of approximately 230,000 compounds using preparations of human recombinant protein and purified coffee bean enzyme in an effort to find activators and inhibitors of this enzyme. Lansoprazole was identified as a small molecule inhibitor of GLA derived from coffee beans (IC50 = 6.4 μM), but no inhibitors or activators were identified for the human enzyme. The screening results indicate that human GLA is a difficult target for small molecule inhibition or activation

The function of phosphatidylinositol 5-phosphate 4-kinase γ (PI5P4Kγ) explored using a specific inhibitor that targets the PI5P-binding site.

NIH-12848 (NCGC00012848-02), a putative phosphatidylinositol 5-phosphate 4-kinase γ (PI5P4Kγ) inhibitor, was explored as a tool for investigating this enigmatic, low activity, lipid kinase. PI5P4K assays in vitro showed that NIH-12848 inhibited PI5P4Kγ with an IC50 of approximately 1 μM but did not inhibit the α and β PI5P4K isoforms at concentrations up to 100 μM. A lack of inhibition of PI5P4Kγ ATPase activity suggested that NIH-12848 does not interact with the enzyme's ATP-binding site and direct exploration of binding using hydrogen-deuterium exchange (HDX)-MS (HDX-MS) revealed the putative PI5P-binding site of PI5P4Kγ to be the likely region of interaction. This was confirmed by a series of mutation experiments which led to the identification of a single PI5P4Kγ amino acid residue that can be mutated to its PI5P4Ks α and β homologue to render PI5P4Kγ resistant NIH-12848 inhibition. NIH-12848 (10 μM) was applied to cultured mouse principal kidney cortical collecting duct (mpkCCD) cells which, we show, express PI5P4Kγ that increases when the cells grow to confluence and polarize. NIH-12848 inhibited the translocation of Na⁺/K⁺-ATPase to the plasma membrane that occurs when mpkCCD cells grow to confluence and also prevented reversibly their forming of 'domes' on the culture dish. Both these NIH-12848-induced effects were mimicked by specific RNAi knockdown of PI5P4Kγ, but not that of PI5P4Ks α or β. Overall, the data reveal a probable contribution of PI5P4Kγ to the development and maintenance of epithelial cell functional polarity and show that NIH-12848 is a potentially powerful tool for exploring the cell physiology of PI5P4Ks.J.H.C was supported by the MRC (Grant RG64071), M-L.G. by the BBSRC (Grant RG65394), and J.H.B. by the BHF (Grant PG11/109/29247).This is the final version of the article. It first appeared from the Biochemical Society via http://dx.doi.org/10.1042/BJ2014133

High Throughput Screening for Small Molecule Therapy for Gaucher Disease Using Patient Tissue as the Source of Mutant Glucocerebrosidase

Gaucher disease (GD), the most common lysosomal storage disorder, results from the inherited deficiency of the lysosomal enzyme glucocerebrosidase (GCase). Previously, wildtype GCase was used for high throughput screening (HTS) of large collections of compounds to identify small molecule chaperones that could be developed as new therapies for GD. However, the compounds identified from HTS usually showed reduced potency later in confirmatory cell-based assays. An alternate strategy is to perform HTS on mutant enzyme to identify different lead compounds, including those enhancing mutant enzyme activities. We developed a new screening assay using enzyme extract prepared from the spleen of a patient with Gaucher disease with genotype N370S/N370S. In tissue extracts, GCase is in a more native physiological environment, and is present with the native activator saposin C and other potential cofactors. Using this assay, we screened a library of 250,000 compounds and identified novel modulators of mutant GCase including 14 new lead inhibitors and 30 lead activators. The activities of some of the primary hits were confirmed in subsequent cell-based assays using patient-derived fibroblasts. These results suggest that primary screening assays using enzyme extracted from tissues is an alternative approach to identify high quality, physiologically relevant lead compounds for drug development

Development of an Aryloxazole Class of Hepatitis C Virus Inhibitors Targeting the Entry Stage of the Viral Replication Cycle

Reliance on hepatitis C virus (HCV) replicon systems and protein-based screening assays has led to treatments that target HCV viral replication proteins. The model does not encompass other viral replication cycle steps such as entry, processing, assembly and secretion, or viral host factors. We previously applied a phenotypic high-throughput screening platform based on an infectious HCV system and discovered an aryloxazole-based anti-HCV hit. Structure– activity relationship studies revealed several compounds exhibiting EC50 values below 100 nM. Lead compounds showed inhibition of the HCV pseudoparticle entry, suggesting a different mode of action from existing HCV drugs. Hit 7a and lead 7ii both showed synergistic effects in combination with existing HCV drugs. In vivo pharmacokinetics studies of 7ii showed high liver distribution and long half-life without obvious hepatotoxicity. The lead compounds are promising as preclinical candidates for the treatment of HCV infection and as molecular probes to study HCV pathogenesis

High-Throughput Screening, Discovery, and Optimization to Develop a Benzofuran Class of Hepatitis C Virus Inhibitors

Using a high-throughput, cell-based HCV luciferase reporter assay to screen a diverse small-molecule compound collection (~300 000 compounds), we identified a benzofuran compound class of HCV inhibitors. The optimization of the benzofuran scaffold led to the identification of several exemplars with potent inhibition (EC50 25 µM), and excellent selectivity (selective index = CC50/EC50, > 371-fold). The structure–activity studies culminated in the design and synthesis of a 45-compound library to comprehensively explore the anti-HCV activity. The identification, design, synthesis, and biological characterization for this benzofuran series is discussed