Towards a Methodology for the Economic Performance Increase of Production Lines using Reinforcement Learning

The increasing number of variants in product portfolios contributes to the challenge of efficient manufacturing on production lines due to the resulting small batch sizes and thus frequent product changes that lower the average overall plant effectiveness. Especially for companies that manufacture at high speed on production lines, such as in the Fast Moving Consumer Good (FMCG) industry, it is a central task of operational management to increase the performance of production lines. Due to the multitude of different adjustment levers at several interdependent machines, the identification of efficient actions and their combination into economic improvement trajectories is challenging. There is a variety of approaches to address this challenge, e.g. simulation-based heuristics. However, these approaches mostly focus on details instead of giving a holistic perspective of the possibilities to improve a production line or are limited in practical application. In other areas of application, reinforcement learning has shown remarkable success in recent years. The principle feasibility of using reinforcement learning in this application context has been demonstrated as well. However, it became apparent that the integration of expert knowledge throughout the improvement process is necessary. For this reason this paper transforms five modules defined from an engineering point of view into the mathematical scheme of a markov decision problem, a default framework for reinforcement learning. This provides the foundation for applying reinforcement learning in combination with expert knowledge from an engineering perspective

Active Zone Scaffold Protein Ratios Tune Functional Diversity across Brain Synapses

Summary: High-throughput electron microscopy has started to reveal synaptic connectivity maps of single circuits and whole brain regions, for example, in the Drosophila olfactory system. However, efficacy, timing, and frequency tuning of synaptic vesicle release are also highly diversified across brain synapses. These features critically depend on the nanometer-scale coupling distance between voltage-gated Ca2+ channels (VGCCs) and the synaptic vesicle release machinery. Combining light super resolution microscopy with in vivo electrophysiology, we show here that two orthogonal scaffold proteins (ELKS family Bruchpilot, BRP, and Syd-1) cluster-specific (M)Unc13 release factor isoforms either close (BRP/Unc13A) or further away (Syd-1/Unc13B) from VGCCs across synapses of the Drosophila olfactory system, resulting in different synapse-characteristic forms of short-term plasticity. Moreover, BRP/Unc13A versus Syd-1/Unc13B ratios were different between synapse types. Thus, variation in tightly versus loosely coupled scaffold protein/(M)Unc13 modules can tune synapse-type-specific release features, and “nanoscopic molecular fingerprints” might identify synapses with specific temporal features. : Fulterer et al. demonstrates that the scaffold proteins Bruchpilot and Syd-1 cluster (M)Unc13 release factor isoforms either close (BRP/Unc13A) or further away (Syd-1/Unc13B) from voltage-gated Ca2+ channels in the Drosophila olfactory system. These scaffold/release factor “modules” varied significantly between different synapse types, thereby tuning release features toward depression or facilitation. Keywords: neurotransmitter release, positional priming, munc13, Bruchpilot, Syd-1, synapse diversity, olfactory system, Drosophila, nanoscopy, synapse physiology, active zon

Properties of the Arg376 residue of the proton-coupled folate transporter (PCFT-SLC46A1) and a glutamine mutant causing hereditary folate malabsorption

The proton-coupled folate transporter (PCFT-SLC46A1) is required for intestinal folate absorption and is mutated in the autosomal recessive disorder, hereditary folate malabsorption (HFM). This report characterizes properties and requirements of the R376 residue in PCFT function, including a R376Q mutant associated with HFM. Gln, Cys, and Ala substitutions resulted in markedly impaired transport of 5-formyltetrahydrofolate (5-FTHF) and 5-methyltetrahydrofolate (5-MTHF) due to an increase in Km and decrease in Vmax in HeLa R1–11 transfectants lacking endogenous folate transport function. In contrast, although the influx Km for pemetrexed was increased, transport was fully preserved at saturating concentrations and enhanced for the like-charged R376K- and R376H-PCFT. Pemetrexed and 5-FTHF influx mediated by R376Q-PCFT was markedly decreased at pH 5.5 compared with wild-type PCFT. However, while pemetrexed transport was substantially preserved at low pH (4.5–5.0), 5-FTHF transport remained very low. Electrophysiological studies in Xenopus oocytes demonstrated that 1) the R376Q mutant, like wild-type PCFT, transports protons in the absence of folate substrate, and in this respect has channel-like properties; and 2) the influx Km mediated by R376Q-PCFT is increased for 5-MTHF, 5-FTHF, and pemetrexed. The data suggest that mutation of the R376 residue to Gln impairs proton binding which, in turn, modulates the folate-binding pocket and depresses the rate of conformational alteration of the carrier, a change that appears to be, in part, substrate dependent

Spermidine Suppresses Age-Associated Memory Impairment by Preventing Adverse Increase of Presynaptic Active Zone Size and Release.

Memories are assumed to be formed by sets of synapses changing their structural or functional performance. The efficacy of forming new memories declines with advancing age, but the synaptic changes underlying age-induced memory impairment remain poorly understood. Recently, we found spermidine feeding to specifically suppress age-dependent impairments in forming olfactory memories, providing a mean to search for synaptic changes involved in age-dependent memory impairment. Here, we show that a specific synaptic compartment, the presynaptic active zone (AZ), increases the size of its ultrastructural elaboration and releases significantly more synaptic vesicles with advancing age. These age-induced AZ changes, however, were fully suppressed by spermidine feeding. A genetically enforced enlargement of AZ scaffolds (four gene-copies of BRP) impaired memory formation in young animals. Thus, in the Drosophila nervous system, aging AZs seem to steer towards the upper limit of their operational range, limiting synaptic plasticity and contributing to impairment of memory formation. Spermidine feeding suppresses age-dependent memory impairment by counteracting these age-dependent changes directly at the synapse

Synthesis, Biological, and Antitumor Activity of a Highly Potent 6-Substituted Pyrrolo[2,3-<i>d</i>]pyrimidine Thienoyl Antifolate Inhibitor with Proton-Coupled Folate Transporter and Folate Receptor Selectivity over the Reduced Folate Carrier That Inhibits β-Glycinamide Ribonucleotide Formyltransferase

2-Amino-4-oxo-6-substituted pyrrolo[2,3-<i>d</i>]pyrimidine antifolates with a thienoyl side chain (compounds <b>1</b>–<b>3</b>, respectively) were synthesized for comparison with compound <b>4</b>, the previous lead compound of this series. Conversion of hydroxyl acetylen-thiophene carboxylic esters to thiophenyl-α-bromomethylketones and condensation with 2,4-diamino-6-hydroxypyrimidine afforded the 6-substituted pyrrolo[2,3-<i>d</i>]pyrimidine compounds of type <b>18</b> and <b>19</b>. Coupling with l-glutamate diethyl ester, followed by saponification, afforded <b>1</b>–<b>3</b>. Compound <b>3</b> selectively inhibited the proliferation of cells expressing folate receptors (FRs) α or β, or the proton-coupled folate transporter (PCFT), including KB and IGROV1 human tumor cells, much more potently than <b>4</b>. Compound <b>3</b> was more inhibitory than <b>4</b> toward β-glycinamide ribonucleotide formyltransferase (GARFTase). Both <b>3</b> and <b>4</b> depleted cellular ATP pools. In SCID mice with IGROV1 tumors, <b>3</b> was more efficacious than <b>4</b>. Collectively, our results show potent antitumor activity for <b>3</b> in vitro and in vivo, associated with its selective membrane transport by FRs and PCFT over RFC and inhibition of GARFTase, clearly establishing the 3-atom bridge as superior to the 1-, 2-, and 4-atom bridge lengths for the activity of this series

High-resolution analysis of PN-to-KC synapses within the mushroom body calyx shows increase in T-bar size.

<p>(a–c) Electron micrographs of calyx region of 3d, 30d, and 30d^Spd <i>w</i>^<i>1118</i> animals showing presynaptic specializations in blue (T-bars) at the PN-to-KC synapses. Scale bar: 50 nm. (d) Quantification representing the average T-bar size in 3d, 30d, and 30d^Spd animals (<i>n</i> = 92–100 electromicrographs across four independent animals, with at least 20 T-bars per animal; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). (e–g) STED images of BRP spots reveal ring-shaped structures (arrowheads) within the calyx of 3d, 30d, and 30d^Spd <i>w</i>^<i>1118</i> flies. Scale bar: 500 nm. (h) Comparison of BRP-spot diameter between 3d, 30d, and 30d^Spd flies (total of 94–112 BRP rings across 15 independent animals, with at least 5 BRP rings per animal; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). (i) Electron micrographs of PN bouton within the calyx region of 3d <i>w</i>^<i>1118</i> flies. Scale bar: 200 nm. (j–l) Higher magnification of AZ within PN bouton immunostained for BRP (large gold particles) and RBP (small gold particles) of 3d, 30d, and 30d^Spd <i>w</i>^<i>1118</i> flies. Scale bar: 50 nm. (m) Quantification of BRP-positive gold particles per T-bar (total of 94–108 individual T-bars across three independent animals, with at least 25 T-bars per animal; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). (n) Quantification of RBP-positive gold particles per T-bar (total of 94–108 individual T-bars across three independent animals, with at least 25 T-bars per animal; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). * <i>p</i> < 0.05, ** <i>p</i> < 0.01, *** <i>p</i> < 0.001, ns = not significant, <i>p</i> ≥ 0.05. Underlying data is shown in <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.1002563#pbio.1002563.s001" target="_blank">S1 Data</a>.</p

Spermidine feeding suppresses age-associated increase in BRP and rim-binding protein (RBP) levels.

<p>(a–c) Adult brains 3d and 30d <i>w</i>^<i>1118</i> flies, together with 30d^Spd <i>w</i>^<i>1118</i> flies immunostained for Synapsin. Scale bar: 50 μm. (d) Quantification of Synapsin intensity within the central brain region normalized to 3d flies (<i>n</i> = 9–10 independent brains; Kruskal-Wallis test). (e–g) Adult brains of 3d and 30d <i>w</i>^<i>1118</i> flies, together with 30d^Spd <i>w</i>^<i>1118</i> flies immunostained for Synaptotagmin-1 (Syt-1). Scale bar: 50 μm. (h) Quantification of signal intensity of Syt-1 in the central brain region normalized to 3d flies (<i>n</i> = 8–9 independent brains; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). (i–k) Adult brains of 3d, 30d <i>w</i>^<i>1118</i>, and 30d^Spd <i>w</i>^<i>1118</i> flies immunostained for BRP (using Nc82 and N-terminal antibodies) and RBP. Scale bar: 50 μm (l–n) Quantification of BRP (using Nc82 and N-terminal antibodies) and RBP intensities within the central brain region normalized to 3d flies (<i>n</i> = 14–18 independent brains; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). ** <i>p</i> < 0.01, *** <i>p</i> < 0.001, ns = not significant, <i>p</i> ≥ 0.05. Underlying data is shown in <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.1002563#pbio.1002563.s001" target="_blank">S1 Data</a>.</p

Increase in levels of core AZ proteins BRP and RBP leads to early memory impairment.

<p>(a–d) Adult brains of 3d and 30d flies expressing 4xBRP together with age-matched controls <i>brp</i> (2xBRP), immunostained for BRP (using Nc82 and BRP N-terminal antibody) and RBP. Scale bar: 50 μm. (e, f) Quantification of BRP (using N-terminal antibody) as well as RBP intensity within the central brain region normalized to 3d flies (<i>n</i> = 12–13 independent brains; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). (g–j) STED images of BRP label within the calyx region of 3d and 30d flies expressing 4xBRP as well as 2xBRP. Ring-shaped structures are indicated (arrowheads). Scale bar: 500 nm. (k) Quantification of BRP ring diameter in 3d and 30d 4xBRP flies along with age-matched 2xBRP flies (total of 47–68 BRP rings across eight independent animals, with at least six BRP rings per animal; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). (l) Aversive associative memory performance 3 min after training (short-term memory; STM) markedly reduced in 3d 4xBRP flies in comparison to 3d wild-type 2xBRP flies (<i>n</i> = 10–16; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). (m) Aversive associative memory performance at 3 h after training (intermediate-term memory; ITM), anesthesia-resistant memory (ARM), and anesthesia-sensitive memory (ASM) of 3d and 30d 4xBRP flies compared to age-matched control (2xBRP) flies (<i>n</i> = 7 independent experiments; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). (n) Aversive olfactory memory performance 3 min after training (STM) higher in appl-gal4 > histone deacetylase-6 (HDAC6) RNAi in comparison to age-matched controls (<i>n</i> = 13–21; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). * <i>p</i> < 0.05, ** <i>p</i> < 0.01, *** <i>p</i> < 0.001, ns = not significant, <i>p</i> ≥ 0.05. Underlying data is shown in <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.1002563#pbio.1002563.s001" target="_blank">S1 Data</a>.</p

Ultrastructural analysis of PN-to-KC synapses within the mushroom body calyx.

<p>(a) Overview of the calyx neuropil, obtained by amalgamation of several images over a whole calyx cross-section of a 3d <i>w</i>^<i>1118</i> fly. Scale bar: 10 μm. (b–d) Higher magnification of PN boutons and dendritic claws of KCs within the calyx of 3d, 30d, and 30d^Spd <i>w</i>^<i>1118</i> flies. Scale bar: 2 μm. The asterisk indicates the PN bouton, and the arrowhead indicates the dendritic claws of KCs. (e) Quantification of AZs normalized to bouton area (1/pm²) (total of 95–103 boutons across three independent animals, with at least 25 boutons per animal; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). (f) Quantification of total SVs within a shell of 150 nm surrounding the AZ scaffold (total of 92–100 electron-micrographs across four independent animals, with at least 20 electron-micrographs per animal; Kruskal-Wallis test with Dunn’s multiple comparison test, <i>p</i>-values were subject to Bonferroni correction). (g) Quantification of SVs touching the presynaptic plasma membrane (total of 92–100 electron-micrographs across four independent animals, with at least 20 electron micrographs per animal; Kruskal-Wallis test with Dunn’s multiple comparison test, p-values were subject to Bonferroni correction). * <i>p</i> < 0.05, ** <i>p</i> < 0.01, *** <i>p</i> < 0.001, ns = not significant, <i>p</i> ≥ 0.05. Underlying data is shown in <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.1002563#pbio.1002563.s001" target="_blank">S1 Data</a>.</p