Recurrent Memory Transformer

Transformer-based models show their effectiveness across multiple domains and tasks. The self-attention allows to combine information from all sequence elements into context-aware representations. However, global and local information has to be stored mostly in the same element-wise representations. Moreover, the length of an input sequence is limited by quadratic computational complexity of self-attention. In this work, we propose and study a memory-augmented segment-level recurrent Transformer (Recurrent Memory Transformer). Memory allows to store and process local and global information as well as to pass information between segments of the long sequence with the help of recurrence. We implement a memory mechanism with no changes to Transformer model by adding special memory tokens to the input or output sequence. Then Transformer is trained to control both memory operations and sequence representations processing. Results of experiments show that our model performs on par with the Transformer-XL on language modeling for smaller memory sizes and outperforms it for tasks that require longer sequence processing. We show that adding memory tokens to Tr-XL is able to improve it performance. This makes Recurrent Memory Transformer a promising architecture for applications that require learning of long-term dependencies and general purpose in memory processing, such as algorithmic tasks and reasoning

Scaling Transformer to 1M tokens and beyond with RMT

This technical report presents the application of a recurrent memory to extend the context length of BERT, one of the most effective Transformer-based models in natural language processing. By leveraging the Recurrent Memory Transformer architecture, we have successfully increased the model's effective context length to an unprecedented two million tokens, while maintaining high memory retrieval accuracy. Our method allows for the storage and processing of both local and global information and enables information flow between segments of the input sequence through the use of recurrence. Our experiments demonstrate the effectiveness of our approach, which holds significant potential to enhance long-term dependency handling in natural language understanding and generation tasks as well as enable large-scale context processing for memory-intensive applications

High-K volcanism in the Afyon region, western Turkey: from Si-oversaturated to Si-undersaturated volcanism

Volcanic rocks of the Afyon province (eastern part of western Anatolia) make up a multistage potassic and ultrapotassic alkaline series dated from 14 to 12 Ma. The early-stage Si-oversaturated volcanic rocks around the Afyon city and further southward are trachyandesitic volcanic activity (14.23 ± 0.09 Ma). Late-stage Si-undersaturated volcanism in the southernmost part of the Afyon volcanic province took place in three episodes inferred from their stratigraphic relationships and ages. Melilite– leucitites (11.50 ± 0.03 Ma), spotted rachyandesites, tephryphonolites and lamproites (11.91 ± 0.13 Ma) formed in the first episode; trachyandesites in the second episode and finally phonotephrites, phonolite, basaltic trachyandesites and nosean-bearing trachyandesites during the last episode. The parameter Q [normative q-(ne + lc + kls + ol)] of western Anatolia volcanism clearly decreased southward with time becoming zero in the time interval 10–15 Ma. The magmatism experienced a sudden change in the extent of Si saturation after 14 Ma, during late-stage volcanic activity of Afyon volcanic province at around 12 Ma, though there was some coexistence of Si-oversaturated and Si-undersaturated magmas during the whole life of Afyon volcanic province

hERG potassium channel gating is mediated by N- and C-terminal region interactions

Human ether-á-go-go–related gene (hERG) potassium channels have voltage-dependent closing (deactivation) kinetics that are unusually slow. A Per-Arnt-Sim (PAS) domain in the cytoplasmic N-terminal region of hERG regulates slow deactivation by making a direct interaction with another part of the hERG channel. The mechanism for slow deactivation is unclear, however, because the other regions of the channel that participate in regulation of deactivation are not known. To identify other functional determinants of slow deactivation, we generated hERG channels with deletions of the cytoplasmic C-terminal regions. We report that hERG channels with deletions of the cyclic nucleotide–binding domain (CNBD) had accelerated deactivation kinetics that were similar to those seen in hERG channels lacking the PAS domain. Channels with dual deletions of the PAS domain and the CNBD did not show further acceleration in deactivation, indicating that the PAS domain and the CNBD regulate deactivation by a convergent mechanism. A recombinant PAS domain that we previously showed could directly regulate PAS domain–deleted channels did not regulate channels with dual deletions of the PAS domain and CNBD, suggesting that the PAS domain did not interact with CNBD-deleted channels. Biochemical protein interaction assays showed that glutathione S-transferase (GST)–PAS (but not GST) bound to a CNBD-containing fusion protein. Coexpression of PAS domain–deleted subunits (with intact C-terminal regions) and CNBD-deleted subunits (with intact N-terminal regions) resulted in channels with partially restored slow deactivation kinetics, suggesting regulatory intersubunit interactions between PAS domains and CNBDs. Together, these data suggest that the mechanism for regulation of slow deactivation in hERG channels is an interaction between the N-terminal PAS domain and the C-terminal CNBD

TLD1433 photosensitizer inhibits conjunctival melanoma cells in zebrafish ectopic and orthotopic tumour models

The ruthenium-based photosensitizer (PS) TLD1433 has completed a phase I clinical trial for photodynamic therapy (PDT) treatment of bladder cancer. Here, we investigated a possible repurposing of this drug for treatment of conjunctival melanoma (CM). CM is a rare but often deadly ocular cancer. The efficacy of TLD1433 was tested on several cell lines from CM (CRMM1, CRMM2 and CM2005), uveal melanoma (OMM1, OMM2.5, MEL270), epidermoid carcinoma (A431) and cutaneous melanoma (A375). Using 15 min green light irradiation (21mW/cm(2), 19 J.cm(-2), 520 nm), the highest phototherapeutic index (PI) was reached in CM cells, with cell death occurring via apoptosis and necrosis. The therapeutic potential of TLD1433 was hence further validated in zebrafish ectopic and newly-developed orthotopic CM models. Fluorescent CRMM1 and CRMM2 cells were injected into the circulation of zebrafish (ectopic model) or behind the eye (orthotopic model) and 24 h later, the engrafted embryos were treated with the maximally-tolerated dose of TLD1433. The drug was administrated in three ways, either by (i) incubating the fish in drug-containing water (WA), or (ii) injecting the drug intravenously into the fish (IV), or (iii) injecting the drug retro-orbitally (RO) into the fish. Optimally, four consecutive PDT treatments were performed on engrafted embryos using 60 min drug-to-light intervals and 90 min green light irradiation (21 mW/cm(2), 114 J.cm(-2), 520 nm). This PDT protocol was not toxic to the fish. In the ectopic tumour model, both systemic administration by IV injection and RO injection of TLD1433 significantly inhibited growth of engrafted CRMM1 and CRMM2 cells. However, in the orthotopic model, tumour growth was only attenuated by localized RO injection of TLD1433. These data unequivocally prove that the zebrafish provides a fast vertebrate cancer model that can be used to test the administration regimen, host toxicity and anti-cancer efficacy of PDT drugs against CM. Based on our results, we suggest repurposing of TLD1433 for treatment of incurable CM and further testing in alternative pre-clinical models.Animal science

Trafficking Defect and Proteasomal Degradation Contribute to the Phenotype of a Novel KCNH2 Long QT Syndrome Mutation

The Kv11.1 (hERG) K+ channel plays a fundamental role in cardiac repolarization. Missense mutations in KCNH2, the gene encoding Kv11.1, cause long QT syndrome (LQTS) and frequently cause channel trafficking-deficiencies. This study characterized the properties of a novel KCNH2 mutation discovered in a LQT2 patient resuscitated from a ventricular fibrillation arrest. Proband genotyping was performed by SSCP and DNA sequencing. The electrophysiological and biochemical properties of the mutant channel were investigated after expression in HEK293 cells. The proband manifested a QTc of 554 ms prior to electrolyte normalization. Mutation analysis revealed an autosomal dominant frameshift mutation at proline 1086 (P1086fs+32X; 3256InsG). Co-immunoprecipitation demonstrated that wild-type Kv11.1 and mutant channels coassemble. Western blot showed that the mutation did not produce mature complex-glycosylated Kv11.1 channels and coexpression resulted in reduced channel maturation. Electrophysiological recordings revealed mutant channel peak currents to be similar to untransfected cells. Co-expression of channels in a 1∶1 ratio demonstrated dominant negative suppression of peak Kv11.1 currents. Immunocytochemistry confirmed that mutant channels were not present at the plasma membrane. Mutant channel trafficking rescue was attempted by incubation at reduced temperature or with the pharmacological agents E-4031. These treatments did not significantly increase peak mutant currents or induce the formation of mature complex-glycosylated channels. The proteasomal inhibitor lactacystin increased the protein levels of the mutant channels demonstrating proteasomal degradation, but failed to induce mutant Kv11.1 protein trafficking. Our study demonstrates a novel dominant-negative Kv11.1 mutation, which results in degraded non-functional channels leading to a LQT2 phenotype

Trafficking Defect and Proteasomal Degradation Contribute to the Phenotype of a Novel KCNH2 Long QT Syndrome Mutation

The Kv11.1 (hERG) K+ channel plays a fundamental role in cardiac repolarization. Missense mutations in KCNH2, the gene encoding Kv11.1, cause long QT syndrome (LQTS) and frequently cause channel trafficking-deficiencies. This study characterized the properties of a novel KCNH2 mutation discovered in a LQT2 patient resuscitated from a ventricular fibrillation arrest. Proband genotyping was performed by SSCP and DNA sequencing. The electrophysiological and biochemical properties of the mutant channel were investigated after expression in HEK293 cells. The proband manifested a QTc of 554 ms prior to electrolyte normalization. Mutation analysis revealed an autosomal dominant frameshift mutation at proline 1086 (P1086fs+32X; 3256InsG). Co-immunoprecipitation demonstrated that wild-type Kv11.1 and mutant channels coassemble. Western blot showed that the mutation did not produce mature complex-glycosylated Kv11.1 channels and coexpression resulted in reduced channel maturation. Electrophysiological recordings revealed mutant channel peak currents to be similar to untransfected cells. Co-expression of channels in a 1∶1 ratio demonstrated dominant negative suppression of peak Kv11.1 currents. Immunocytochemistry confirmed that mutant channels were not present at the plasma membrane. Mutant channel trafficking rescue was attempted by incubation at reduced temperature or with the pharmacological agents E-4031. These treatments did not significantly increase peak mutant currents or induce the formation of mature complex-glycosylated channels. The proteasomal inhibitor lactacystin increased the protein levels of the mutant channels demonstrating proteasomal degradation, but failed to induce mutant Kv11.1 protein trafficking. Our study demonstrates a novel dominant-negative Kv11.1 mutation, which results in degraded non-functional channels leading to a LQT2 phenotype

Investigation of oncolytic potential of vaccine strains of yellow fever and tick-borne encephalitis viruses against glioblastoma and pancreatic carcinoma cell lines

Introduction. Flaviviruses, possessing natural neurotropicity could be used in glioblastoma therapy using attenuated strains or as a delivery system for antitumor agents in an inactivated form. Objective. To investigate the sensitivity of glioblastoma and pancreatic carcinoma cell lines to vaccine strains of yellow fever and tick-borne encephalitis viruses. Materials and methods. Cell lines: glioblastoma GL-6, T98G, LN-229, pancreatic carcinoma MIA RaCa-2 and human pancreatic ductal carcinoma PANC-1. Viral strains: 17D yellow fever virus (YF), Sofjin tick-borne encephalitis virus (TBEV). Virus concentration were determined by plaque assay and quantitative PCR. Determination of cell sensitivity to viruses by MTT assay. Results. 17D YF was effective only against pancreatic carcinoma tumor cells MIA Paca-2 and had a limited effect against PANC-1. In glioblastoma cell lines (LN229, GL6, T98G), virus had no oncolytic effect and the viral RNA concentration fell in the culture medium. Sofjin TBEV showed CPE50 against MIA Paca-2 and a very limited cytotoxic effect against PANC-1. However, it had no oncolytic effect against glioblastoma cell lines (LN229, T98G and GL6), although virus reproduction continued in these cultures. For the GL6 glioblastoma cell line, the viral RNA concentration at the level with the infection dose was determined within 13 days, despite medium replacement, while in the case of the LN229 cell line, the virus concentration increased from 1 × 109 to 1 × 1010 copies/ml. Conclusion. Tumor behavior in organism is more complex and is determined by different microenvironmental factors and immune status. In the future, it is advisable to continue studying the antitumor oncolytic and immunomodulatory effects of viral strains 17D YF and Sofjin TBEV using in vivo models

Transient Receptor Potential (TRP) and Cch1-Yam8 Channels Play Key Roles in the Regulation of Cytoplasmic Ca2+ in Fission Yeast

The regulation of cytoplasmic Ca2+ is crucial for various cellular processes. Here, we examined the cytoplasmic Ca2+ levels in living fission yeast cells by a highly sensitive bioluminescence resonance energy transfer-based assay using GFP-aequorin fusion protein linked by 19 amino acid. We monitored the cytoplasmic Ca2+ level and its change caused by extracellular stimulants such as CaCl2 or NaCl plus FK506 (calcineurin inhibitor). We found that the extracellularly added Ca2+ caused a dose-dependent increase in the cytoplasmic Ca2+ level and resulted in a burst-like peak. The overexpression of two transient receptor potential (TRP) channel homologues, Trp1322 or Pkd2, markedly enhanced this response. Interestingly, the burst-like peak upon TRP overexpression was completely abolished by gene deletion of calcineurin and was dramatically decreased by gene deletion of Prz1, a downstream transcription factor activated by calcineurin. Furthermore, 1 hour treatment with FK506 failed to suppress the burst-like peak. These results suggest that the burst-like Ca2+ peak is dependent on the transcriptional activity of Prz1, but not on the direct TRP dephosphorylation. We also found that extracellularly added NaCl plus FK506 caused a synergistic cytosolic Ca2+ increase that is dependent on the inhibition of calcineurin activity, but not on the inhibition of Prz1. The synergistic Ca2+ increase is abolished by the addition of the Ca2+ chelator BAPTA into the media, and is also abolished by deletion of the gene encoding a subunit of the Cch1-Yam8 Ca2+ channel complex, indicating that the synergistic increase is caused by the Ca2+ influx from the extracellular medium via the Cch1-Yam8 complex. Furthermore, deletion of Pmk1 MAPK abolished the Ca2+ influx, and overexpression of the constitutively active Pek1 MAPKK enhanced the influx. These results suggest that Pmk1 MAPK and calcineurin positively and negatively regulate the Cch1-Yam8 complex, respectively, via modulating the balance between phosphorylation and dyphosphorylation state