THE INFLUENCE OF ZINC STATUS ON AKT SIGNALING PATHWAY IN HUMAN NORMAL PROSTATE EPITHELIAL CELLS AND HUMAN MALIGNANT PROSTATE CELLS

Akt is known for promoting tumorigenesis through cellular proliferation. Supra-physiologic levels of zinc has been shown to stimulate the phosphorylation of Akt (p-Akt), which is frequently detectable in prostate tumors. Zinc content of malignant prostate epithelial cells is substantially lower than that of the surrounding normal epithelial cells. The influence of physiologic level of zinc on cell cycle progression via phosphoinositide-3-OH-kinase (PI3K)/Akt signaling pathway was examined in human normal prostate epithelial cells (PrEC) and human prostate malignant LNCaP cells. These cells were selected because of their susceptibility to zinc uptake and ability to express wild-type phosphatase and tensin homolog (PTEN) gene, the tumor suppressor responsible for blocking PI3K/Akt signaling. As a downstream effector of Akt, Mdm2 can be phosphorylated and translocated into the nucleus, subsequently promoting the ubiquitin-dependent degradation of tumor suppressor p53 protein. p-Akt can also affect cell cycle progression by phosphorylating p21, which restricts p21's nuclear entry to induce cell cycle arrest. Cells were cultured for 6 d in low-zinc growth medium added with 0 (zinc-deficient; ZD), 4 (zinc-normal; ZN), 16 (zinc-adequate; ZA), or 32 (zinc-supplemented; ZS) micromol/L of zinc. The effects of zinc on intracellular zinc status and cell cycle progression were determined by atomic absorption spectrophotometry and flow cytometry, respectively. Cytoplasmic and nuclear levels of p-Akt, p-PTEN, p-Mdm2, p53, and p21 proteins were analyzed by Western blotting. In addition, the dependence of zinc-induced Akt phosphorylation on the modulation of p-Akt, p-Mdm2, p53, and p21 protein levels was ascertained by using a PI3K/Akt inhibitor LY294002. Cellular zinc status of PrEC was more readily altered in a dose-dependent manner than LNCaP cells. In both cells, p-Akt was higher in ZD than ZN cells and both levels were normalized to that of ZN cells by LY294002. p-PTEN was higher in ZD than ZN-PrEC. Nuclear p-Mdm2 was higher in PrEC, while nuclear p53 was depressed in both PrEC and LNCaP cells by zinc deficiency. Nuclear p21 was unaffected in ZD-PrEC, but it was depressed in ZD-LNCaP cells. Nuclear p21 was higher in ZA and ZS than ZN-PrEC which coincided with faster G2/M progression. With LY294002, nuclear p21 protein was elevated in all groups, which correlated with an inhibition of G1/S cell cycle progression. Hence, zinc may affect cell cycle through Akt-Mdm2-p53 signaling axis in normal versus Akt-p21 in malignant prostate cells

Advancing Regular Language Reasoning in Linear Recurrent Neural Networks

In recent studies, linear recurrent neural networks (LRNNs) have achieved Transformer-level performance in natural language modeling and long-range modeling while offering rapid parallel training and constant inference costs. With the resurged interest in LRNNs, we study whether they can learn the hidden rules in training sequences, such as the grammatical structures of regular language. We theoretically analyze some existing LRNNs and discover their limitations on regular language. Motivated by the analysis, we propose a new LRNN equipped with a block-diagonal and input-dependent transition matrix. Experiments suggest that the proposed model is the only LRNN that can perform length extrapolation on regular language tasks such as Sum, Even Pair, and Modular Arithmetic.Comment: The first two authors contributed equally to this wor

Attention Alignment and Flexible Positional Embeddings Improve Transformer Length Extrapolation

An ideal length-extrapolatable Transformer language model can handle sequences longer than the training length without any fine-tuning. Such long-context utilization capability relies heavily on a flexible positional embedding design. Upon investigating the flexibility of existing large pre-trained Transformer language models, we find that the T5 family deserves a closer look, as its positional embeddings capture rich and flexible attention patterns. However, T5 suffers from the dispersed attention issue: the longer the input sequence, the flatter the attention distribution. To alleviate the issue, we propose two attention alignment strategies via temperature scaling. Our findings show improvement on the long-context utilization capability of T5 on language modeling, retrieval, multi-document question answering, and code completion tasks without any fine-tuning. This suggests that a flexible positional embedding design and attention alignment can go a long way toward Transformer length extrapolation

A Local-Pattern Related Look-Up Table

This paper describes a Relevance-Zone pattern table (RZT) that can be used to replace a traditional transposition table. An RZT stores exact game values for patterns that are discovered during a Relevance-Zone-Based Search (RZS), which is the current state-of-the-art in solving L&D problems in Go. Positions that share the same pattern can reuse the same exact game value in the RZT. The pattern matching scheme for RZTs is implemented using a radix tree, taking into consideration patterns with different shapes. To improve the efficiency of table lookups, we designed a heuristic that prevents redundant lookups. The heuristic can safely skip previously queried patterns for a given position, reducing the overhead to 10% of the original cost. We also analyze the time complexity of the RZT both theoretically and empirically. Experiments show the overhead of traversing the radix tree in practice during lookup remain flat logarithmically in relation to the number of entries stored in the table. Experiments also show that the use of an RZT instead of a traditional transposition table significantly reduces the number of searched nodes on two data sets of 7x7 and 19x19 L&D Go problems.Comment: Submitted to IEEE Transactions on Games (under review

Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings

The use of positional embeddings in transformer language models is widely accepted. However, recent research has called into question the necessity of such embeddings. We further extend this inquiry by demonstrating that a randomly initialized and frozen transformer language model, devoid of positional embeddings, inherently encodes strong positional information through the shrinkage of self-attention variance. To quantify this variance, we derive the underlying distribution of each step within a transformer layer. Through empirical validation using a fully pretrained model, we show that the variance shrinkage effect still persists after extensive gradient updates. Our findings serve to justify the decision to discard positional embeddings and thus facilitate more efficient pretraining of transformer language models.Comment: Accepted by ACL 202

Algorithmic Views of Vectorized Polynomial Multipliers for NTRU and NTRU Prime (Long Paper)

This paper explores the design space of vector-optimized polynomial multiplications in the lattice-based key-encapsulation mechanisms NTRU and NTRU Prime. Since NTRU and NTRU Prime do not support straightforward applications of number– theoretic transforms, the state-of-the-art vector code either resorted to Toom–Cook, or introduced various techniques for coefficient ring extensions. All these techniques lead to a large number of small-degree polynomial multiplications, which is the bottleneck in our experiments. For NTRU Prime, we show how to reduce the number of small-degree polynomial multiplications to nearly 1/4 times compared to the previous vectorized code with the same functionality. Our transformations are based on careful choices of FFTs, including Good–Thomas, Rader’s, Schönhage’s, and Bruun’s FFTs. For NTRU, we show how to deploy Toom-5 with 3-bit losses. Furthermore, we show that the Toeplitz matrix–vector product naturally translates into efficient implementations with vector-by-scalar multiplication instructions which do not appear in all prior vector-optimized implementations. We choose the ARM Cortex-A72 CPU which implements the Armv8-A architecture for experiments, because of its wide uses in smartphones, and also the Neon vector instruction set implementing vector-by-scalar multiplications that do not appear in most other vector instruction sets like Intel’s AVX2. Even for platforms without vector-by-scalar multiplications, we expect significant improvements compared to the state of the art, since our transformations reduce the number of multiplication instructions by a large margin. Compared to the state-of-the-art optimized implementations, we achieve 2.18× and 6.7× faster polynomial multiplications for NTRU and NTRU Prime, respectively. For full schemes, we additionally vectorize the polynomial inversions, sorting network, and encoding/decoding subroutines in NTRU and NTRU Prime. For ntruhps2048677, we achieve 7.67×, 2.48×, and 1.77× faster key generation, encapsulation, and decapsulation, respectively. For ntrulpr761, we achieve 3×, 2.87×, and 3.25× faster key generation, encapsulation, and decapsulation, respectively. For sntrup761, there are no previously optimized implementations and we significantly outperform the reference implementation

Game Solving with Online Fine-Tuning

Game solving is a similar, yet more difficult task than mastering a game. Solving a game typically means to find the game-theoretic value (outcome given optimal play), and optionally a full strategy to follow in order to achieve that outcome. The AlphaZero algorithm has demonstrated super-human level play, and its powerful policy and value predictions have also served as heuristics in game solving. However, to solve a game and obtain a full strategy, a winning response must be found for all possible moves by the losing player. This includes very poor lines of play from the losing side, for which the AlphaZero self-play process will not encounter. AlphaZero-based heuristics can be highly inaccurate when evaluating these out-of-distribution positions, which occur throughout the entire search. To address this issue, this paper investigates applying online fine-tuning while searching and proposes two methods to learn tailor-designed heuristics for game solving. Our experiments show that using online fine-tuning can solve a series of challenging 7x7 Killall-Go problems, using only 23.54% of computation time compared to the baseline without online fine-tuning. Results suggest that the savings scale with problem size. Our method can further be extended to any tree search algorithm for problem solving. Our code is available at https://rlg.iis.sinica.edu.tw/papers/neurips2023-online-fine-tuning-solver.Comment: Accepted by the 37th Conference on Neural Information Processing Systems (NeurIPS 2023

On the Impossibility of General Parallel Fast-Forwarding of Hamiltonian Simulation

Hamiltonian simulation is one of the most important problems in the field of quantum computing. There have been extended efforts on designing algorithms for faster simulation, and the evolution time T for the simulation greatly affect algorithm runtime as expected. While there are some specific types of Hamiltonians that can be fast-forwarded, i.e., simulated within time o(T), for some large classes of Hamiltonians (e.g., all local/sparse Hamiltonians), existing simulation algorithms require running time at least linear in the evolution time T. On the other hand, while there exist lower bounds of ?(T) circuit size for some large classes of Hamiltonian, these lower bounds do not rule out the possibilities of Hamiltonian simulation with large but "low-depth" circuits by running things in parallel. As a result, physical systems with system size scaling with T can potentially do a fast-forwarding simulation. Therefore, it is intriguing whether we can achieve fast Hamiltonian simulation with the power of parallelism. In this work, we give a negative result for the above open problem in various settings. In the oracle model, we prove that there are time-independent sparse Hamiltonians that cannot be simulated via an oracle circuit of depth o(T). In the plain model, relying on the random oracle heuristic, we show that there exist time-independent local Hamiltonians and time-dependent geometrically local Hamiltonians on n qubits that cannot be simulated via an oracle circuit of depth o(T/n^c), where the Hamiltonians act on n qubits, and c is a constant. Lastly, we generalize the above results and show that any simulators that are geometrically local Hamiltonians cannot do the simulation much faster than parallel quantum algorithms