Tensorized Self-Attention: Efficiently Modeling Pairwise and Global Dependencies Together

Neural networks equipped with self-attention have parallelizable computation, light-weight structure, and the ability to capture both long-range and local dependencies. Further, their expressive power and performance can be boosted by using a vector to measure pairwise dependency, but this requires to expand the alignment matrix to a tensor, which results in memory and computation bottlenecks. In this paper, we propose a novel attention mechanism called "Multi-mask Tensorized Self-Attention" (MTSA), which is as fast and as memory-efficient as a CNN, but significantly outperforms previous CNN-/RNN-/attention-based models. MTSA 1) captures both pairwise (token2token) and global (source2token) dependencies by a novel compatibility function composed of dot-product and additive attentions, 2) uses a tensor to represent the feature-wise alignment scores for better expressive power but only requires parallelizable matrix multiplications, and 3) combines multi-head with multi-dimensional attentions, and applies a distinct positional mask to each head (subspace), so the memory and computation can be distributed to multiple heads, each with sequential information encoded independently. The experiments show that a CNN/RNN-free model based on MTSA achieves state-of-the-art or competitive performance on nine NLP benchmarks with compelling memory- and time-efficiency

Analysis of significant factors on cable failure using the Cox proportional hazard model

This paper proposes the use of the Cox proportional hazard model (Cox PHM), a statistical model, for the analysis of early-failure data associated with power cables. The Cox PHM analyses simultaneously a set of covariates and identifies those which have significant effects on the cable failures. In order to demonstrate the appropriateness of the model, relevant historical failure data related to medium voltage (MV, rated at 10 kV) distribution cables and High Voltage (HV, 110 kV and 220 kV) transmission cables have been collected from a regional electricity company in China. Results prove that the model is more robust than the Weibull distribution, in that failure data does not have to be homogeneous. Results also demonstrate that the method can single out a case of poor manufacturing quality with a particular cable joint provider by using a statistical hypothesis test. The proposed approach can potentially help to resolve any legal dispute that may arise between a manufacturer and a network operator, in addition to providing guidance for improving future practice in cable procurement, design, installations and maintenance

Generating Chinese Couplets and Quatrain Using a Statistical Approach

PACLIC 23 / City University of Hong Kong / 3-5 December 200

Odorant Receptors and Odorant-Binding Proteins as Insect Pest Control Targets: A Comparative Analysis

Recently, two alternative targets in insect periphery nerve system have been explored for environmentally-friendly approaches in insect pest management, namely odorant-binding proteins (OBPs) and odorant receptors (ORs). Located in insect antennae, OBPs are thought to be involved in the transport of odorants to ORs for the specific signal transduction of behaviorally active odorants. There is rich information on OBP binding affinity and molecular docking to bioactive compounds as well as ample 3D crystal structures due to feasible production of recombinant proteins. Although these provide excellent opportunities for them to be considered as pest control targets and a tool to design pest control agents, the debates on their binding specificity represent an obstacle. On the other hand, ORs have recently been functionally characterized with increasing evidence for their specificity, sensitivity and functional roles in pest behaviors. However, a major barrier to use ORs for semiochemical discovery is the lack of 3D crystal structures. Thus, OBPs and ORs have not been analyzed comparatively together so far for their feasibility as pest control targets. Here, we summarize the state of OBPs and ORs research in terms of its application in insect pest management. We discuss the suitability of both proteins as pest control targets and their selection toward the discovery of new potent semiochemicals. We argue that both proteins represent promising targets for pest control and can be used to identify new super-ligands likely present in nature and with reduced risk of resistance development than insect pesticides currently used in agriculture. We discuss that with the massive identification of OBPs through RNA-seq and improved binding affinity measurements, these proteins could be reconsidered as suitable targets for semiochemical discovery

Neural differences between the processing of musical meaning conveyed by direction of pitch change and natural music in congenital amusia

Music is a unique communication system for human beings. Iconic musical meaning is one dimension of musical meaning, which emerges from musical information resembling sounds of objects, qualities of objects, or qualities of abstract concepts. The present study investigated whether congenital amusia, a disorder of musical pitch perception, impacts processing of iconic musical meaning. With a cross-modal semantic priming paradigm, target images were primed by semantically congruent or incongruent musical excerpts, which were characterized by direction (upward or downward) of pitch change (Experiment 1), or were selected from natural music (Experiment 2). Twelve Mandarin-speaking amusics and 12 controls performed a recognition (implicit) and a semantic congruency judgment (explicit) task while their EEG waveforms were recorded. Unlike controls, amusics failed to elicit an N400 effect when musical meaning was represented by direction of pitch change, regardless of the nature of the tasks (implicit versus explicit). However, the N400 effect in response to musical meaning in natural musical excerpts was observed for both groups in both types of tasks. These results indicate that amusics are able to process iconic musical meaning through multiple acoustic cues in natural musical excerpts, but not through direction of pitch change. This is the first study to investigate the processing of musical meaning in congenital amusia, providing evidence in support of the “melodic contour deafness hypothesis” with regard to iconic musical meaning processing in this disorder

Unique Features of Odorant-Binding Proteins of the Parasitoid Wasp Nasonia vitripennis Revealed by Genome Annotation and Comparative Analyses

Insects are the most diverse group of animals on the planet, comprising over 90% of all metazoan life forms, and have adapted to a wide diversity of ecosystems in nearly all environments. They have evolved highly sensitive chemical senses that are central to their interaction with their environment and to communication between individuals. Understanding the molecular bases of insect olfaction is therefore of great importance from both a basic and applied perspective. Odorant binding proteins (OBPs) are some of most abundant proteins found in insect olfactory organs, where they are the first component of the olfactory transduction cascade, carrying odorant molecules to the olfactory receptors. We carried out a search for OBPs in the genome of the parasitoid wasp Nasonia vitripennis and identified 90 sequences encoding putative OBPs. This is the largest OBP family so far reported in insects. We report unique features of the N. vitripennis OBPs, including the presence and evolutionary origin of a new subfamily of double-domain OBPs (consisting of two concatenated OBP domains), the loss of conserved cysteine residues and the expression of pseudogenes. This study also demonstrates the extremely dynamic evolution of the insect OBP family: (i) the number of different OBPs can vary greatly between species; (ii) the sequences are highly diverse, sometimes as a result of positive selection pressure with even the canonical cysteines being lost; (iii) new lineage specific domain arrangements can arise, such as the double domain OBP subfamily of wasps and mosquitoes.Rothamsted Research receives grant-aided support from the BBSRC of the UK. The authors thank Prof. David M. Shuker, University of Edinburgh, UK, who provided us with N. vitripennis. FGV was supported by a predoctoral fellowship SFRH/BD/22360/2005 from the ‘Fundac¸a˜o para a Cieˆncia e a Tecnologı´a’ (Portugal). This work was funded by grants BFU2007-62927 and BFU2010-15484 from the ‘Direccio´n General de Investigacio´n Cientı´fica y Te´cnica’ (Spain) to JR. JR was partially supported by ICREA Academia (Generalitat de Catalunya). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript

Influence of Fermion Velocity Renormalization on Dynamical Mass Generation in QED3_3

We study dynamical fermion mass generation in (2+1)-dimensional quantum electrodynamics with a gauge field coupling to massless Dirac fermions and non-relativistic scalar bosons. We calculate the fermion velocity renormalization and then examine its influence on dynamical mass generation by using the Dyson-Schwinger equation. It is found that dynamical mass generation takes place even after including the scalar bosons as long as the bosonic compressibility parameter $\xi$ is sufficiently small. In addition, the fermion velocity renormalization enhances the dynamically generated mass.Comment: 6 pages, 3 figures, Chinese Physics Letter, Vol 29, page 057401(2012