What does Attention in Neural Machine Translation Pay Attention to?

Attention in neural machine translation provides the possibility to encode relevant parts of the source sentence at each translation step. As a result, attention is considered to be an alignment model as well. However, there is no work that specifically studies attention and provides analysis of what is being learned by attention models. Thus, the question still remains that how attention is similar or different from the traditional alignment. In this paper, we provide detailed analysis of attention and compare it to traditional alignment. We answer the question of whether attention is only capable of modelling translational equivalent or it captures more information. We show that attention is different from alignment in some cases and is capturing useful information other than alignments.Comment: To appear in IJCNLP 201

SVITE: A Spike-Based VITE Neuro-Inspired Robot Controller

This paper presents an implementation of a neuro-inspired algorithm called VITE (Vector Integration To End Point) in FPGA in the spikes domain. VITE aims to generate a non-planned trajectory for reaching tasks in robots. The algorithm has been adapted to work completely in the spike domain under Simulink simulations. The FPGA implementation consists in 4 VITE in parallel for controlling a 4-degree-of-freedom stereo-vision robot. This work represents the main layer of a complex spike-based architecture for robot neuro-inspired reaching tasks in FPGAs. It has been implemented in two Xilinx FPGA families: Virtex-5 and Spartan-6. Resources consumption comparative between both devices is presented. Results obtained for Spartan device could allow controlling complex robotic structures with up to 96 degrees of freedom per FPGA, providing, in parallel, high speed connectivity with other neuromorphic systems sending movement references. An exponential and gamma distribution test over the inter spike interval has been performed to proof the approach to the neural code proposed.Ministerio de Economía y Competitividad TEC2012-37868-C04-0

A Century of American Economic Review

Using information collected from American Economic Review publications of the last 100 years, we try to provide answers to various questions: Which are the top AER publishing institutions and countries? Which are the top AER papers based on citation success? How frequently is someone able to publish in AER? How equally is citation success distributed? Who are the top AER publishing authors? What is the level of cooperation among the authors? What drives the alphabetical name ordering? What are the individual characteristics of the AER authors, editors, editorial board members, and referees? How frequently do women publish in AER? What is the relationship between academic age, publication performance, and citation success? What are the paper characteristics? What influences the level of technique used in articles? Do connections have an influence on citation success? Who receives awards? Can awards increase the probability of publishing in AER at a later stage?American Economic Review, Publishing Economics, Rankings, Cooperation, Authors, Editors, Board Members, Referees, Connections, Awards, Paper Characteristics, Economic History, History of Economic Thought

A Century of American Economic Review

Using information collected from American Economic Review publications of the last 100 years, we try to provide answers to various questions: Which are the top AER publishing institutions and countries? Which are the top AER papers based on citation success? How frequently is someone able to publish in AER? How equally is citation success distributed? Who are the top AER publishing authors? What is the level of cooperation among the authors? What drives the alphabetical name ordering? What are the individual characteristics of the AER authors, editors, editorial board members, and referees? How frequently do women publish in AER? What is the relationship between academic age, publication performance, and citation success? What are the paper characteristics? What influences the level of technique used in articles? Do connections have an influence on citation success? Who receives awards? Can awards increase the probability of publishing in AER at a later stage?American Economic Review, publishing economics, rankings, cooperation,authors, editors, board members, referees, connections, awards, paper characteristics, economic history, history of economic thoug

Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings

In neural machine translation, a source sequence of words is encoded into a vector from which a target sequence is generated in the decoding phase. Differently from statistical machine translation, the associations between source words and their possible target counterparts are not explicitly stored. Source and target words are at the two ends of a long information processing procedure, mediated by hidden states at both the source encoding and the target decoding phases. This makes it possible that a source word is incorrectly translated into a target word that is not any of its admissible equivalent counterparts in the target language. In this paper, we seek to somewhat shorten the distance between source and target words in that procedure, and thus strengthen their association, by means of a method we term bridging source and target word embeddings. We experiment with three strategies: (1) a source-side bridging model, where source word embeddings are moved one step closer to the output target sequence; (2) a target-side bridging model, which explores the more relevant source word embeddings for the prediction of the target sequence; and (3) a direct bridging model, which directly connects source and target word embeddings seeking to minimize errors in the translation of ones by the others. Experiments and analysis presented in this paper demonstrate that the proposed bridging models are able to significantly improve quality of both sentence translation, in general, and alignment and translation of individual source words with target words, in particular.Comment: 9 pages, 6 figures. Accepted by ACL201

Inter-spikes-intervals exponential and gamma distributions study of neuron firing rate for SVITE motor control model on FPGA

This paper presents a statistical study on a neuro-inspired spike-based implementation of the Vector-Integration-To-End-Point motor controller (SVITE) and compares its deterministic neuron-model stream of spikes with a proposed modification that converts the model, and thus the controller, in a Poisson like spike stream distribution. A set of hardware pseudo-random numbers generators, based on a Linear Feedback Shift Register (LFSR), have been introduced in the neuron-model so that they reach a closer biological neuron behavior. To validate the new neuron-model behavior a comparison between the Inter-Spikes-Interval empirical data and the Exponential and Gamma distributions has been carried out using the Kolmogorov–Smirnoff test. An in-hardware validation of the controller has been performed in a Spartan6 FPGA to drive directly with spikes DC motors from robotics to study the behavior and viability of the modified controller with random components. The results show that the original deterministic spikes distribution of the controller blocks can be swapped with Poisson distributions using 30-bit LFSRs. The comparative between the usable controlling signals such as the trajectory and the speed profile using a deterministic and the new controller show a standard deviation of 11.53 spikes/s and 3.86 spikes/s respectively. These rates do not affect our system because, within Pulse Frequency Modulation, in order to drive the motors, time length can be fixed to spread the spikes. Tuning this value, the slow rates could be filtered by the motor. Therefore, this SVITE neuro-inspired controller can be integrated within complex neuromorphic architectures with Poisson-like neurons

Braneworld stars and black holes

We look for spherically symmetric star or black hole solutions on a Randall-Sundrum brane from the perspective of the bulk. We take a known bulk solution, and analyse possible braneworld trajectories within it that correspond, from the braneworld point of view, to solutions of the brane Tolman-Oppenheimer-Volkoff equations. Our solutions are therefore embedded consistently into a full bulk solution. We find the full set of static gravitating matter sources on a brane in a range of bulk spacetimes, analyzing which can correspond to physically sensible sources. Finally, we look at time-dependent trajectories in a Schwarzschild--anti de Sitter spacetime as possible descriptions of time-dependent braneworld black holes, highlighting some of the general features one might expect, as well as some of the difficulties involved in getting a full solution to the question.Comment: 39 pages, 15 figure

Regional Trade Agreements and U.S. Agriculture

Regional trade agreements (RTA's) have become a fixture in the global trade arena. Their advocates contend that RTA's can serve as building blocks for multilateral trade liberalization. Their opponents argue that these trade pacts will divert trade from more efficient nonmember producing countries. U.S. agriculture can benefit from participating in RTA's and may lose when it does not. Agriculture is an important source of potential U.S. gains from RTA's. While the United States, as a global trader with diverse trade partners, can gain potentially more from global free trade than from RTA's, many recent RTA's have been more comprehensive in their liberalization of agricultural trade liberalization than the Uruguay Round. A strong multilateral process can help ensure that RTA's are trade creating, rather than protectionist.International Relations/Trade,