Exploring Interpretable LSTM Neural Networks over Multi-Variable Data

For recurrent neural networks trained on time series with target and exogenous variables, in addition to accurate prediction, it is also desired to provide interpretable insights into the data. In this paper, we explore the structure of LSTM recurrent neural networks to learn variable-wise hidden states, with the aim to capture different dynamics in multi-variable time series and distinguish the contribution of variables to the prediction. With these variable-wise hidden states, a mixture attention mechanism is proposed to model the generative process of the target. Then we develop associated training methods to jointly learn network parameters, variable and temporal importance w.r.t the prediction of the target variable. Extensive experiments on real datasets demonstrate enhanced prediction performance by capturing the dynamics of different variables. Meanwhile, we evaluate the interpretation results both qualitatively and quantitatively. It exhibits the prospect as an end-to-end framework for both forecasting and knowledge extraction over multi-variable data.Comment: Accepted to International Conference on Machine Learning (ICML), 201

Negative Statements Considered Useful

Knowledge bases (KBs), pragmatic collections of knowledge about notable entities, are an important asset in applications such as search, question answering and dialogue. Rooted in a long tradition in knowledge representation, all popular KBs only store positive information, while they abstain from taking any stance towards statements not contained in them. In this paper, we make the case for explicitly stating interesting statements which are not true. Negative statements would be important to overcome current limitations of question answering, yet due to their potential abundance, any effort towards compiling them needs a tight coupling with ranking. We introduce two approaches towards compiling negative statements. (i) In peer-based statistical inferences, we compare entities with highly related entities in order to derive potential negative statements, which we then rank using supervised and unsupervised features. (ii) In query-log-based text extraction, we use a pattern-based approach for harvesting search engine query logs. Experimental results show that both approaches hold promising and complementary potential. Along with this paper, we publish the first datasets on interesting negative information, containing over 1.1M statements for 100K popular Wikidata entities

Patterns of Comovement: The Role of Information Technology in the U.S. Economy

Firm-specific variation in stock returns and fundamental performance measures is significantly higher in industries that have a history of more investment in information technology (IT). We hypothesise that IT is associated with creative destruction or product differentiation, either of which can widen the performance difference between winner and loser firms. Thus, economy-level volatility can fall while firm-level volatility rises because firm-specific volatility cancels out in the aggregate. Our results are consistent with rising firm-specific variation in US stocks reflecting a rising pace of creative destruction; and with greater firm-specific variation in richer and faster growing countries reflecting more intensive creative destruction in those economies, though other explanations are probably valid as well.

Local Protectionism and Regional Specialization: Evidence from China’s Industries

This paper uses a dynamic panel estimation method to investigate the determinants of regional specialization in China’s industries, paying particular attention to local protectionism. Less geographic concentration is found in industries where the past tax-plus-profit margins and the shares of state ownership are high, re- flecting stronger local government protection of these industries. The evidence also supports the scale-economies theory of regional specialization. Finally, the overall time trend of regional specialization of China’s industries is found to have reversed an early drop in the mid 1980s, and registered a significant increase in the later years.http://deepblue.lib.umich.edu/bitstream/2027.42/39951/3/wp565.pd

Unsupervised Visual Feature Learning with Spike-timing-dependent Plasticity: How Far are we from Traditional Feature Learning Approaches?

Spiking neural networks (SNNs) equipped with latency coding and spike-timing dependent plasticity rules offer an alternative to solve the data and energy bottlenecks of standard computer vision approaches: they can learn visual features without supervision and can be implemented by ultra-low power hardware architectures. However, their performance in image classification has never been evaluated on recent image datasets. In this paper, we compare SNNs to auto-encoders on three visual recognition datasets, and extend the use of SNNs to color images. The analysis of the results helps us identify some bottlenecks of SNNs: the limits of on-center/off-center coding, especially for color images, and the ineffectiveness of current inhibition mechanisms. These issues should be addressed to build effective SNNs for image recognition

Novel characterization method of impedance cardiography signals using time-frequency distributions

The purpose of this document is to describe a methodology to select the most adequate time-frequency distribution (TFD) kernel for the characterization of impedance cardiography signals (ICG). The predominant ICG beat was extracted from a patient and was synthetized using time-frequency variant Fourier approximations. These synthetized signals were used to optimize several TFD kernels according to a performance maximization. The optimized kernels were tested for noise resistance on a clinical database. The resulting optimized TFD kernels are presented with their performance calculated using newly proposed methods. The procedure explained in this work showcases a new method to select an appropriate kernel for ICG signals and compares the performance of different time-frequency kernels found in the literature for the case of ICG signals. We conclude that, for ICG signals, the performance (P) of the spectrogram with either Hanning or Hamming windows (P¿=¿0.780) and the extended modified beta distribution (P¿=¿0.765) provided similar results, higher than the rest of analyzed kernels.Peer ReviewedPostprint (published version