Towards better understanding of gradient-based attribution methods for Deep Neural Networks

Understanding the flow of information in Deep Neural Networks (DNNs) is a challenging problem that has gain increasing attention over the last few years. While several methods have been proposed to explain network predictions, there have been only a few attempts to compare them from a theoretical perspective. What is more, no exhaustive empirical comparison has been performed in the past. In this work, we analyze four gradient-based attribution methods and formally prove conditions of equivalence and approximation between them. By reformulating two of these methods, we construct a unified framework which enables a direct comparison, as well as an easier implementation. Finally, we propose a novel evaluation metric, called Sensitivity-n and test the gradient-based attribution methods alongside with a simple perturbation-based attribution method on several datasets in the domains of image and text classification, using various network architectures.Comment: ICLR 201

On the Decoding of Polar Codes on Permuted Factor Graphs

Polar codes are a channel coding scheme for the next generation of wireless communications standard (5G). The belief propagation (BP) decoder allows for parallel decoding of polar codes, making it suitable for high throughput applications. However, the error-correction performance of polar codes under BP decoding is far from the requirements of 5G. It has been shown that the error-correction performance of BP can be improved if the decoding is performed on multiple permuted factor graphs of polar codes. However, a different BP decoding scheduling is required for each factor graph permutation which results in the design of a different decoder for each permutation. Moreover, the selection of the different factor graph permutations is at random, which prevents the decoder to achieve a desirable error-correction performance with a small number of permutations. In this paper, we first show that the permutations on the factor graph can be mapped into suitable permutations on the codeword positions. As a result, we can make use of a single decoder for all the permutations. In addition, we introduce a method to construct a set of predetermined permutations which can provide the correct codeword if the decoding fails on the original permutation. We show that for the 5G polar code of length $1024$, the error-correction performance of the proposed decoder is more than $0.25$ dB better than that of the BP decoder with the same number of random permutations at the frame error rate of $10^{-4}$

Rate-Flexible Fast Polar Decoders

Polar codes have gained extensive attention during the past few years and recently they have been selected for the next generation of wireless communications standards (5G). Successive-cancellation-based (SC-based) decoders, such as SC list (SCL) and SC flip (SCF), provide a reasonable error performance for polar codes at the cost of low decoding speed. Fast SC-based decoders, such as Fast-SSC, Fast-SSCL, and Fast-SSCF, identify the special constituent codes in a polar code graph off-line, produce a list of operations, store the list in memory, and feed the list to the decoder to decode the constituent codes in order efficiently, thus increasing the decoding speed. However, the list of operations is dependent on the code rate and as the rate changes, a new list is produced, making fast SC-based decoders not rate-flexible. In this paper, we propose a completely rate-flexible fast SC-based decoder by creating the list of operations directly in hardware, with low implementation complexity. We further propose a hardware architecture implementing the proposed method and show that the area occupation of the rate-flexible fast SC-based decoder in this paper is only $38\%$ of the total area of the memory-based base-line decoder when 5G code rates are supported

Mind the output gap: the disconnect of growth and inflation during recessions and convex Phillips curves in the euro area

We develop a theoretical model that features a business cycle-dependent relation between out- put, price inflation and inflation expectations, augmenting the model by Svensson (1997) with a nonlinear Phillips curve that reflects the rationale underlying the capacity constraint theory (Macklem (1997)). The theoretical model motivates our empirical assessment for the euro area, based on a regime-switching Phillips curve and a regime-switching monetary structural VAR, employing different filter-based, semi-structural model-based and Bayesian factor model-implied output gaps. The analysis confirms the presence of a pronounced convex relationship between inflation and the output gap, meaning that the coefficient in the Phillips curve on the output gap recurringly increases during times of expansion and abates during recessions. The regime switching VAR reveals the business cycle dependence of macroeconomic responses to monetary policy shocks: Expansionary monetary policy induces less pressure on inflation at times of weak as opposed to strong growth; thereby rationalizing relatively stronger expansionary policy, including unconventional volume-based policy such as the Expanded Asset Purchase Programme (EAPP) of the ECB, during times of deep recession

Information flows and disagreement

The aim of this study is to assess the extent to which the degree of heterogeneity of inflation expectations is driven by the flow of information related to current and future price developments. To that end, we follow three routes: i) We propose different measures of information flow that have either a sender or a receiver perspective; ii) We present empirical results for the US and selected EU countries that aim to corroborate the hypothesis that news have the ability to densify expectations, i.e. to reduce forecast heterogeneity; and iii) We augment some otherwise standard models of expectation formation by allowing the individual updating frequency to depend on the observed measure of information flow; since the updating frequency is higher at times of high inflation and decreasing thereafter, this mechanism can contribute to upward biases in inflation expectations over long periods of time

Regime-switching global vector autoregressive models

The purpose of the paper is to develop a Regime-Switching Global Vector Autoregressive (RS-GVAR) model. The RS-GVAR model allows for recurring or non-recurring structural changes in all or a subset of countries. It can be used to generate regime-dependent impulse response functions which are conditional upon a regime-constellation across countries. Coupling the RS and the GVAR methodology improves out-of-sample forecast accuracy significantly in an application to real GDP, price inflation, and stock prices

A false sense of security in applying handpicked equations for stress test purposes

The purpose of this paper is to promote the use of Bayesian model averaging for the design of satellite models that financial institutions employ for stress testing. Banks employing ’handpicked’ equations – while meeting standard economic and econometric soundness criteria – risk significantly underestimating the response of risk parameters and therefore overestimating their capital absorption capacity. We present a set of credit risk models for 18 EU countries based both on the model averaging scheme as well as a series of handpicked equations and apply them to a sample of 108 SSM banks. We thereby aim to illustrate that the handpicked equations may indeed imply significantly lower default flow estimates and therefore overoptimistic estimates for the banks’ capital absorption capacity. The model averaging scheme that we promote should mitigate that risk and also help establish a level playing field with regard to a common level of conservatism across banks

Making the Most of Failure and Uncertainty: Welcome Surprises and Contingency in Energy Transition Research

Energy transitions inherit complex processes full of surprises, unintended consequences, erroneous decisions, uncertainties, paradoxical situations, and sometimes sheer failures [...