Neural reactivations during sleep determine network credit assignment.

A fundamental goal of motor learning is to establish the neural patterns that produce a desired behavioral outcome. It remains unclear how and when the nervous system solves this 'credit assignment' problem. Using neuroprosthetic learning, in which we could control the causal relationship between neurons and behavior, we found that sleep-dependent processing was required for credit assignment and the establishment of task-related functional connectivity reflecting the casual neuron-behavior relationship. Notably, we observed a strong link between the microstructure of sleep reactivations and credit assignment, with downscaling of non-causal activity. Decoupling of spiking to slow oscillations using optogenetic methods eliminated rescaling. Thus, our results suggest that coordinated firing during sleep is essential for establishing sparse activation patterns that reflect the causal neuron-behavior relationship

Threshold Accepting for Credit Risk Assessment and Validation

According to the latest Basel framework of Banking Supervision, financial institutions should internally assign their borrowers into a number of homogeneous groups. Each group is assigned a probability of default which distinguishes it from other groups. This study aims at determining the optimal number and size of groups that allow for statistical ex post validation of the efficiency of the credit risk assignment system. Our credit risk assignment approach is based on Threshold Accepting, a local search optimization technique, which has recently performed reliably in credit risk clustering especially when considering several realistic constraints. Using a relatively large real-world retail credit portfolio, we propose a new technique to validate ex post the precision of the grading system.credit risk assignment, Threshold Accepting, statistical validation

Retrospective model-based inference guides model-free credit assignment

An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF credit-assignment. Using a task in which there was initial uncertainty about the lotteries that were chosen, we found that when participants’ momentary uncertainty about which lottery had generated an outcome was resolved by provision of subsequent information, participants preferentially assigned credit within a MF system to the lottery they retrospectively inferred was responsible for this outcome. These findings extend our knowledge about the range of MB functions and the scope of system interactions

Empowering the Marginal Student: An Innovative Skills-Based Extra Credit Assignment

A simple extra-credit assignment explicitly rewarded marginal or failing students for improving their learning and study strategies. The instructor approached individual students who were at risk for failing the course following the midterm exam and gave them the option of earning extra-credit points for regularly documenting a variety of effective learning and study skills. In contrast to control groups of matched marginal students and of nonfailing students, those attempting the extra-credit assignment improved their test performance from midterm to final exam. They were more likely to earn at least a grade of C and less likely to drop out of the course than the matched control group. They also evaluated the experience quite positively

Crediting multi-authored papers to single authors

A fair assignment of credit for multi-authored publications is a long-standing issue in scientometrics. In the calculation of the $h$-index, for instance, all co-authors receive equal credit for a given publication, independent of a given author's contribution to the work or of the total number of co-authors. Several attempts have been made to distribute the credit in a more appropriate manner. In a recent paper, Hirsch has suggested a new way of credit assignment that is fundamentally different from the previous ones: All credit for a multi-author paper goes to a single author, the called ``$\alpha$-author'', defined as the person with the highest current $h$-index not the highest $h$-index at the time of the paper's publication) (J. E. Hirsch, Scientometrics 118, 673 (2019)). The collection of papers this author has received credit for as $\alpha$-author is then used to calculate a new index, $h_{\alpha}$, following the same recipe as for the usual $h$ index. The objective of this new assignment is not a fairer distribution of credit, but rather the determination of an altogether different property, the degree of a person's scientific leadership. We show that given the complex time dependence of $h$ for individual scientists, the approach of using the current $h$ value instead of the historic one is problematic, and we argue that it would be feasible to determine the $\alpha$-author at the time of the paper's publication instead. On the other hand, there are other practical considerations that make the calculation of the proposed $h_{\alpha}$ very difficult. As an alternative, we explore other ways of crediting papers to a single author in order to test early career achievement or scientific leadership.Comment: 6 pages, 4 figure

Differentiable Scheduled Sampling for Credit Assignment

We demonstrate that a continuous relaxation of the argmax operation can be used to create a differentiable approximation to greedy decoding for sequence-to-sequence (seq2seq) models. By incorporating this approximation into the scheduled sampling training procedure (Bengio et al., 2015)--a well-known technique for correcting exposure bias--we introduce a new training objective that is continuous and differentiable everywhere and that can provide informative gradients near points where previous decoding decisions change their value. In addition, by using a related approximation, we demonstrate a similar approach to sampled-based training. Finally, we show that our approach outperforms cross-entropy training and scheduled sampling procedures in two sequence prediction tasks: named entity recognition and machine translation.Comment: Accepted at ACL2017 (http://bit.ly/2oj1muX

Credit Assignment in Adaptive Evolutionary Algorithms

In this paper, a new method for assigning credit to search\ud operators is presented. Starting with the principle of optimizing\ud search bias, search operators are selected based on an ability to\ud create solutions that are historically linked to future generations.\ud Using a novel framework for defining performance\ud measurements, distributing credit for performance, and the\ud statistical interpretation of this credit, a new adaptive method is\ud developed and shown to outperform a variety of adaptive and\ud non-adaptive competitors

Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding

Learning long-term dependencies in extended temporal sequences requires credit assignment to events far back in the past. The most common method for training recurrent neural networks, back-propagation through time (BPTT), requires credit information to be propagated backwards through every single step of the forward computation, potentially over thousands or millions of time steps. This becomes computationally expensive or even infeasible when used with long sequences. Importantly, biological brains are unlikely to perform such detailed reverse replay over very long sequences of internal states (consider days, months, or years.) However, humans are often reminded of past memories or mental states which are associated with the current mental state. We consider the hypothesis that such memory associations between past and present could be used for credit assignment through arbitrarily long sequences, propagating the credit assigned to the current state to the associated past state. Based on this principle, we study a novel algorithm which only back-propagates through a few of these temporal skip connections, realized by a learned attention mechanism that associates current states with relevant past states. We demonstrate in experiments that our method matches or outperforms regular BPTT and truncated BPTT in tasks involving particularly long-term dependencies, but without requiring the biologically implausible backward replay through the whole history of states. Additionally, we demonstrate that the proposed method transfers to longer sequences significantly better than LSTMs trained with BPTT and LSTMs trained with full self-attention.Comment: To appear as a Spotlight presentation at NIPS 201