Forgetting

Forgetting is importantly related to remembering, evidence possession, epistemic virtue, personal identity, and a host of highly-researched memory conditions. In this paper I examine the nature of forgetting. I canvass the viable options for forgetting’s ontological category, type of content, characteristic relation to content, and scale. I distinguish several theories of forgetting in the philosophy and psychology of memory literatures, theories that diverge on these options. The best theories from the literature, I claim, fail two critical tests that I develop (the metacognition and prospection tests), underwriting arguments against the theories. I introduce a new theory about the state of forgetting—the learning, access failure, dispositional (LEAD) theory: to forget is to fail to access something that is both learned and either inaccessible or intended to be accessed. I argue that the LEAD theory of forgetting is the lead theory of forgetting. It passes the metacognition and prospection tests, and has several further virtues at no cost. Finally, I advocate reductionism about the process of forgetting; the process reduces wholly to states of forgetting. In particular, a process of forgetting is just a sequence of increasingly strong states of forgetting

Inhibitory effects of thought substitution in the think/no-think task: evidence from independent cues

When people try not to think about a certain item, they can accomplish this goal by using a thought substitution strategy and think about something else. Research conducted with the think/no-think (TNT) paradigm indicates that such strategy leads subsequently to forgetting the information participants tried not to think about. The present study pursued two goals. First, it investigated the mechanism of forgetting due to thought substitution, contrasting the hypothesis by which forgetting is due to blocking caused by substitutes with the hypothesis that forgetting is due to inhibition (using an independent cue methodology). Second, a boundary condition for forgetting due to thought substitution was examined by creating conditions under which the generation of appropriate substitutes would be impaired. In two experiments, participants completed a TNT task under thought substitution instructions in which either words or pseudo-words were used as original cues and memory was assessed with original and independent cues. The results revealed forgetting in both original and independent cue tests, supporting the inhibitory account of thought substitution, but only when cues were words, and not when they were non-words, pointing to the ineffectiveness of a thought substitution strategy when original cues lack semantic content

An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks

Catastrophic forgetting is a problem faced by many machine learning models and algorithms. When trained on one task, then trained on a second task, many machine learning models "forget" how to perform the first task. This is widely believed to be a serious problem for neural networks. Here, we investigate the extent to which the catastrophic forgetting problem occurs for modern neural networks, comparing both established and recent gradient-based training algorithms and activation functions. We also examine the effect of the relationship between the first task and the second task on catastrophic forgetting. We find that it is always best to train using the dropout algorithm--the dropout algorithm is consistently best at adapting to the new task, remembering the old task, and has the best tradeoff curve between these two extremes. We find that different tasks and relationships between tasks result in very different rankings of activation function performance. This suggests the choice of activation function should always be cross-validated

The role of oblivion, memory size and spatial separation in dynamic language games

In this paper we present some multiagent simulations in which the individuals try to reach a uniform vocabulary to name spatial movements. Each agent has initially a random vocabulary that can be modified by means of interactions with the other agents. As the objective is to name movements, the topic of conversation is chosen by moving. Each agent can remember a finite number of words per movement, with certain strength. We show the importance of the forgetting process and memory size in these simulations, discuss the effect of the number of agents on the time to agree and present a few experiments where the evolution of vocabularies takes place in a divided range.This paper has been sponsored by the Spanish Interdepartmental Commission of Science and Technology (CICYT), project numbers TEL1999-0181, and TIC 2001-0685-C02-01

Best They Forget: Challenging Notions of Remembering and Forgetting

In Jeremiah 31:34 the LORD declares, “No longer will a man teach his neighbour, or a man his brother, saying, ‘Know the LORD,’ because they will all know me, from the least of them to the greatest… For I will forgive their wickedness and will remember their sins no more” (The NIV Study Bible, 1995, p.1170). It is not the intention of this paper to enter into a theological discussion as to whether or not God is capable of forgetting; however, at very least He chooses the metaphor of forgetting to display his forgiveness for his people. This seems to conflict with a commonly held negative stigma attached to forgetting. It has long been the case, specifically in the classroom, that remembering is considered a positive activity while forgetting is considered a negative one. It is the purpose of this paper to question this assumption by consolidating research done on multiple advantages of forgetting as well as many disadvantages connected to remembering. The discussion will begin with a glimpse at the direction our world could be moving towards in terms of collected memory, an emerging world which brings with it many problems that seem to be solvable only through intentional forgetting. Keeping in mind the theoretical disadvantages of complete memory, one must also recognize the flaws of memory today as well as the possible dangers that memory poses. Last, the research will be made applicable to the classroom and methods of forgetting will be proposed in order to benefit student-learning. This discussion is leading one towards the final conclusion that, at specific times, forgetting is beneficial, ethical, and necessary for advancing student learning

Memory Aware Synapses: Learning what (not) to forget

Humans can learn in a continuous manner. Old rarely utilized knowledge can be overwritten by new incoming information while important, frequently used knowledge is prevented from being erased. In artificial learning systems, lifelong learning so far has focused mainly on accumulating knowledge over tasks and overcoming catastrophic forgetting. In this paper, we argue that, given the limited model capacity and the unlimited new information to be learned, knowledge has to be preserved or erased selectively. Inspired by neuroplasticity, we propose a novel approach for lifelong learning, coined Memory Aware Synapses (MAS). It computes the importance of the parameters of a neural network in an unsupervised and online manner. Given a new sample which is fed to the network, MAS accumulates an importance measure for each parameter of the network, based on how sensitive the predicted output function is to a change in this parameter. When learning a new task, changes to important parameters can then be penalized, effectively preventing important knowledge related to previous tasks from being overwritten. Further, we show an interesting connection between a local version of our method and Hebb's rule,which is a model for the learning process in the brain. We test our method on a sequence of object recognition tasks and on the challenging problem of learning an embedding for predicting $$ triplets. We show state-of-the-art performance and, for the first time, the ability to adapt the importance of the parameters based on unlabeled data towards what the network needs (not) to forget, which may vary depending on test conditions.Comment: ECCV 201