Control of consolidation in neural networks: Avoiding runaway effects.

Abstract. Consolidation has been implemented in two ways: as straight rehearsal of patterns or as pseudorehearsal, in which pseudoitems are created by sampling attractors or input–output combinations from the network. Although both implementations have been investigated by several authors, few have explored how it is decided which pattern or pseudoitem is consolidated. Controlling consolidation is not trivial, as it is susceptible to a corruption. In runaway consolidation, one or two patterns monopolize all consolidation resources and come to dominate the entire network. Runaway consolidation is analysed, and three solutions are explored. Suppressing transmission in the connections in which consolidation takes place is shown to work best. Placing bounds on connections or unlearning attractors also alleviates runaway consolidation, though less effectively so