9,569 research outputs found
Binding and Normalization of Binary Sparse Distributed Representations by Context-Dependent Thinning
Distributed representations were often criticized as inappropriate for encoding of data with a complex structure. However Plate's Holographic Reduced Representations and Kanerva's Binary Spatter Codes are recent schemes that allow on-the-fly encoding of nested compositional structures by real-valued or dense binary vectors of fixed dimensionality.
In this paper we consider procedures of the Context-Dependent Thinning which were developed for representation of complex hierarchical items in the architecture of Associative-Projective Neural Networks. These procedures provide binding of items represented by sparse binary codevectors (with low probability of 1s). Such an encoding is biologically plausible and allows a high storage capacity of distributed associative memory where the codevectors may be stored.
In contrast to known binding procedures, Context-Dependent Thinning preserves the same low density (or sparseness) of the bound codevector for varied number of component codevectors. Besides, a bound codevector is not only similar to another one with similar component codevectors (as in other schemes), but it is also similar to the component codevectors themselves. This allows the similarity of structures to be estimated just by the overlap of their codevectors, without retrieval of the component codevectors. This also allows an easy retrieval of the component codevectors.
Examples of algorithmic and neural-network implementations of the thinning procedures are considered. We also present representation examples for various types of nested structured data (propositions using role-filler and predicate-arguments representation schemes, trees, directed acyclic graphs) using sparse codevectors of fixed dimension. Such representations may provide a fruitful alternative to the symbolic representations of traditional AI, as well as to the localist and microfeature-based connectionist representations
The capacity of multilevel threshold functions
Lower and upper bounds for the capacity of multilevel threshold elements are estimated, using two essentially different enumeration techniques. It is demonstrated that the exact number of multilevel threshold functions depends strongly on the relative topology of the input set. The results correct a previously published estimate and indicate that adding threshold levels enhances the capacity more than adding variables
Multilevel domain decomposition-based architectures for physics-informed neural networks
Physics-informed neural networks (PINNs) are a popular and powerful approach
for solving problems involving differential equations, yet they often struggle
to solve problems with high frequency and/or multi-scale solutions. Finite
basis physics-informed neural networks (FBPINNs) improve the performance of
PINNs in this regime by combining them with an overlapping domain decomposition
approach. In this paper, the FBPINN approach is extended by adding multiple
levels of domain decompositions to their solution ansatz, inspired by classical
multilevel Schwarz domain decomposition methods (DDMs). Furthermore, analogous
to typical tests for classical DDMs, strong and weak scaling studies designed
for measuring how the accuracy of PINNs and FBPINNs behaves with respect to
computational effort and solution complexity are carried out. Our numerical
results show that the proposed multilevel FBPINNs consistently and
significantly outperform PINNs across a range of problems with high frequency
and multi-scale solutions. Furthermore, as expected in classical DDMs, we show
that multilevel FBPINNs improve the scalability of FBPINNs to large numbers of
subdomains by aiding global communication between subdomains.Comment: 25 pages, 9 figure
- …