Towards Building Deep Networks with Bayesian Factor Graphs

We propose a Multi-Layer Network based on the Bayesian framework of the Factor Graphs in Reduced Normal Form (FGrn) applied to a two-dimensional lattice. The Latent Variable Model (LVM) is the basic building block of a quadtree hierarchy built on top of a bottom layer of random variables that represent pixels of an image, a feature map, or more generally a collection of spatially distributed discrete variables. The multi-layer architecture implements a hierarchical data representation that, via belief propagation, can be used for learning and inference. Typical uses are pattern completion, correction and classification. The FGrn paradigm provides great flexibility and modularity and appears as a promising candidate for building deep networks: the system can be easily extended by introducing new and different (in cardinality and in type) variables. Prior knowledge, or supervised information, can be introduced at different scales. The FGrn paradigm provides a handy way for building all kinds of architectures by interconnecting only three types of units: Single Input Single Output (SISO) blocks, Sources and Replicators. The network is designed like a circuit diagram and the belief messages flow bidirectionally in the whole system. The learning algorithms operate only locally within each block. The framework is demonstrated in this paper in a three-layer structure applied to images extracted from a standard data set.Comment: Submitted for journal publicatio

Deterministic hierarchical networks

It has been shown that many networks associated with complex systems are small-world (they have both a large local clustering coefficient and a small diameter) and also scale-free (the degrees are distributed according to a power law). Moreover, these networks are very often hierarchical, as they describe the modularity of the systems that are modeled. Most of the studies for complex networks are based on stochastic methods. However, a deterministic method, with an exact determination of the main relevant parameters of the networks, has proven useful. Indeed, this approach complements and enhances the probabilistic and simulation techniques and, therefore, it provides a better understanding of the modeled systems. In this paper we find the radius, diameter, clustering coefficient and degree distribution of a generic family of deterministic hierarchical small-world scale-free networks that has been considered for modeling real-life complex systems.Peer ReviewedPostprint (author's final draft

Bus interconnection networks

AbstractIn bus interconnection networks every bus provides a communication medium between a set of processors. These networks are modeled by hypergraphs where vertices represent the processors and edges represent the buses. We survey the results obtained on the construction methods that connect a large number of processors in a bus network with given maximum processor degree Δ, maximum bus size r, and network diameter D. (In hypergraph terminology this problem is known as the (Δ,D, r)-hypergraph problem.)The problem for point-to-point networks (the case r = 2) has been extensively studied in the literature. As a result, several families of networks have been proposed. Some of these point-to-point networks can be used in the construction of bus networks. One approach is to consider the dual of the network. We survey some families of bus networks obtained in this manner. Another approach is to view the point-to-point networks as a special case of the bus networks and to generalize the known constructions to bus networks. We provide a summary of the tools developed in the theory of hypergraphs and directed hypergraphs to handle this approach

Instantaneously trained neural networks with complex and quaternion inputs

Neural network architectures such as backpropagation networks, perceptrons or generalized Hopfield networks can handle complex inputs but they require a large amount of time and resources for the training process. This thesis investigates instantaneously trained feedforward neural networks that can handle complex and quaternion inputs. The performance of the basic algorithm has been analyzed and shown how it provides a plausible model of human perception and understanding of images. The motivation for studying quaternion inputs is their use in representing spatial rotations that find applications in computer graphics, robotics, global navigation, computer vision and the spatial orientation of instruments. The problem of efficient mapping of data in quaternion neural networks is examined. Some problems that need to be addressed before quaternion neural networks find applications are identified

Some recent asynchronous system design methodologies

Journal ArticleWe present an in-depth study of some techniques for asynchronous system design, analysis, and verification. After defining basic terminology, we take one simple example - a four-phase t o two-phase converter - and present its design using (a) classical flow-tables; (b) Signal Transition Graphs of [8]; and (c) Trace Theory of [15]. We then present necessary and sufficient conditions for Delay Insensitivity, proposed by [38], and illustrate it on our example. Finally, we present the work of [13] on the verification of asynchronous circuits, and illustrate it on the circuits derived in the paper. The following points are emphasized: (i) presentation of techniques at more depth than in a general survey; (ii) illustration of all t h e aspects discussed on a common example; (hi) comparative study of the works presented. Many interesting works had to be left out, solely because of our lack of space and time

Image Understanding by Hierarchical Symbolic Representation and Inexact Matching of Attributed Graphs

We study the symbolic representation of imagery information by a powerful global representation scheme in the form of Attributed Relational Graph (ARG), and propose new techniques for the extraction of such representation from spatial-domain images, and for performing the task of image understanding through the analysis of the extracted ARG representation. To achieve practical image understanding tasks, the system needs to comprehend the imagery information in a global form. Therefore, we propose a multi-layer hierarchical scheme for the extraction of global symbolic representation from spatial-domain images. The proposed scheme produces a symbolic mapping of the input data in terms of an output alphabet, whose elements are defined over global subimages. The proposed scheme uses a combination of model-driven and data-driven concepts. The model- driven principle is represented by a graph transducer, which is used to specify the alphabet at each layer in the scheme. A symbolic mapping is driven by the input data to map the input local alphabet into the output global alphabet. Through the iterative application of the symbolic transformational mapping at different levels of hierarchy, the system extracts a global representation from the image in the form of attributed relational graphs. Further processing and interpretation of the imagery information can, then, be performed on their ARG representation. We also propose an efficient approach for calculating a distance measure and finding the best inexact matching configuration between attributed relational graphs. For two ARGs, we define sequences of weighted error-transformations which when performed on one ARG (or a subgraph of it), will produce the other ARG. A distance measure between two ARGs is defined as the weight of the sequence which possesses minimum total-weight. Moreover, this minimum-total weight sequence defines the best inexact matching configuration between the two ARGs. The global minimization over the possible sequences is performed by a dynamic programming technique, the approach shows good results for ARGs of practical sizes. The proposed system possesses the capability to inference the alphabets of the ARG representation which it uses. In the inference phase, the hierarchical scheme is usually driven by the input data only, which normally consist of images of model objects. It extracts the global alphabet of the ARG representation of the models. The extracted model representation is then used in the operation phase of the system to: perform the mapping in the multi-layer scheme. We present our experimental results for utilizing the proposed system for locating objects in complex scenes