Function and form in networks of interacting agents

The main problem we address in this paper is whether function determines form when a society of agents organizes itself for some purpose or whether the organizing method is more important than the functionality in determining the structure of the ensemble. As an example, we use a neural network that learns the same function by two different learning methods. For sufficiently large networks, very different structures may indeed be obtained for the same functionality. Clustering, characteristic path length and hierarchy are structural differences, which in turn have implications on the robustness and adaptability of the networks. In networks, as opposed to simple graphs, the connections between the agents are not necessarily symmetric and may have positive or negative signs. New characteristic coefficients are introduced to characterize this richer connectivity structure.Comment: 27 pages Latex, 11 figure

Deep Learning on Lie Groups for Skeleton-based Action Recognition

In recent years, skeleton-based action recognition has become a popular 3D classification problem. State-of-the-art methods typically first represent each motion sequence as a high-dimensional trajectory on a Lie group with an additional dynamic time warping, and then shallowly learn favorable Lie group features. In this paper we incorporate the Lie group structure into a deep network architecture to learn more appropriate Lie group features for 3D action recognition. Within the network structure, we design rotation mapping layers to transform the input Lie group features into desirable ones, which are aligned better in the temporal domain. To reduce the high feature dimensionality, the architecture is equipped with rotation pooling layers for the elements on the Lie group. Furthermore, we propose a logarithm mapping layer to map the resulting manifold data into a tangent space that facilitates the application of regular output layers for the final classification. Evaluations of the proposed network for standard 3D human action recognition datasets clearly demonstrate its superiority over existing shallow Lie group feature learning methods as well as most conventional deep learning methods.Comment: Accepted to CVPR 201

Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks

In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses. As a result, our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution. Finally, we demonstrate generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets

A memory ann computing structure for nonlinear systems emulation identification

Currently, almost all efforts for using artificial neural networks for control oriented process identification are based on feed-forward networks. Provided the system order or the upper limit of the order is known, a neural network design is feasible for which all the collection of previous values of the inputs and outputs of the system to be identified can be used as input data to train in the network computing structures to learn the input-output map. This work reports on a novel technique that makes use of memory artificial neural network architecture that can learn and transform so as to emulate any non-linear input-output map for multi-input-multi-output systems when no prior knowledge on specific system features exists

Dynamic Function Learning through Control of Ensemble Systems

Learning tasks involving function approximation are preva- lent in numerous domains of science and engineering. The underlying idea is to design a learning algorithm that gener- ates a sequence of functions converging to the desired target function with arbitrary accuracy by using the available data samples. In this paper, we present a novel interpretation of iterative function learning through the lens of ensemble dy- namical systems, with an emphasis on establishing the equiv- alence between convergence of function learning algorithms and asymptotic behavior of ensemble systems. In particular, given a set of observation data in a function learning task, we prove that the procedure of generating an approximation sequence can be represented as a steering problem of a dy- namic ensemble system defined on a function space. This in turn gives rise to an ensemble systems-theoretic approach to the design of “continuous-time” function learning algorithms, which have a great potential to reach better generalizability compared with classical “discrete-time” learning algorithms

Non-parametric estimation of mixed discrete choice models

In this paper, different strands of literature are combined in order to obtain algorithms for semi-parametric estimation of discrete choice models that include the modelling of unobserved heterogeneity by using mixing distributions for the parameters defining the preferences. The models use the theory on non-parametric maximum likelihood estimation (NP-MLE) that has been developed for general mixing models. The expectation-maximization (EM) techniques used in the NP-MLE literature are combined with strategies for choosing appropriate approximating models using adaptive grid techniques. \\ Jointly this leads to techniques for specification and estimation that can be used to obtain a consistent specification of the mixing distribution. Additionally, also algorithms for the estimation are developed that help to decrease problems due to the curse of dimensionality. \\ The proposed algorithms are demonstrated in a small scale simulation study to be useful for the specification and estimation of mixture models in the discrete choice context providing some information on the specification of the mixing distribution. The simulations document that some aspects of the mixing distribution such as the expectation can be estimated reliably. They also demonstrate, however, that typically different approximations to the mixing distribution lead to similar values of the likelihood and hence are hard to discriminate. Therefore it does not appear to be possible to reliably infer the most appropriate parametric form for the estimated mixing distribution.Comment: Paper presented at the International Choice Modelling Conference (ICMC2019) in Kobe, Japa

Nonlinear Systems Identification Using Additive Dynamic Neural Networks--Two On Line Approaches.

This paper proposes a class of additive dynamic connectionist (ADC) models for identification of unknown dynamic systems. These models work in continuous time and are linear in their parameters. Also, for this kind of model two on-line learning or parameter adaptation algorithms are developed: one based on gradient techniques and sensitivity analysis of the model output trajectories versus the model parameters and the other based on variational calculus, that lead to an off-line solution and an invariant imbedding technique that converts the off-line solution to an on-line one. These learning methods are developed using matrix calculus techniques in order to implement them in an automatic manner with the help of a symbolic manipulation package. The good behavior of the class of identification models and the two learning methods is tested on two simulated plants and a data set from a real plant and compared, in this case, with a feedforward static (FFS) identifier.Peer Reviewe