Tree Echo State Networks

In this paper we present the Tree Echo State Network (TreeESN) model, generalizing the paradigm of Reservoir Computing to tree structured data. TreeESNs exploit an untrained generalized recursive reservoir, exhibiting extreme efficiency for learning in structured domains. In addition, we highlight through the paper other characteristics of the approach: First, we discuss the Markovian characterization of reservoir dynamics, extended to the case of tree domains, that is implied by the contractive setting of the TreeESN state transition function. Second, we study two types of state mapping functions to map the tree structured state of TreeESN into a fixed-size feature representation for classification or regression tasks. The critical role of the relation between the choice of the state mapping function and the Markovian characterization of the task is analyzed and experimentally investigated on both artificial and real-world tasks. Finally, experimental results on benchmark and real-world tasks show that the TreeESN approach, in spite of its efficiency, can achieve comparable results with state-of-the-art, although more complex, neural and kernel based models for tree structured data

Fast and compact self-stabilizing verification, computation, and fault detection of an MST

This paper demonstrates the usefulness of distributed local verification of proofs, as a tool for the design of self-stabilizing algorithms.In particular, it introduces a somewhat generalized notion of distributed local proofs, and utilizes it for improving the time complexity significantly, while maintaining space optimality. As a result, we show that optimizing the memory size carries at most a small cost in terms of time, in the context of Minimum Spanning Tree (MST). That is, we present algorithms that are both time and space efficient for both constructing an MST and for verifying it.This involves several parts that may be considered contributions in themselves.First, we generalize the notion of local proofs, trading off the time complexity for memory efficiency. This adds a dimension to the study of distributed local proofs, which has been gaining attention recently. Specifically, we design a (self-stabilizing) proof labeling scheme which is memory optimal (i.e., $O(\log n)$ bits per node), and whose time complexity is $O(\log ^2 n)$ in synchronous networks, or $O(\Delta \log ^3 n)$ time in asynchronous ones, where $\Delta$ is the maximum degree of nodes. This answers an open problem posed by Awerbuch and Varghese (FOCS 1991). We also show that $\Omega(\log n)$ time is necessary, even in synchronous networks. Another property is that if $f$ faults occurred, then, within the requireddetection time above, they are detected by some node in the $O(f\log n)$ locality of each of the faults.Second, we show how to enhance a known transformer that makes input/output algorithms self-stabilizing. It now takes as input an efficient construction algorithm and an efficient self-stabilizing proof labeling scheme, and produces an efficient self-stabilizing algorithm. When used for MST, the transformer produces a memory optimal self-stabilizing algorithm, whose time complexity, namely, $O(n)$, is significantly better even than that of previous algorithms. (The time complexity of previous MST algorithms that used $\Omega(\log^2 n)$ memory bits per node was $O(n^2)$, and the time for optimal space algorithms was $O(n|E|)$.) Inherited from our proof labelling scheme, our self-stabilising MST construction algorithm also has the following two properties: (1) if faults occur after the construction ended, then they are detected by some nodes within $O(\log ^2 n)$ time in synchronous networks, or within $O(\Delta \log ^3 n)$ time in asynchronous ones, and (2) if $f$ faults occurred, then, within the required detection time above, they are detected within the $O(f\log n)$ locality of each of the faults. We also show how to improve the above two properties, at the expense of some increase in the memory

Verifying and Monitoring IoTs Network Behavior using MUD Profiles

IoT devices are increasingly being implicated in cyber-attacks, raising community concern about the risks they pose to critical infrastructure, corporations, and citizens. In order to reduce this risk, the IETF is pushing IoT vendors to develop formal specifications of the intended purpose of their IoT devices, in the form of a Manufacturer Usage Description (MUD), so that their network behavior in any operating environment can be locked down and verified rigorously. This paper aims to assist IoT manufacturers in developing and verifying MUD profiles, while also helping adopters of these devices to ensure they are compatible with their organizational policies and track devices network behavior based on their MUD profile. Our first contribution is to develop a tool that takes the traffic trace of an arbitrary IoT device as input and automatically generates the MUD profile for it. We contribute our tool as open source, apply it to 28 consumer IoT devices, and highlight insights and challenges encountered in the process. Our second contribution is to apply a formal semantic framework that not only validates a given MUD profile for consistency, but also checks its compatibility with a given organizational policy. We apply our framework to representative organizations and selected devices, to demonstrate how MUD can reduce the effort needed for IoT acceptance testing. Finally, we show how operators can dynamically identify IoT devices using known MUD profiles and monitor their behavioral changes on their network.Comment: 17 pages, 17 figures. arXiv admin note: text overlap with arXiv:1804.0435

Recurrent Dynamic Message Passing with Loops for Epidemics on Networks

Several theoretical methods have been developed to approximate prevalence and threshold of epidemics on networks. Among them, the recurrent dynamic message-passing (rDMP) theory offers a state-of-the-art performance by preventing the echo chamber effect in network edges. However, the rDMP theory was derived in an intuitive ad-hoc way, lacking a solid theoretical foundation and resulting in a probabilistic inconsistency flaw. Furthermore, real-world networks are clustered and full of local loops like triangles, whereas rDMP is based on the assumption of a locally tree-like network structure, which makes rDMP potentially inefficient on real applications. In this work, for the recurrent-state epidemics, we first demonstrate that the echo chamber effect exits not only in edges but also in local loops, which rDMP-like method can not avoid. We then correct the deficiency of rDMP in a principled manner, leading to the natural introduction of new higher-order dynamic messages, extending rDMP to handle local loops. By linearizing the extended message-passing equations, a new epidemic threshold estimation is given by the inverse of the leading eigenvalue of a matrix named triangular non-backtracking matrix. Numerical experiments conducted on synthetic and real-world networks to evaluate our method, the efficacy of which is validated in epidemic prevalence and threshold prediction tasks. In addition, our method has the potential to speed up the solution of the immunization, influence maximization, and robustness optimization problems in the networks.Comment: Submitted, 14 pages, 7 figure

Deep Tree Transductions - A Short Survey

The paper surveys recent extensions of the Long-Short Term Memory networks to handle tree structures from the perspective of learning non-trivial forms of isomorph structured transductions. It provides a discussion of modern TreeLSTM models, showing the effect of the bias induced by the direction of tree processing. An empirical analysis is performed on real-world benchmarks, highlighting how there is no single model adequate to effectively approach all transduction problems.Comment: To appear in the Proceedings of the 2019 INNS Big Data and Deep Learning (INNSBDDL 2019). arXiv admin note: text overlap with arXiv:1809.0909

Resting state correlates of subdimensions of anxious affect

Resting state fMRI may help identify markers of risk for affective disorder. Given the comorbidity of anxiety and depressive disorders and the heterogeneity of these disorders as defined by DSM, an important challenge is to identify alterations in resting state brain connectivity uniquely associated with distinct profiles of negative affect. The current study aimed to address this by identifying differences in brain connectivity specifically linked to cognitive and physiological profiles of anxiety, controlling for depressed affect. We adopted a two-stage multivariate approach. Hierarchical clustering was used to independently identify dimensions of negative affective style and resting state brain networks. Combining the clustering results, we examined individual differences in resting state connectivity uniquely associated with subdimensions of anxious affect, controlling for depressed affect. Physiological and cognitive subdimensions of anxious affect were identified. Physiological anxiety was associated with widespread alterations in insula connectivity, including decreased connectivity between insula subregions and between the insula and other medial frontal and subcortical networks. This is consistent with the insula facilitating communication between medial frontal and subcortical regions to enable control of physiological affective states. Meanwhile, increased connectivity within a frontoparietal-posterior cingulate cortex-precunous network was specifically associated with cognitive anxiety, potentially reflecting increased spontaneous negative cognition (e.g., worry). These findings suggest that physiological and cognitive anxiety comprise subdimensions of anxiety-related affect and reveal associated alterations in brain connectivity