Hierarchical Temporal Representation in Linear Reservoir Computing

Recently, studies on deep Reservoir Computing (RC) highlighted the role of layering in deep recurrent neural networks (RNNs). In this paper, the use of linear recurrent units allows us to bring more evidence on the intrinsic hierarchical temporal representation in deep RNNs through frequency analysis applied to the state signals. The potentiality of our approach is assessed on the class of Multiple Superimposed Oscillator tasks. Furthermore, our investigation provides useful insights to open a discussion on the main aspects that characterize the deep learning framework in the temporal domain.Comment: This is a pre-print of the paper submitted to the 27th Italian Workshop on Neural Networks, WIRN 201

Time Series Clustering with Deep Reservoir Computing

This paper proposes a method for clustering of time series, based upon the ability of deep Reservoir Computing networks to grasp the dynamical structure of the series that is presented as input. A standard clustering algorithm, such as k-means, is applied to the network states, rather than the input series themselves. Clustering is thus embedded into the network dynamical evolution, since a clustering result is obtained at every time step, which in turn serves as initialisation at the next step. We empirically assess the performance of deep reservoir systems in time series clustering on benchmark datasets, considering the influence of crucial hyperparameters. Experimentation with the proposed model shows enhanced clustering quality, measured by the silhouette coefficient, when compared to both static clustering of data, and dynamic clustering with a shallow network

Deep Tree Transductions - A Short Survey

The paper surveys recent extensions of the Long-Short Term Memory networks to handle tree structures from the perspective of learning non-trivial forms of isomorph structured transductions. It provides a discussion of modern TreeLSTM models, showing the effect of the bias induced by the direction of tree processing. An empirical analysis is performed on real-world benchmarks, highlighting how there is no single model adequate to effectively approach all transduction problems.Comment: To appear in the Proceedings of the 2019 INNS Big Data and Deep Learning (INNSBDDL 2019). arXiv admin note: text overlap with arXiv:1809.0909

Richness of Deep Echo State Network Dynamics

Reservoir Computing (RC) is a popular methodology for the efficient design of Recurrent Neural Networks (RNNs). Recently, the advantages of the RC approach have been extended to the context of multi-layered RNNs, with the introduction of the Deep Echo State Network (DeepESN) model. In this paper, we study the quality of state dynamics in progressively higher layers of DeepESNs, using tools from the areas of information theory and numerical analysis. Our experimental results on RC benchmark datasets reveal the fundamental role played by the strength of inter-reservoir connections to increasingly enrich the representations developed in higher layers. Our analysis also gives interesting insights into the possibility of effective exploitation of training algorithms based on stochastic gradient descent in the RC field.Comment: Preprint of the paper accepted at IWANN 201

Internet of robotic things : converging sensing/actuating, hypoconnectivity, artificial intelligence and IoT Platforms

The Internet of Things (IoT) concept is evolving rapidly and influencing newdevelopments in various application domains, such as the Internet of MobileThings (IoMT), Autonomous Internet of Things (A-IoT), Autonomous Systemof Things (ASoT), Internet of Autonomous Things (IoAT), Internetof Things Clouds (IoT-C) and the Internet of Robotic Things (IoRT) etc.that are progressing/advancing by using IoT technology. The IoT influencerepresents new development and deployment challenges in different areassuch as seamless platform integration, context based cognitive network integration,new mobile sensor/actuator network paradigms, things identification(addressing, naming in IoT) and dynamic things discoverability and manyothers. The IoRT represents new convergence challenges and their need to be addressed, in one side the programmability and the communication ofmultiple heterogeneous mobile/autonomous/robotic things for cooperating,their coordination, configuration, exchange of information, security, safetyand protection. Developments in IoT heterogeneous parallel processing/communication and dynamic systems based on parallelism and concurrencyrequire new ideas for integrating the intelligent “devices”, collaborativerobots (COBOTS), into IoT applications. Dynamic maintainability, selfhealing,self-repair of resources, changing resource state, (re-) configurationand context based IoT systems for service implementation and integrationwith IoT network service composition are of paramount importance whennew “cognitive devices” are becoming active participants in IoT applications.This chapter aims to be an overview of the IoRT concept, technologies,architectures and applications and to provide a comprehensive coverage offuture challenges, developments and applications

Multivariate discrimination and the Higgs + W/Z search

A systematic method for optimizing multivariate discriminants is developed and applied to the important example of a light Higgs boson search at the Tevatron and the LHC. The Significance Improvement Characteristic (SIC), defined as the signal efficiency of a cut or multivariate discriminant divided by the square root of the background efficiency, is shown to be an extremely powerful visualization tool. SIC curves demonstrate numerical instabilities in the multivariate discriminants, show convergence as the number of variables is increased, and display the sensitivity to the optimal cut values. For our application, we concentrate on Higgs boson production in association with a W or Z boson with H -> bb and compare to the irreducible standard model background, Z/W + bb. We explore thousands of experimentally motivated, physically motivated, and unmotivated single variable discriminants. Along with the standard kinematic variables, a number of new ones, such as twist, are described which should have applicability to many processes. We find that some single variables, such as the pull angle, are weak discriminants, but when combined with others they provide important marginal improvement. We also find that multiple Higgs boson-candidate mass measures, such as from mild and aggressively trimmed jets, when combined may provide additional discriminating power. Comparing the significance improvement from our variables to those used in recent CDF and DZero searches, we find that a 10-20% improvement in significance against Z/W + bb is possible. Our analysis also suggests that the H + W/Z channel with H -> bb is also viable at the LHC, without requiring a hard cut on the W/Z transverse momentum.Comment: 41 pages, 5 tables, 29 figure

Climate variability during MIS 20–18 as recorded by alkenone-SST and calcareous plankton in the Ionian Basin (central Mediterranean)

This study shows the first Mediterranean high-resolution record of alkenone-derived sea surface temperature (SST) in the marine sediments outcropping at the Ideale section (IS) (southern Italy, central Mediterranean) from late marine isotope stage (MIS) 20 - through early MIS 18. The SST pattern evidences glacial-interglacial up to submillennial-scale temperature variation, with lower values (~13 °C) in late MIS 20 and substage 19b, and higher values (up to 21 °C) in MIS 19c and in the interstadials of MIS 19a. The SST data are combined with the new calcareous plankton analysis and the available, chronologically well-constrained carbon and oxygen isotope records in the IS. The multi-proxy approach, together with the location of the IS near the Italian coasts, the lower circalittoral-upper bathyal depositional setting, and high sedimentation rate allow to document long-and short-term paleoenvironmental modifications (sea level, rainfall, inorganic/organic/fresh water input to the basin), as a response to regional and global climate changes. The combined proxies reveal the occurrence of a terminal stadial event in late MIS 20 (here Med-HTIX), and warm-cold episodes (here Med-BATIX and Med-YDTIX) during Termination IX (TIX), which recall those that occurred through the last termination (TI). During these periods and the following ghost sapropel layer (insolation cycle 74, 784 ka) in the early MIS 19, high frequency internal changes are synchronously recorded by all proxies. The substage MIS 19c is warm but quite unstable, with several episodes of paleoenvironmental changes, associated with fluctuating tropical-subtropical water inflow through the Gibraltar Strait, variations of the cyclonic regime in the Ionian basin, and the southward shift of westerly winds and winter precipitation over southern Europe and Mediterranean basin. Three high-amplitude millennial-scale oscillations in the patterns of SST and calcareous plankton key taxa during MIS 19a are interpreted as linked to changes in temperature as well as in salinity due to periodical water column stratification and mixing. The main processes involved in the climate variability include changes in oceanographic exchanges through the Gibraltar Strait during modulations of Atlantic meridional overturning circulation and/or variations in atmospheric dynamics related to the influence of westerly and polar winds acting in the paleo-Ionian basin. A strong climate teleconnection between the North Atlantic and Mediterranean is discussed, and a prominent role of atmospheric processes in the central Mediterranean is evidenced by comparing data sets at the IS with Italian and extra-Mediterranean marine and terrestrial records

Jet Substructure Without Trees

We present an alternative approach to identifying and characterizing jet substructure. An angular correlation function is introduced that can be used to extract angular and mass scales within a jet without reference to a clustering algorithm. This procedure gives rise to a number of useful jet observables. As an application, we construct a top quark tagging algorithm that is competitive with existing methods.Comment: 22 pages, 16 figures, version accepted by JHE