Concentric ESN: Assessing the Effect of Modularity in Cycle Reservoirs

The paper introduces concentric Echo State Network, an approach to design reservoir topologies that tries to bridge the gap between deterministically constructed simple cycle models and deep reservoir computing approaches. We show how to modularize the reservoir into simple unidirectional and concentric cycles with pairwise bidirectional jump connections between adjacent loops. We provide a preliminary experimental assessment showing how concentric reservoirs yield to superior predictive accuracy and memory capacity with respect to single cycle reservoirs and deep reservoir models

Continual Learning with Gated Incremental Memories for sequential data processing

The ability to learn in dynamic, nonstationary environments without forgetting previous knowledge, also known as Continual Learning (CL), is a key enabler for scalable and trustworthy deployments of adaptive solutions. While the importance of continual learning is largely acknowledged in machine vision and reinforcement learning problems, this is mostly under-documented for sequence processing tasks. This work proposes a Recurrent Neural Network (RNN) model for CL that is able to deal with concept drift in input distribution without forgetting previously acquired knowledge. We also implement and test a popular CL approach, Elastic Weight Consolidation (EWC), on top of two different types of RNNs. Finally, we compare the performances of our enhanced architecture against EWC and RNNs on a set of standard CL benchmarks, adapted to the sequential data processing scenario. Results show the superior performance of our architecture and highlight the need for special solutions designed to address CL in RNNs.Comment: Accepted as a conference paper at 2020 International Joint Conference on Neural Networks (IJCNN 2020). Part of 2020 IEEE World Congress on Computational Intelligence (IEEE WCCI 2020

A Protocol for Continual Explanation of SHAP

Continual Learning trains models on a stream of data, with the aim of learning new information without forgetting previous knowledge. Given the dynamic nature of such environments, explaining the predictions of these models can be challenging. We study the behavior of SHAP values explanations in Continual Learning and propose an evaluation protocol to robustly assess the change of explanations in Class-Incremental scenarios. We observed that, while Replay strategies enforce the stability of SHAP values in feedforward/convolutional models, they are not able to do the same with fully-trained recurrent models. We show that alternative recurrent approaches, like randomized recurrent models, are more effective in keeping the explanations stable over time.Comment: ESANN 2023, 6 pages, added link to cod

Renormalized Graph Neural Networks

Graph Neural Networks (GNNs) have become essential for studying complex data, particularly when represented as graphs. Their value is underpinned by their ability to reflect the intricacies of numerous areas, ranging from social to biological networks. GNNs can grapple with non-linear behaviors, emerging patterns, and complex connections; these are also typical characteristics of complex systems. The renormalization group (RG) theory has emerged as the language for studying complex systems. It is recognized as the preferred lens through which to study complex systems, offering a framework that can untangle their intricate dynamics. Despite the clear benefits of integrating RG theory with GNNs, no existing methods have ventured into this promising territory. This paper proposes a new approach that applies RG theory to devise a novel graph rewiring to improve GNNs' performance on graph-related tasks. We support our proposal with extensive experiments on standard benchmarks and baselines. The results demonstrate the effectiveness of our method and its potential to remedy the current limitations of GNNs. Finally, this paper marks the beginning of a new research direction. This path combines the theoretical foundations of RG, the magnifying glass of complex systems, with the structural capabilities of GNNs. By doing so, we aim to enhance the potential of GNNs in modeling and unraveling the complexities inherent in diverse systems

Effects of Hybrid and Maturity Stage on in Vitro Rumen Digestibility of Immature Corn Grain

This study aimed to evaluate the influences of hybrids (HYB) and maturity stage (SAMP) on in vitro rumen digestibility of immature corn grain. Four HYB (Gigantic, Y43, Klips and 9575) from the FAO group 700 were grown under identical agronomic conditions. First sampling (T1) was done after 95 days from seedling and then 4, 8, 13, 18 and 27 days later (T2 to T6). In vitro starch digestibility (STD_7h) and gas production (72 h) were measured. Whole plant and grain dry matter (WP_DM and GR_DM, respectively) and zein content were significantly affected (P<0.01) by HYB and SAMP. Starch content was significantly affected by HYB, SAMP and their interaction. It increased from T1 to T4 (from 67.47 to 72.82% of GR_DM) and then tended to plateau. Concurrently, STD_7h significantly decreased with advancing SAMP and was also affected by HYB. With advancing maturity, total volatile fatty acids (VFA) significantly decreased, with an increase of acetate and a decrease of propionate molar proportion (P<0.01). Gas production rate (GP_c) was significantly affected by HYB, SAMP and HYB×SAMP. Whole plant grain DM correlated (P<0.01) positively with grain starch content (r=0.60 and 0.64) but negatively with STD_7h (r=-0.39 and r=-0.63) and VFA concentration (r=-0.59 and -0.75). Zein percentage in crude protein negatively affected (P<0.01) total DM (r=-0.65,), STD_7h (r=-0.73) and GP_c (r=- 0.68). Results suggest that genotypes and maturity stages influence DM and rumen starch digestibility of immature corn grain and in this respect zein can play a significant role

Projected Latent Distillation for Data-Agnostic Consolidation in Distributed Continual Learning

Distributed learning on the edge often comprises self-centered devices (SCD) which learn local tasks independently and are unwilling to contribute to the performance of other SDCs. How do we achieve forward transfer at zero cost for the single SCDs? We formalize this problem as a Distributed Continual Learning scenario, where SCD adapt to local tasks and a CL model consolidates the knowledge from the resulting stream of models without looking at the SCD's private data. Unfortunately, current CL methods are not directly applicable to this scenario. We propose Data-Agnostic Consolidation (DAC), a novel double knowledge distillation method that consolidates the stream of SC models without using the original data. DAC performs distillation in the latent space via a novel Projected Latent Distillation loss. Experimental results show that DAC enables forward transfer between SCDs and reaches state-of-the-art accuracy on Split CIFAR100, CORe50 and Split TinyImageNet, both in reharsal-free and distributed CL scenarios. Somewhat surprisingly, even a single out-of-distribution image is sufficient as the only source of data during consolidation

RRAML: Reinforced Retrieval Augmented Machine Learning

The emergence of large language models (LLMs) has revolutionized machine learning and related fields, showcasing remarkable abilities in comprehending, generating, and manipulating human language. However, their conventional usage through API-based text prompt submissions imposes certain limitations in terms of context constraints and external source availability. To address these challenges, we propose a novel framework called Reinforced Retrieval Augmented Machine Learning (RRAML). RRAML integrates the reasoning capabilities of LLMs with supporting information retrieved by a purpose-built retriever from a vast user-provided database. By leveraging recent advancements in reinforcement learning, our method effectively addresses several critical challenges. Firstly, it circumvents the need for accessing LLM gradients. Secondly, our method alleviates the burden of retraining LLMs for specific tasks, as it is often impractical or impossible due to restricted access to the model and the computational intensity involved. Additionally we seamlessly link the retriever's task with the reasoner, mitigating hallucinations and reducing irrelevant, and potentially damaging retrieved documents. We believe that the research agenda outlined in this paper has the potential to profoundly impact the field of AI, democratizing access to and utilization of LLMs for a wide range of entities