Learning in Feedforward Neural Networks Accelerated by Transfer Entropy

Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an information transfer measure used to quantify the statistical coherence between events (time series). Later, it was related to causality, even if they are not the same. There are only few papers reporting applications of causality or TE in neural networks. Our contribution is an information-theoretical method for analyzing information transfer between the nodes of feedforward neural networks. The information transfer is measured by the TE of feedback neural connections. Intuitively, TE measures the relevance of a connection in the network and the feedback amplifies this connection. We introduce a backpropagation type training algorithm that uses TE feedback connections to improve its performance

The flow of information in trading: an entropy approach to market regimes

In this study, we use entropy-based measures to identify different types of trading behaviors.1We detect the return-driven trading using the conditional block entropy that dynamically reflects the “self-causality' of market return flows. Then we use the transfer entropy to identify the news-driven3trading activity that is revealed by the information flows from news sentiment to market returns. We argue that when certain trading behaviour becomes dominant or jointly dominant, the market will form a specific regime, namely return-, news- or mixed regime. Based on 11 years of news and market data, we find that the evolution of financial market regimes in terms of adaptive trading activities over the 2008 liquidity and euro-zone debt crises can be explicitly explained by the information flows. The proposed method can be expanded to make “causal' inferences on other types of economic phenomena

Complex systems methods characterizing nonlinear processes in the near-Earth electromagnetic environment: recent advances and open challenges

Learning from successful applications of methods originating in statistical mechanics, complex systems science, or information theory in one scientific field (e.g., atmospheric physics or climatology) can provide important insights or conceptual ideas for other areas (e.g., space sciences) or even stimulate new research questions and approaches. For instance, quantification and attribution of dynamical complexity in output time series of nonlinear dynamical systems is a key challenge across scientific disciplines. Especially in the field of space physics, an early and accurate detection of characteristic dissimilarity between normal and abnormal states (e.g., pre-storm activity vs. magnetic storms) has the potential to vastly improve space weather diagnosis and, consequently, the mitigation of space weather hazards. This review provides a systematic overview on existing nonlinear dynamical systems-based methodologies along with key results of their previous applications in a space physics context, which particularly illustrates how complementary modern complex systems approaches have recently shaped our understanding of nonlinear magnetospheric variability. The rising number of corresponding studies demonstrates that the multiplicity of nonlinear time series analysis methods developed during the last decades offers great potentials for uncovering relevant yet complex processes interlinking different geospace subsystems, variables and spatiotemporal scales

Complexity in Economic and Social Systems

There is no term that better describes the essential features of human society than complexity. On various levels, from the decision-making processes of individuals, through to the interactions between individuals leading to the spontaneous formation of groups and social hierarchies, up to the collective, herding processes that reshape whole societies, all these features share the property of irreducibility, i.e., they require a holistic, multi-level approach formed by researchers from different disciplines. This Special Issue aims to collect research studies that, by exploiting the latest advances in physics, economics, complex networks, and data science, make a step towards understanding these economic and social systems. The majority of submissions are devoted to financial market analysis and modeling, including the stock and cryptocurrency markets in the COVID-19 pandemic, systemic risk quantification and control, wealth condensation, the innovation-related performance of companies, and more. Looking more at societies, there are papers that deal with regional development, land speculation, and the-fake news-fighting strategies, the issues which are of central interest in contemporary society. On top of this, one of the contributions proposes a new, improved complexity measure