JIDT: An information-theoretic toolkit for studying the dynamics of complex systems

Complex systems are increasingly being viewed as distributed information processing systems, particularly in the domains of computational neuroscience, bioinformatics and Artificial Life. This trend has resulted in a strong uptake in the use of (Shannon) information-theoretic measures to analyse the dynamics of complex systems in these fields. We introduce the Java Information Dynamics Toolkit (JIDT): a Google code project which provides a standalone, (GNU GPL v3 licensed) open-source code implementation for empirical estimation of information-theoretic measures from time-series data. While the toolkit provides classic information-theoretic measures (e.g. entropy, mutual information, conditional mutual information), it ultimately focusses on implementing higher-level measures for information dynamics. That is, JIDT focusses on quantifying information storage, transfer and modification, and the dynamics of these operations in space and time. For this purpose, it includes implementations of the transfer entropy and active information storage, their multivariate extensions and local or pointwise variants. JIDT provides implementations for both discrete and continuous-valued data for each measure, including various types of estimator for continuous data (e.g. Gaussian, box-kernel and Kraskov-Stoegbauer-Grassberger) which can be swapped at run-time due to Java's object-oriented polymorphism. Furthermore, while written in Java, the toolkit can be used directly in MATLAB, GNU Octave, Python and other environments. We present the principles behind the code design, and provide several examples to guide users.Comment: 37 pages, 4 figure

Geometry of free loci and factorization of noncommutative polynomials

The free singularity locus of a noncommutative polynomial f is defined to be the sequence $Z_n(f)=\{X\in M_n^g : \det f(X)=0\}$ of hypersurfaces. The main theorem of this article shows that f is irreducible if and only if $Z_n(f)$ is eventually irreducible. A key step in the proof is an irreducibility result for linear pencils. Apart from its consequences to factorization in a free algebra, the paper also discusses its applications to invariant subspaces in perturbation theory and linear matrix inequalities in real algebraic geometry.Comment: v2: 32 pages, includes a table of content

Realization of 2D convolutional codes of rate 1/n by separable Roesser models

In this paper, two-dimensional convolutional codes constituted by sequences in where is a finite field, are considered. In particular, we restrict to codes with rate and we investigate the problem of minimal dimension for realizations of such codes by separable Roesser models. The encoders which allow to obtain such minimal realizations, called R-minimal encoders, are characterized

Spin glass overlap barriers in three and four dimensions

For the Edwards-Anderson Ising spin-glass model in three and four dimensions (3d and 4d) we have performed high statistics Monte Carlo calculations of those free-energy barriers $F^q_B$ which are visible in the probability density $P_J(q)$ of the Parisi overlap parameter $q$. The calculations rely on the recently introduced multi-overlap algorithm. In both dimensions, within the limits of lattice sizes investigated, these barriers are found to be non-self-averaging and the same is true for the autocorrelation times of our algorithm. Further, we present evidence that barriers hidden in $q$ dominate the canonical autocorrelation times.Comment: 20 pages, Latex, 12 Postscript figures, revised version to appear in Phys. Rev.

Rigid Transformations for Stabilized Lower Dimensional Space to Support Subsurface Uncertainty Quantification and Interpretation

Subsurface datasets inherently possess big data characteristics such as vast volume, diverse features, and high sampling speeds, further compounded by the curse of dimensionality from various physical, engineering, and geological inputs. Among the existing dimensionality reduction (DR) methods, nonlinear dimensionality reduction (NDR) methods, especially Metric-multidimensional scaling (MDS), are preferred for subsurface datasets due to their inherent complexity. While MDS retains intrinsic data structure and quantifies uncertainty, its limitations include unstabilized unique solutions invariant to Euclidean transformations and an absence of out-of-sample points (OOSP) extension. To enhance subsurface inferential and machine learning workflows, datasets must be transformed into stable, reduced-dimension representations that accommodate OOSP. Our solution employs rigid transformations for a stabilized Euclidean invariant representation for LDS. By computing an MDS input dissimilarity matrix, and applying rigid transformations on multiple realizations, we ensure transformation invariance and integrate OOSP. This process leverages a convex hull algorithm and incorporates loss function and normalized stress for distortion quantification. We validate our approach with synthetic data, varying distance metrics, and real-world wells from the Duvernay Formation. Results confirm our method's efficacy in achieving consistent LDS representations. Furthermore, our proposed "stress ratio" (SR) metric provides insight into uncertainty, beneficial for model adjustments and inferential analysis. Consequently, our workflow promises enhanced repeatability and comparability in NDR for subsurface energy resource engineering and associated big data workflows.Comment: 30 pages, 17 figures, Submitted to Computational Geosciences Journa