1,443 research outputs found
The k-error Linear Complexity Distribution for Periodic Sequences
This thesis proposes various novel approaches for studying the k-error linear complexity distribution of periodic binary sequences for k > 2, and the second descent point and beyond of k-error linear complexity critical error points. We present a new tool called Cube Theory. Based on Games-Chan algorithm and the cube theory, a constructive approach is presented to construct periodic sequences with the given k-error linear complexity profile. All examples are verified by computer programs
Predictability: a way to characterize Complexity
Different aspects of the predictability problem in dynamical systems are
reviewed. The deep relation among Lyapunov exponents, Kolmogorov-Sinai entropy,
Shannon entropy and algorithmic complexity is discussed. In particular, we
emphasize how a characterization of the unpredictability of a system gives a
measure of its complexity. Adopting this point of view, we review some
developments in the characterization of the predictability of systems showing
different kind of complexity: from low-dimensional systems to high-dimensional
ones with spatio-temporal chaos and to fully developed turbulence. A special
attention is devoted to finite-time and finite-resolution effects on
predictability, which can be accounted with suitable generalization of the
standard indicators. The problems involved in systems with intrinsic randomness
is discussed, with emphasis on the important problems of distinguishing chaos
from noise and of modeling the system. The characterization of irregular
behavior in systems with discrete phase space is also considered.Comment: 142 Latex pgs. 41 included eps figures, submitted to Physics Reports.
Related information at this http://axtnt2.phys.uniroma1.i
Predictability, complexity and learning
We define {\em predictive information} as the mutual
information between the past and the future of a time series. Three
qualitatively different behaviors are found in the limit of large observation
times : can remain finite, grow logarithmically, or grow
as a fractional power law. If the time series allows us to learn a model with a
finite number of parameters, then grows logarithmically with
a coefficient that counts the dimensionality of the model space. In contrast,
power--law growth is associated, for example, with the learning of infinite
parameter (or nonparametric) models such as continuous functions with
smoothness constraints. There are connections between the predictive
information and measures of complexity that have been defined both in learning
theory and in the analysis of physical systems through statistical mechanics
and dynamical systems theory. Further, in the same way that entropy provides
the unique measure of available information consistent with some simple and
plausible conditions, we argue that the divergent part of
provides the unique measure for the complexity of dynamics underlying a time
series. Finally, we discuss how these ideas may be useful in different problems
in physics, statistics, and biology.Comment: 53 pages, 3 figures, 98 references, LaTeX2
Applied Harmonic Analysis and Data Science (hybrid meeting)
Data science has become a field of major importance for science and technology
nowadays and poses a large variety of
challenging mathematical questions.
The area
of applied harmonic analysis has a significant impact on such problems by providing methodologies
both for theoretical questions and for a wide range of applications
in signal and image processing and machine learning.
Building on the success of three previous workshops on applied harmonic analysis in 2012, 2015 and 2018,
this workshop focused
on several exciting novel directions such as mathematical theory of
deep learning, but also reported progress on long-standing open problems in the field
Black-Box Parallelization for Machine Learning
The landscape of machine learning applications is changing rapidly: large centralized datasets are replaced by high volume, high velocity data streams generated by a vast number of geographically distributed, loosely connected devices, such as mobile phones, smart sensors, autonomous vehicles or industrial machines. Current learning approaches centralize the data and process it in parallel in a cluster or computing center. This has three major disadvantages: (i) it does not scale well with the number of data-generating devices since their growth exceeds that of computing centers, (ii) the communication costs for centralizing the data are prohibitive in many applications, and (iii) it requires sharing potentially privacy-sensitive data. Pushing computation towards the data-generating devices alleviates these problems and allows to employ their otherwise unused computing power. However, current parallel learning approaches are designed for tightly integrated systems with low latency and high bandwidth, not for loosely connected distributed devices. Therefore, I propose a new paradigm for parallelization that treats the learning algorithm as a black box, training local models on distributed devices and aggregating them into a single strong one. Since this requires only exchanging models instead of actual data, the approach is highly scalable, communication-efficient, and privacy-preserving. Following this paradigm, this thesis develops black-box parallelizations for two broad classes of learning algorithms. One approach can be applied to incremental learning algorithms, i.e., those that improve a model in iterations. Based on the utility of aggregations it schedules communication dynamically, adapting it to the hardness of the learning problem. In practice, this leads to a reduction in communication by orders of magnitude. It is analyzed for (i) online learning, in particular in the context of in-stream learning, which allows to guarantee optimal regret and for (ii) batch learning based on empirical risk minimization where optimal convergence can be guaranteed. The other approach is applicable to non-incremental algorithms as well. It uses a novel aggregation method based on the Radon point that allows to achieve provably high model quality with only a single aggregation. This is achieved in polylogarithmic runtime on quasi-polynomially many processors. This relates parallel machine learning to Nick's class of parallel decision problems and is a step towards answering a fundamental open problem about the abilities and limitations of efficient parallel learning algorithms. An empirical study on real distributed systems confirms the potential of the approaches in realistic application scenarios
- …