Multivariable generalizations of the Schur class: positive kernel characterization and transfer function realization

The operator-valued Schur-class is defined to be the set of holomorphic functions $S$ mapping the unit disk into the space of contraction operators between two Hilbert spaces. There are a number of alternate characterizations: the operator of multiplication by $S$ defines a contraction operator between two Hardy Hilbert spaces, $S$ satisfies a von Neumann inequality, a certain operator-valued kernel associated with $S$ is positive-definite, and $S$ can be realized as the transfer function of a dissipative (or even conservative) discrete-time linear input/state/output linear system. Various multivariable generalizations of this class have appeared recently,one of the most encompassing being that of Muhly and Solel where the unit disk is replaced by the strict unit ball of the elements of a dual correspondence $E^{\sigma}$ associated with a $W^{*}$-correspondence $E$ over a $W^{*}$-algebra \cA together with a $*$-representation $\sigma$ of \cA. The main new point which we add here is the introduction of the notion of reproducing kernel Hilbert correspondence and identification of the Muhly-Solel Hardy spaces as reproducing kernel Hilbert correspondences associated with a completely positive analogue of the classical Szeg\"o kernel. In this way we are able to make the analogy between the Muhly-Solel Schur class and the classical Schur class more complete. We also illustrate the theory by specializing it to some well-studied special cases; in some instances there result new kinds of realization theorems.Comment: adjusted the definition of completely positve kernel on page 12 and did minor modifications corresponding to this adjustmen

Weighted Bergman spaces: shift-invariant subspaces and input/state/output linear systems

It is well known that subspaces of the Hardy space over the unit disk which are invariant under the backward shift occur as the image of an observability operator associated with a discrete-time linear system with stable state-dynamics, as well as the functional-model space for a Hilbert space contraction operator, while forward shift-invariant subspaces have a representation in terms of an inner function. We discuss several variants of these statements in the context of weighted Bergman spaces on the unit disk

A Novel Fourier Theory on Non-linear Phases and Applications

Positive time varying frequency representation for transient signals has been a hearty desire of signal analysts due to its theoretical and practical importance. During approximately the last two decades there has formulated a signal decomposition and reconstruction method rooted in harmonic and complex analysis giving rise to the desired signal representation. The method decomposes any signal into a few basic signals that possess positive instantaneous frequencies. The theory has profound relations with classical mathematics and can be generalized to signals defined in higher dimensional manifolds with vector and matrix values, and in particular, promotes rational approximation in higher dimensions. This article mainly serves as a survey. It also gives a new proof for a general convergence result, as well as a proof for the necessity of multiple selection of the parameters. Mono-components are crucial to understand the concept instantaneous frequency. We will present several most important mono-component function classes. Decompositions of signals into mono-components are called adaptive Fourier decompositions (AFDs). We note that some scopes of the studies on the 1D mono-components and AFDs can be extended to vector-valued or even matrix-valued signals defined on higher dimensional manifolds. We finally provide an account of related studies in pure and applied mathematics, and in signal analysis, as well as applications of the theory found in the literature.Comment: 33 page

Function spaces obeying a time-varying bandlimit

Motivated by applications to signal processing and mathematical physics, recent work on the concept of time-varying bandwidth has produced a class of function spaces which generalize the Paley-Wiener spaces of bandlimited functions: any regular simple symmetric linear transformation with deficiency indices $(1,1)$ is naturally represented as multiplication by the independent variable in one of these spaces. We explicitly demonstrate the equivalence of this model for such linear transformations to several other functional models based on the theories of meromorphic model spaces of Hardy space and purely atomic Herglotz measures on the real line, respectively. This theory provides a precise notion of a time-varying or local bandwidth, and we describe how it may be applied to construct signal processing techniques that are adapted to signals obeying a time-varying bandlimit

Adaptive Estimation for Nonlinear Systems using Reproducing Kernel Hilbert Spaces

This paper extends a conventional, general framework for online adaptive estimation problems for systems governed by unknown nonlinear ordinary differential equations. The central feature of the theory introduced in this paper represents the unknown function as a member of a reproducing kernel Hilbert space (RKHS) and defines a distributed parameter system (DPS) that governs state estimates and estimates of the unknown function. This paper 1) derives sufficient conditions for the existence and stability of the infinite dimensional online estimation problem, 2) derives existence and stability of finite dimensional approximations of the infinite dimensional approximations, and 3) determines sufficient conditions for the convergence of finite dimensional approximations to the infinite dimensional online estimates. A new condition for persistency of excitation in a RKHS in terms of its evaluation functionals is introduced in the paper that enables proof of convergence of the finite dimensional approximations of the unknown function in the RKHS. This paper studies two particular choices of the RKHS, those that are generated by exponential functions and those that are generated by multiscale kernels defined from a multiresolution analysis.Comment: 24 pages, Submitted to CMAM

Online dictionary learning for kernel LMS. Analysis and forward-backward splitting algorithm

Adaptive filtering algorithms operating in reproducing kernel Hilbert spaces have demonstrated superiority over their linear counterpart for nonlinear system identification. Unfortunately, an undesirable characteristic of these methods is that the order of the filters grows linearly with the number of input data. This dramatically increases the computational burden and memory requirement. A variety of strategies based on dictionary learning have been proposed to overcome this severe drawback. Few, if any, of these works analyze the problem of updating the dictionary in a time-varying environment. In this paper, we present an analytical study of the convergence behavior of the Gaussian least-mean-square algorithm in the case where the statistics of the dictionary elements only partially match the statistics of the input data. This allows us to emphasize the need for updating the dictionary in an online way, by discarding the obsolete elements and adding appropriate ones. We introduce a kernel least-mean-square algorithm with L1-norm regularization to automatically perform this task. The stability in the mean of this method is analyzed, and its performance is tested with experiments

Random Euler Complex-Valued Nonlinear Filters

Over the last decade, both the neural network and kernel adaptive filter have successfully been used for nonlinear signal processing. However, they suffer from high computational cost caused by their complex/growing network structures. In this paper, we propose two random Euler filters for complex-valued nonlinear filtering problem, i.e., linear random Euler complex-valued filter (LRECF) and its widely-linear version (WLRECF), which possess a simple and fixed network structure. The transient and steady-state performances are studied in a non-stationary environment. The analytical minimum mean square error (MSE) and optimum step-size are derived. Finally, numerical simulations on complex-valued nonlinear system identification and nonlinear channel equalization are presented to show the effectiveness of the proposed methods

Two-Manifold Problems with Applications to Nonlinear System Identification

Recently, there has been much interest in spectral approaches to learning manifolds---so-called kernel eigenmap methods. These methods have had some successes, but their applicability is limited because they are not robust to noise. To address this limitation, we look at two-manifold problems, in which we simultaneously reconstruct two related manifolds, each representing a different view of the same data. By solving these interconnected learning problems together, two-manifold algorithms are able to succeed where a non-integrated approach would fail: each view allows us to suppress noise in the other, reducing bias. We propose a class of algorithms for two-manifold problems, based on spectral decomposition of cross-covariance operators in Hilbert space, and discuss when two-manifold problems are useful. Finally, we demonstrate that solving a two-manifold problem can aid in learning a nonlinear dynamical system from limited data.Comment: ICML2012. arXiv admin note: text overlap with arXiv:1112.639

Sufficient Conditions for Parameter Convergence over Embedded Manifolds using Kernel Techniques

The persistence of excitation (PE) condition is sufficient to ensure parameter convergence in adaptive estimation problems. Recent results on adaptive estimation in reproducing kernel Hilbert spaces (RKHS) introduce PE conditions for RKHS. This paper presents sufficient conditions for PE for the particular class of uniformly embedded reproducing kernel Hilbert spaces (RKHS) defined over smooth Riemannian manifolds. This paper also studies the implications of the sufficient condition in the case when the RKHS is finite or infinite-dimensional. When the RKHS is finite-dimensional, the sufficient condition implies parameter convergence as in the conventional analysis. On the other hand, when the RKHS is infinite-dimensional, the same condition implies that the function estimate error is ultimately bounded by a constant that depends on the approximation error in the infinite-dimensional RKHS. We illustrate the effectiveness of the sufficient condition in a practical example.Comment: 11 pages, 2 figure

Short-term time series prediction using Hilbert space embeddings of autoregressive processes

Linear autoregressive models serve as basic representations of discrete time stochastic processes. Different attempts have been made to provide non-linear versions of the basic autoregressive process, including different versions based on kernel methods. Motivated by the powerful framework of Hilbert space embeddings of distributions, in this paper we apply this methodology for the kernel embedding of an autoregressive process of order $p$. By doing so, we provide a non-linear version of an autoregressive process, that shows increased performance over the linear model in highly complex time series. We use the method proposed for one-step ahead forecasting of different time-series, and compare its performance against other non-linear methods