Regularized Nonparametric Volterra Kernel Estimation

In this paper, the regularization approach introduced recently for nonparametric estimation of linear systems is extended to the estimation of nonlinear systems modelled as Volterra series. The kernels of order higher than one, representing higher dimensional impulse responses in the series, are considered to be realizations of multidimensional Gaussian processes. Based on this, prior information about the structure of the Volterra kernel is introduced via an appropriate penalization term in the least squares cost function. It is shown that the proposed method is able to deliver accurate estimates of the Volterra kernels even in the case of a small amount of data points

Sparse Volterra and Polynomial Regression Models: Recoverability and Estimation

Volterra and polynomial regression models play a major role in nonlinear system identification and inference tasks. Exciting applications ranging from neuroscience to genome-wide association analysis build on these models with the additional requirement of parsimony. This requirement has high interpretative value, but unfortunately cannot be met by least-squares based or kernel regression methods. To this end, compressed sampling (CS) approaches, already successful in linear regression settings, can offer a viable alternative. The viability of CS for sparse Volterra and polynomial models is the core theme of this work. A common sparse regression task is initially posed for the two models. Building on (weighted) Lasso-based schemes, an adaptive RLS-type algorithm is developed for sparse polynomial regressions. The identifiability of polynomial models is critically challenged by dimensionality. However, following the CS principle, when these models are sparse, they could be recovered by far fewer measurements. To quantify the sufficient number of measurements for a given level of sparsity, restricted isometry properties (RIP) are investigated in commonly met polynomial regression settings, generalizing known results for their linear counterparts. The merits of the novel (weighted) adaptive CS algorithms to sparse polynomial modeling are verified through synthetic as well as real data tests for genotype-phenotype analysis.Comment: 20 pages, to appear in IEEE Trans. on Signal Processin

Volterra black-box models identification methods: direct collocation vs least squares

The Volterra integral-functional series is the classic approach for nonlinear black box dynamical systems modeling. It is widely employed in many domains including radiophysics, aerodynamics, electronic and electrical engineering and many other. Identifying the time-varying functional parameters, also known as Volterra kernels, poses a difficulty due to the curse of dimensionality. This refers to the exponential growth in the number of model parameters as the complexity of the input-output response increases. The least squares method (LSM) is widely acknowledged as the standard approach for tackling the issue of identifying parameters. Unfortunately, the LSM suffers with many drawbacks such as the sensitivity to outliers causing biased estimation, multicollinearity, overfitting and inefficiency with large datasets. This paper presents alternative approach based on direct estimation of the Volterra kernels using the collocation method. Two model examples are studied. It is found that the collocation method presents a promising alternative for optimization, surpassing the traditional least squares method when it comes to the Volterra kernels identification including the case when input and output signals suffer from considerable measurement errors

Kernel-based methods for Volterra series identification

Volterra series approximate a broad range of nonlinear systems. Their identification is challenging due to the curse of dimensionality: the number of model parameters grows exponentially with the complexity of the input-output response. This fact limits the applicability of such models and has stimulated recently much research on regularized solutions. Along this line, we propose two new strategies that use kernel-based methods. First, we introduce the multiplicative polynomial kernel (MPK). Compared to the standard polynomial kernel, the MPK is equipped with a richer set of hyperparameters, increasing flexibility in selecting the monomials that really influence the system output. Second, we introduce the smooth exponentially decaying multiplicative polynomial kernel (SEDMPK), that is a regularized version of MPK which requires less hyperparameters, allowing to handle also high-order Volterra series. Numerical results show the effectiveness of the two approaches. (C) 2021 Elsevier Ltd. All rights reserved

State–of–the–art report on nonlinear representation of sources and channels

This report consists of two complementary parts, related to the modeling of two important sources of nonlinearities in a communications system. In the first part, an overview of important past work related to the estimation, compression and processing of sparse data through the use of nonlinear models is provided. In the second part, the current state of the art on the representation of wireless channels in the presence of nonlinearities is summarized. In addition to the characteristics of the nonlinear wireless fading channel, some information is also provided on recent approaches to the sparse representation of such channels

Stage-discharge relationship in tidal channels

Author Posting. © The Author(s), 2016. This is the author's version of the work. It is posted here by permission of Association for the Sciences of Limnology and Oceanography for personal use, not for redistribution. The definitive version was published in Limnology and Oceanography: Methods 15 (2017): 394–407, doi:10.1002/lom3.10168.Long-term records of the flow of water through tidal channels are essential to constrain the budgets of sediments and biogeochemical compounds in salt marshes. Statistical models which relate discharge to water level allow the estimation of such records from more easily obtained records of water stage in the channel. Here we compare four different types of stage-discharge models, each of which captures different characteristics of the stage-discharge relationship. We estimate and validate each of these models on a two-month long time series of stage and discharge obtained with an Acoustic Doppler Current Profiler in a salt marsh channel. We find that the best performance is obtained by models that account for the nonlinear and time-varying nature of the stage-discharge relationship. Good performance can also be obtained from a simplified version of these models, which captures nonlinearity and nonstationarity without the complexity of the fully nonlinear or time-varying models.This research was supported by the National Science Foundation (awards OCE1354251, OCE1354494, and OCE1238212).2018-04-2

Modelling and Analysis of Drosophila Early Visual System A Systems Engineering Approach

Over the past century or so Drosophila has been established as an ideal model organism to study, among other things, neural computation and in particular sensory processing. In this respect there are many features that make Drosophila an ideal model organism, especially the fact that it offers a vast amount of genetic and experimental tools for manipulating and interrogating neural circuits. Whilst comprehensive models of sensory processing in Drosophila are not yet available, considerable progress has been made in recent years in modelling the early stages of sensory processing. When it comes to visual processing, accurate empirical and biophysical models of the R1-R6 photoreceptors were developed and used to characterize nonlinear processing at photoreceptor level and to demonstrate that R1-R6 photoreceptors encode phase congruency. A limitation of the latest photoreceptor models is that these do not account explicitly for the modulation of photoreceptor responses by the network of interneurones hosted in the lamina. As a consequence, these models cannot describe in a unifying way the photoreceptor response in the absence of the feedback from the downstream neurons and thus cannot be used to elucidate the role of interneurones in photoreceptor adaptation. In this thesis, electrophysiological photoreceptor recordings acquired in-vivo from wild- type and histamine defficient mutant fruit flies are used to develop and validate new com- prehensive models of R1-R6 photoreceptors, which not only predict the response of these photoreceptors in wild-type and mutant fruit flies, over the entire environmental range of light intensities but also characterize explicitly the contribution of lamina neurons to photore- ceptor adaptation. As a consequence, the new models provide suitable building blocks for assembling a complete model of the retina which takes into account the true connectivity between photoreceptors and downstream interneurones. A recent study has demonstrated that R1-R6 photoreceptors employ nonlinear processing to selectively encode and enhance temporal phase congruency. It has been suggested that this processing strategy achieves an optimal trade-off between the two competing goals of minimizing distortion in decoding behaviourally relevant stimuli features and minimizing the information rate, which ultimately enables more efficient downstream processing of spatio-temporal visual stimuli for edge and motion detection.Using rigorous information theoretic tools, this thesis derives and analyzes the rate-distortion characteristics associated with the linear and nonlinear transformations performed by photoreceptors on a stimulus generated by a signal source with a well defined distribution