Bayesian functional linear regression with sparse step functions

The functional linear regression model is a common tool to determine the relationship between a scalar outcome and a functional predictor seen as a function of time. This paper focuses on the Bayesian estimation of the support of the coefficient function. To this aim we propose a parsimonious and adaptive decomposition of the coefficient function as a step function, and a model including a prior distribution that we name Bayesian functional Linear regression with Sparse Step functions (Bliss). The aim of the method is to recover areas of time which influences the most the outcome. A Bayes estimator of the support is built with a specific loss function, as well as two Bayes estimators of the coefficient function, a first one which is smooth and a second one which is a step function. The performance of the proposed methodology is analysed on various synthetic datasets and is illustrated on a black P\'erigord truffle dataset to study the influence of rainfall on the production

Error Bounds for Piecewise Smooth and Switching Regression

The paper deals with regression problems, in which the nonsmooth target is assumed to switch between different operating modes. Specifically, piecewise smooth (PWS) regression considers target functions switching deterministically via a partition of the input space, while switching regression considers arbitrary switching laws. The paper derives generalization error bounds in these two settings by following the approach based on Rademacher complexities. For PWS regression, our derivation involves a chaining argument and a decomposition of the covering numbers of PWS classes in terms of the ones of their component functions and the capacity of the classifier partitioning the input space. This yields error bounds with a radical dependency on the number of modes. For switching regression, the decomposition can be performed directly at the level of the Rademacher complexities, which yields bounds with a linear dependency on the number of modes. By using once more chaining and a decomposition at the level of covering numbers, we show how to recover a radical dependency. Examples of applications are given in particular for PWS and swichting regression with linear and kernel-based component functions.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice,after which this version may no longer be accessibl

Evolutionary Computation in System Identification: Review and Recommendations

Two of the steps in system identification are model structure selection and parameter estimation. In model structure selection, several model structures are evaluated and selected. Because the evaluation of all possible model structures during selection and estimation of the parameters requires a lot of time, a rigorous method in which these tasks can be simplified is usually preferred. This paper reviews cumulatively some of the methods that have been tried since the past 40 years. Among the methods, evolutionary computation is known to be the most recent one and hereby being reviewed in more detail, including what advantages the method contains and how it is specifically implemented. At the end of the paper, some recommendations are provided on how evolutionary computation can be utilized in a more effective way. In short, these are by modifying the search strategy and simplifying the procedure based on problem a priori knowledge

A Deterministic Annealing Framework for Global Optimization of Delay-Constrained Communication and Control Strategies

This dissertation is concerned with the problem of global optimization of delay constrained communication and control strategies. Specifically, the objective is to obtain optimal encoder and decoder functions that map between the source space and the channel space, to minimize a given cost functional. The cost surfaces associated with these problems are highly complex and riddled with local minima, rendering gradient descent based methods ineffective. This thesis proposes and develops a powerful non-convex optimization method based on the concept of deterministic annealing (DA) - which is derived from information theoretic principles with analogies to statistical physics, and was successfully employed in several problems including vector quantization, classification and regression. DA has several useful properties including reduced sensitivity to initialization and strong potential to avoid poor local minima. DA-based optimization methods are developed here for the following fundamental communication problems: the Wyner-Ziv setting where only a decoder has access to side information, the distributed setting where independent encoders transmit over independent channels to a central decoder, and analog multiple descriptions setting which is an extension of the well known source coding problem of multiple descriptions. Comparative numerical results are presented, which show strict superiority of the proposed method over gradient descent based optimization methods as well as prior approaches in literature. Detailed analysis of the highly non-trivial structure of obtained mappings is provided. The thesis further studies the related problem of global optimization of controller mappings in decentralized stochastic control problems, including Witsenhausen's celebrated 1968 counter-example. It is well-known that most decentralized control problems do not admit closed-form solutions and require numerical optimization. An optimization method is developed, based on DA, for a class of decentralized stochastic control problems. Comparative numerical results are presented for two test problems that show strict superiority of the proposed method over prior approaches in literature, and analyze the structure of obtained controller functions