Matrix product representation and synthesis for random vectors: Insight from statistical physics

Inspired from modern out-of-equilibrium statistical physics models, a matrix product based framework permits the formal definition of random vectors (and random time series) whose desired joint distributions are a priori prescribed. Its key feature consists of preserving the writing of the joint distribution as the simple product structure it has under independence, while inputing controlled dependencies amongst components: This is obtained by replacing the product of distributions by a product of matrices of distributions. The statistical properties stemming from this construction are studied theoretically: The landscape of the attainable dependence structure is thoroughly depicted and a stationarity condition for time series is notably obtained. The remapping of this framework onto that of Hidden Markov Models enables us to devise an efficient and accurate practical synthesis procedure. A design procedure is also described permitting the tuning of model parameters to attain targeted properties. Pedagogical well-chosen examples of times series and multivariate vectors aim at illustrating the power and versatility of the proposed approach and at showing how targeted statistical properties can be actually prescribed.Comment: 10 pages, 4 figures, submitted to IEEE Transactions on Signal Processin

Asymptotically exact data augmentation : models and Monte Carlo sampling with applications to Bayesian inference

Numerous machine learning and signal/image processing tasks can be formulated as statistical inference problems. As an archetypal example, recommendation systems rely on the completion of partially observed user/item matrix, which can be conducted via the joint estimation of latent factors and activation coefficients. More formally, the object to be inferred is usually defined as the solution of a variational or stochastic optimization problem. In particular, within a Bayesian framework, this solution is defined as the minimizer of a cost function, referred to as the posterior loss. In the simple case when this function is chosen as quadratic, the Bayesian estimator is known to be the posterior mean which minimizes the mean square error and defined as an integral according to the posterior distribution. In most real-world applicative contexts, computing such integrals is not straightforward. One alternative lies in making use of Monte Carlo integration, which consists in approximating any expectation according to the posterior distribution by an empirical average involving samples from the posterior. This so-called Monte Carlo integration requires the availability of efficient algorithmic schemes able to generate samples from a desired posterior distribution. A huge literature dedicated to random variable generation has proposed various Monte Carlo algorithms. For instance, Markov chain Monte Carlo (MCMC) methods, whose particular instances are the famous Gibbs sampler and Metropolis-Hastings algorithm, define a wide class of algorithms which allow a Markov chain to be generated with the desired stationary distribution. Despite their seemingly simplicity and genericity, conventional MCMC algorithms may be computationally inefficient for large-scale, distributed and/or highly structured problems. The main objective of this thesis consists in introducing new models and related MCMC approaches to alleviate these issues. The intractability of the posterior distribution is tackled by proposing a class of approximate but asymptotically exact augmented (AXDA) models. Then, two Gibbs samplers targetting approximate posterior distributions based on the AXDA framework, are proposed and their benefits are illustrated on challenging signal processing, image processing and machine learning problems. A detailed theoretical study of the convergence rates associated to one of these two Gibbs samplers is also conducted and reveals explicit dependences with respect to the dimension, condition number of the negative log-posterior and prescribed precision. In this work, we also pay attention to the feasibility of the sampling steps involved in the proposed Gibbs samplers. Since one of this step requires to sample from a possibly high-dimensional Gaussian distribution, we review and unify existing approaches by introducing a framework which stands for the stochastic counterpart of the celebrated proximal point algorithm. This strong connection between simulation and optimization is not isolated in this thesis. Indeed, we also show that the derived Gibbs samplers share tight links with quadratic penalty methods and that the AXDA framework yields a class of envelope functions related to the Moreau one

Bayesian estimation of the parameters of the joint multifractal spectrum of signals and images

Multifractal analysis has become a reference tool for signal and image processing. Grounded in the quantification of local regularity fluctuations, it has proven useful in an increasing range of applications, yet so far involving only univariate data (scalar-valued time series or single channel images). Recently the theoretical ground for multivariate multifractal analysis has been devised, showing potential for quantifying transient higher-order dependence beyond linear correlation among collections of data. However, the accurate estimation of the parameters associated with a multivariate multifractal model remains challenging, severely limiting their actual use in applications. The main goal of this thesis is to propose and study practical contributions on multivariate multifractal analysis of signals and images. Specifically, the proposed approach relies on a novel and original joint Gaussian model for the logarithm of wavelet leaders and leverages on a Whittle-based likelihood approximation and data augmentation for the matrix-valued parameters of interest. This careful design enables efficient estimation procedures to be constructed for two relevant choices of priors using Bayesian inference. Algorithms based on Monte Carlo Markov Chain and Expectation Maximization strategies are designed and used to approximate the Bayesian estimators. Monte Carlo simulations, conducted on synthetic multivariate signals and images with various sample sizes, numbers of components and multifractal parameter settings, demonstrate significant performance improvements over the state of the art. In addition, theoretical lower bounds on the variance of the estimators are designed to study their asymptotic behavior. Finally, the relevance of the proposed multivariate multifractal estimation framework is shown for two real-world data examples: drowsiness detection from multichannel physiological signals and potential remote sensing applications in multispectral satellite imagery

Multifractal analysis for multivariate data with application to remote sensing

Texture characterization is a central element in many image processing applications. Texture analysis can be embedded in the mathematical framework of multifractal analysis, enabling the study of the fluctuations in regularity of image intensity and providing practical tools for their assessment, the coefficients or wavelet leaders. Although successfully applied in various contexts, multi fractal analysis suffers at present from two major limitations. First, the accurate estimation of multifractal parameters for image texture remains a challenge, notably for small sample sizes. Second, multifractal analysis has so far been limited to the analysis of a single image, while the data available in applications are increasingly multivariate. The main goal of this thesis is to develop practical contributions to overcome these limitations. The first limitation is tackled by introducing a generic statistical model for the logarithm of wavelet leaders, parametrized by multifractal parameters of interest. This statistical model enables us to counterbalance the variability induced by small sample sizes and to embed the estimation in a Bayesian framework. This yields robust and accurate estimation procedures, effective both for small and large images. The multifractal analysis of multivariate images is then addressed by generalizing this Bayesian framework to hierarchical models able to account for the assumption that multifractal properties evolve smoothly in the dataset. This is achieved via the design of suitable priors relating the dynamical properties of the multifractal parameters of the different components composing the dataset. Different priors are investigated and compared in this thesis by means of numerical simulations conducted on synthetic multivariate multifractal images. This work is further completed by the investigation of the potential benefit of multifractal analysis and the proposed Bayesian methodology for remote sensing via the example of hyperspectral imaging

Control Theory: On the Way to New Application Fields

Control theory is an interdisciplinary field that is located at the crossroads of pure and applied mathematics with systems engineering and the sciences. Recently, deep interactions are emerging with new application areas, such as systems biology, quantum control and information technology. In order to address the new challenges posed by the new application disciplines, a special focus of this workshop has been on the interaction between control theory and mathematical systems biology. To complement these more biology oriented focus, a series of lectures in this workshop was devoted to the control of networks of systems, fundamentals of nonlinear control systems, model reduction and identification, algorithmic aspects in control, as well as open problems in control

Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)

The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in Belgium, from Wednesday August 27th till Friday August 29th, 2014. The workshop was conveniently located in "The Arsenal" building within walking distance of both hotels and town center. iTWIST'14 has gathered about 70 international participants and has featured 9 invited talks, 10 oral presentations, and 14 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing; Union of low dimensional subspaces; Beyond linear and convex inverse problem; Matrix/manifold/graph sensing/processing; Blind inverse problems and dictionary learning; Sparsity and computational neuroscience; Information theory, geometry and randomness; Complexity/accuracy tradeoffs in numerical methods; Sparsity? What's next?; Sparse machine learning and inference.Comment: 69 pages, 24 extended abstracts, iTWIST'14 website: http://sites.google.com/site/itwist1

Untangling hotel industry’s inefficiency: An SFA approach applied to a renowned Portuguese hotel chain

The present paper explores the technical efficiency of four hotels from Teixeira Duarte Group - a renowned Portuguese hotel chain. An efficiency ranking is established from these four hotel units located in Portugal using Stochastic Frontier Analysis. This methodology allows to discriminate between measurement error and systematic inefficiencies in the estimation process enabling to investigate the main inefficiency causes. Several suggestions concerning efficiency improvement are undertaken for each hotel studied.info:eu-repo/semantics/publishedVersio

Genetic determination and layout rules of visual cortical architecture

The functional architecture of the primary visual cortex is set up by neurons that preferentially respond to visual stimuli with contours of a specific orientation in visual space. In primates and placental carnivores, orientation preference is arranged into continuous and roughly repetitive (iso-) orientation domains. Exceptions are pinwheels that are surrounded by all orientation preferences. The configuration of pinwheels adheres to quantitative species-invariant statistics, the common design. This common design most likely evolved independently at least twice in the course of the past 65 million years, which might indicate a functionally advantageous trait. The possible acquisition of environment-dependent functional traits by genes, the Baldwin effect, makes it conceivable that visual cortical architecture is partially or redundantly encoded by genetic information. In this conception, genetic mechanisms support the emergence of visual cortical architecture or even establish it under unfavorable environments. In this dissertation, I examine the capability of genetic mechanisms for encoding visual cortical architecture and mathematically dissect the pinwheel configuration under measurement noise as well as in different geometries. First, I theoretically explore possible roles of genetic mechanisms in visual cortical development that were previously excluded from theoretical research, mostly because the information capacity of the genome appeared too small to contain a blueprint for wiring up the cortex. For the first time, I provide a biologically plausible scheme for quantitatively encoding functional visual cortical architecture by genetic information that circumvents the alleged information bottleneck. Key ingredients for this mechanism are active transport and trans-neuronal signaling as well as joined dynamics of morphogens and connectome. This theory provides predictions for experimental tests and thus may help to clarify the relative importance of genes and environments on complex human traits. Second, I disentangle the link between orientation domain ensembles and the species-invariant pinwheel statistics of the common design. This examination highlights informative measures of pinwheel configurations for model benchmarking. Third, I mathematically investigate the susceptibility of the pinwheel configuration to measurement noise. The results give rise to an extrapolation method of pinwheel densities to the zero noise limit and provide an approximated analytical expression for confidence regions of pinwheel centers. Thus, the work facilitates high-precision measurements and enhances benchmarking for devising more accurate models of visual cortical development. Finally, I shed light on genuine three-dimensional properties of functional visual cortical architectures. I devise maximum entropy models of three-dimensional functional visual cortical architectures in different geometries. This theory enables the examination of possible evolutionary transitions between different functional architectures for which intermediate organizations might still exist