An Iterative Shrinkage Approach to Total-Variation Image Restoration

The problem of restoration of digital images from their degraded measurements plays a central role in a multitude of practically important applications. A particularly challenging instance of this problem occurs in the case when the degradation phenomenon is modeled by an ill-conditioned operator. In such a case, the presence of noise makes it impossible to recover a valuable approximation of the image of interest without using some a priori information about its properties. Such a priori information is essential for image restoration, rendering it stable and robust to noise. Particularly, if the original image is known to be a piecewise smooth function, one of the standard priors used in this case is defined by the Rudin-Osher-Fatemi model, which results in total variation (TV) based image restoration. The current arsenal of algorithms for TV-based image restoration is vast. In the present paper, a different approach to the solution of the problem is proposed based on the method of iterative shrinkage (aka iterated thresholding). In the proposed method, the TV-based image restoration is performed through a recursive application of two simple procedures, viz. linear filtering and soft thresholding. Therefore, the method can be identified as belonging to the group of first-order algorithms which are efficient in dealing with images of relatively large sizes. Another valuable feature of the proposed method consists in its working directly with the TV functional, rather then with its smoothed versions. Moreover, the method provides a single solution for both isotropic and anisotropic definitions of the TV functional, thereby establishing a useful connection between the two formulae.Comment: The paper was submitted to the IEEE Transactions on Image Processing on October 22nd, 200

Data Assimilation: A Mathematical Introduction

These notes provide a systematic mathematical treatment of the subject of data assimilation

Dynamic Machine Learning with Least Square Objectives

As of the writing of this thesis, machine learning has become one of the most active research fields. The interest comes from a variety of disciplines which include computer science, statistics, engineering, and medicine. The main idea behind learning from data is that, when an analytical model explaining the observations is hard to find ---often in contrast to the models in physics such as Newton's laws--- a statistical approach can be taken where one or more candidate models are tuned using data. Since the early 2000's this challenge has grown in two ways: (i) The amount of collected data has seen a massive growth due to the proliferation of digital media, and (ii) the data has become more complex. One example for the latter is the high dimensional datasets, which can for example correspond to dyadic interactions between two large groups (such as customer and product information a retailer collects), or to high resolution image/video recordings. Another important issue is the study of dynamic data, which exhibits dependence on time. Virtually all datasets fall into this category as all data collection is performed over time, however I use the term dynamic to hint at a system with an explicit temporal dependence. A traditional example is target tracking from signal processing literature. Here the position of a target is modeled using Newton's laws of motion, which relates it to time via the target's velocity and acceleration. Dynamic data, as I defined above, poses two important challenges. Firstly, the learning setup is different from the standard theoretical learning setup, also known as Probably Approximately Correct (PAC) learning. To derive PAC learning bounds one assumes a collection of data points sampled independently and identically from a distribution which generates the data. On the other hand, dynamic systems produce correlated outputs. The learning systems we use should accordingly take this difference into consideration. Secondly, as the system is dynamic, it might be necessary to perform the learning online. In this case the learning has to be done in a single pass. Typical applications include target tracking and electricity usage forecasting. In this thesis I investigate several important dynamic and online learning problems, where I develop novel tools to address the shortcomings of the previous solutions in the literature. The work is divided into three parts for convenience. The first part is about matrix factorization for time series analysis which is further divided into two chapters. In the first chapter, matrix factorization is used within a Bayesian framework to model time-varying dyadic interactions, with examples in predicting user-movie ratings and stock prices. In the next chapter, a matrix factorization which uses autoregressive models to forecast future values of multivariate time series is proposed, with applications in predicting electricity usage and traffic conditions. Inspired by the machinery we use in the first part, the second part is about nonlinear Kalman filtering, where a hidden state is estimated over time given observations. The nonlinearity of the system generating the observations is the main challenge here, where a divergence minimization approach is used to unify the seemingly unrelated methods in the literature, and propose new ones. This has applications in target tracking and options pricing. The third and last part is about cost sensitive learning, where a novel method for maximizing area under receiver operating characteristics curve is proposed. Our method has theoretical guarantees and favorable sample complexity. The method is tested on a variety of benchmark datasets, and also has applications in online advertising

Multi-Bernoulli Sensor-Control via Minimization of Expected Estimation Errors

This paper presents a sensor-control method for choosing the best next state of the sensor(s), that provide(s) accurate estimation results in a multi-target tracking application. The proposed solution is formulated for a multi-Bernoulli filter and works via minimization of a new estimation error-based cost function. Simulation results demonstrate that the proposed method can outperform the state-of-the-art methods in terms of computation time and robustness to clutter while delivering similar accuracy

Sensor Signal and Information Processing II [Editorial]

This Special Issue compiles a set of innovative developments on the use of sensor signals and information processing. In particular, these contributions report original studies on a wide variety of sensor signals including wireless communication, machinery, ultrasound, imaging, and internet data, and information processing methodologies such as deep learning, machine learning, compressive sensing, and variational Bayesian. All these devices have one point in common: These algorithms have incorporated some form of computational intelligence as part of their core framework in problem solving. They have the capacity to generalize and discover knowledge for themselves, learning to learn new information whenever unseen data are captured

Multivariate Spectral Estimation based on the concept of Optimal Prediction

In this technical note, we deal with a spectrum approximation problem arising in THREE-like multivariate spectral estimation approaches. The solution to the problem minimizes a suitable divergence index with respect to an a priori spectral density. We derive a new divergence family between multivariate spectral densities which takes root in the prediction theory. Under mild assumptions on the a priori spectral density, the approximation problem, based on this new divergence family, admits a family of solutions. Moreover, an upper bound on the complexity degree of these solutions is provided

Electroencephalograph (EEG) signal processing techniques for motor imagery Brain Computer interface systems

Brain-Computer Interface (BCI) system provides a channel for the brain to control external devices using electrical activities of the brain without using the peripheral nervous system. These BCI systems are being used in various medical applications, for example controlling a wheelchair and neuroprosthesis devices for the disabled, thereby assisting them in activities of daily living. People suffering from Amyotrophic Lateral Sclerosis (ALS), Multiple Sclerosis and completely locked in are unable to perform any body movements because of the damage of the peripheral nervous system, but their cognitive function is still intact. BCIs operate external devices by acquiring brain signals and converting them to control commands to operate external devices. Motor-imagery (MI) based BCI systems, in particular, are based on the sensory-motor rhythms which are generated by the imagination of body limbs. These signals can be decoded as control commands in BCI application. Electroencephalogram (EEG) is commonly used for BCI applications because it is non-invasive. The main challenges of decoding the EEG signal are because it is non-stationary and has a low spatial resolution. The common spatial pattern algorithm is considered to be the most effective technique for discrimination of spatial filter but is easily affected by the presence of outliers. Therefore, a robust algorithm is required for extraction of discriminative features from the motor imagery EEG signals. This thesis mainly aims in developing robust spatial filtering criteria which are effective for classification of MI movements. We have proposed two approaches for the robust classification of MI movements. The first approach is for the classification of multiclass MI movements based on the thinICA (Independent Component Analysis) and mCSP (multiclass Common Spatial Pattern Filter) method. The observed results indicate that these approaches can be a step towards the development of robust feature extraction for MI-based BCI system. The main contribution of the thesis is the second criterion, which is based on Alpha- Beta logarithmic-determinant divergence for the classification of two class MI movements. A detailed study has been done by obtaining a link between the AB log det divergence and CSP criterion. We propose a scaling parameter to enable a similar way for selecting the respective filters like the CSP algorithm. Additionally, the optimization of the gradient of AB log-det divergence for this application was also performed. The Sub-ABLD (Subspace Alpha-Beta Log-Det divergence) algorithm is proposed for the discrimination of two class MI movements. The robustness of this algorithm is tested with both the simulated and real data from BCI competition dataset. Finally, the resulting performances of the proposed algorithms have been favorably compared with other existing algorithms

Sensor Control for Multi-Object Tracking Using Labeled Multi-Bernoulli Filter

The recently developed labeled multi-Bernoulli (LMB) filter uses better approximations in its update step, compared to the unlabeled multi-Bernoulli filters, and more importantly, it provides us with not only the estimates for the number of targets and their states, but also with labels for existing tracks. This paper presents a novel sensor-control method to be used for optimal multi-target tracking within the LMB filter. The proposed method uses a task-driven cost function in which both the state estimation errors and cardinality estimation errors are taken into consideration. Simulation results demonstrate that the proposed method can successfully guide a mobile sensor in a challenging multi-target tracking scenario

Information Theoretic Approaches for Motor-Imagery BCI Systems: Review and Experimental Comparison

Brain computer interfaces (BCIs) have been attracting a great interest in recent years. The common spatial patterns (CSP) technique is a well-established approach to the spatial filtering of the electroencephalogram (EEG) data in BCI applications. Even though CSP was originally proposed from a heuristic viewpoint, it can be also built on very strong foundations using information theory. This paper reviews the relationship between CSP and several information-theoretic approaches, including the Kullback–Leibler divergence, the Beta divergence and the Alpha-Beta log-det (AB-LD)divergence. We also revise other approaches based on the idea of selecting those features that are maximally informative about the class labels. The performance of all the methods will be also compared via experiments.Gobierno Español MICINN TEC2014-53103-