Audio Inpainting

(c) 2012 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Published version: IEEE Transactions on Audio, Speech and Language Processing 20(3): 922-932, Mar 2012. DOI: 10.1090/TASL.2011.2168211

IST Austria Thesis

Modern computer vision systems heavily rely on statistical machine learning models, which typically require large amounts of labeled data to be learned reliably. Moreover, very recently computer vision research widely adopted techniques for representation learning, which further increase the demand for labeled data. However, for many important practical problems there is relatively small amount of labeled data available, so it is problematic to leverage full potential of the representation learning methods. One way to overcome this obstacle is to invest substantial resources into producing large labelled datasets. Unfortunately, this can be prohibitively expensive in practice. In this thesis we focus on the alternative way of tackling the aforementioned issue. We concentrate on methods, which make use of weakly-labeled or even unlabeled data. Specifically, the first half of the thesis is dedicated to the semantic image segmentation task. We develop a technique, which achieves competitive segmentation performance and only requires annotations in a form of global image-level labels instead of dense segmentation masks. Subsequently, we present a new methodology, which further improves segmentation performance by leveraging tiny additional feedback from a human annotator. By using our methods practitioners can greatly reduce the amount of data annotation effort, which is required to learn modern image segmentation models. In the second half of the thesis we focus on methods for learning from unlabeled visual data. We study a family of autoregressive models for modeling structure of natural images and discuss potential applications of these models. Moreover, we conduct in-depth study of one of these applications, where we develop the state-of-the-art model for the probabilistic image colorization task

Bayesian Demographic Modeling and Forecasting: An Application to U.S. Mortality

We present a new way to model age-specific demographic variables with the example of age-specific mortality in the U.S., building on the Lee-Carter approach and extending it in several dimensions. We incorporate covariates and model their dynamics jointly with the latent variables underlying mortality of all age classes. In contrast to previous models, a similar development of adjacent age groups is assured allowing for consistent forecasts. We develop an appropriate Markov Chain Monte Carlo algorithm to estimate the parameters and the latent variables in an efficient one-step procedure. Via the Bayesian approach we are able to asses uncertainty intuitively by constructing error bands for the forecasts. We observe that in particular parameter uncertainty is important for long-run forecasts. This implies that hitherto existing forecasting methods, which ignore certain sources of uncertainty, may yield misleadingly sure predictions. To test the forecast ability of our model we perform in-sample and out-of-sample forecasts up to 2050, revealing that covariates can help to improve the forecasts for particular age classes. A structural analysis of the relationship between age-specific mortality and covariates is conducted in a companion paper.Demography, Age-specific, Mortality, Lee-Carter, Stochastic, Bayesian, State Space Models, Forecasts

Content-based image retrieval of museum images

Content-based image retrieval (CBIR) is becoming more and more important with the advance of multimedia and imaging technology. Among many retrieval features associated with CBIR, texture retrieval is one of the most difficult. This is mainly because no satisfactory quantitative definition of texture exists at this time, and also because of the complex nature of the texture itself. Another difficult problem in CBIR is query by low-quality images, which means attempts to retrieve images using a poor quality image as a query. Not many content-based retrieval systems have addressed the problem of query by low-quality images. Wavelet analysis is a relatively new and promising tool for signal and image analysis. Its time-scale representation provides both spatial and frequency information, thus giving extra information compared to other image representation schemes. This research aims to address some of the problems of query by texture and query by low quality images by exploiting all the advantages that wavelet analysis has to offer, particularly in the context of museum image collections. A novel query by low-quality images algorithm is presented as a solution to the problem of poor retrieval performance using conventional methods. In the query by texture problem, this thesis provides a comprehensive evaluation on wavelet-based texture method as well as comparison with other techniques. A novel automatic texture segmentation algorithm and an improved block oriented decomposition is proposed for use in query by texture. Finally all the proposed techniques are integrated in a content-based image retrieval application for museum image collections

Digital Color Imaging

This paper surveys current technology and research in the area of digital color imaging. In order to establish the background and lay down terminology, fundamental concepts of color perception and measurement are first presented us-ing vector-space notation and terminology. Present-day color recording and reproduction systems are reviewed along with the common mathematical models used for representing these devices. Algorithms for processing color images for display and communication are surveyed, and a forecast of research trends is attempted. An extensive bibliography is provided

Multiresolution image modelling and estimation

Multiresolution representations make explicit the notion of scale in images, and facilitate the combination of information from different scales. To date, however, image modelling and estimation schemes have not exploited such representations and tend rather to be derived from two- dimensional extensions of traditional one-dimensional signal processing techniques. In the causal case, autoregressive (AR) and ARMA models lead to minimum mean square error (MMSE) estimators which are two-dimensional variants of the well-established Kalman filter. Noncausal approaches tend to be transform-based and the MMSE estimator is the two- dimensional Wiener filter. However, images contain profound nonstationarities such as edges, which are beyond the descriptive capacity of such signal models, and defects such as blurring (and streaking in the causal case) are apparent in the results obtained by the associated estimators. This thesis introduces a new multiresolution image model, defined on the quadtree data structure. The model is a one-dimensional, first-order gaussian martingale process causal in the scale dimension. The generated image, however, is noncausal and exhibits correlations at all scales unlike those generated by traditional models. The model is capable of nonstationary behaviour in all three dimensions (two position and one scale) and behaves isomorphically but independently at each scale, in keeping with the notion of scale invariance in natural images. The optimal (MMSE) estimator is derived for the case of corruption by additive white gaussian noise (AWGN). The estimator is a one-dimensional, first-order linear recursive filter with a computational burden far lower than that of traditional estimators. However, the simple quadtree data structure leads to aliasing and 'block' artifacts in the estimated images. This could be overcome by spatial filtering, but a faster method is introduced which requires no additional multiplications but involves the insertion of some extra nodes into the quadtree. Nonstationarity is introduced by a fast, scale-invariant activity detector defined on the quadtree. Activity at all scales is combined in order to achieve noise rejection. The estimator is modified at each scale and position by the detector output such that less smoothing is applied near edges and more in smooth regions. Results demonstrate performance superior to that of existing methods, and at drastically lower computational cost. The estimation scheme is further extended to include anisotropic processing, which has produced good results in image restoration. An orientation estimator controls anisotropic filtering, the output of which is made available to the image estimator

Data comparison schemes for Pattern Recognition in Digital Images using Fractals

Pattern recognition in digital images is a common problem with application in remote sensing, electron microscopy, medical imaging, seismic imaging and astrophysics for example. Although this subject has been researched for over twenty years there is still no general solution which can be compared with the human cognitive system in which a pattern can be recognised subject to arbitrary orientation and scale. The application of Artificial Neural Networks can in principle provide a very general solution providing suitable training schemes are implemented. However, this approach raises some major issues in practice. First, the CPU time required to train an ANN for a grey level or colour image can be very large especially if the object has a complex structure with no clear geometrical features such as those that arise in remote sensing applications. Secondly, both the core and file space memory required to represent large images and their associated data tasks leads to a number of problems in which the use of virtual memory is paramount. The primary goal of this research has been to assess methods of image data compression for pattern recognition using a range of different compression methods. In particular, this research has resulted in the design and implementation of a new algorithm for general pattern recognition based on the use of fractal image compression. This approach has for the first time allowed the pattern recognition problem to be solved in a way that is invariant of rotation and scale. It allows both ANNs and correlation to be used subject to appropriate pre-and post-processing techniques for digital image processing on aspect for which a dedicated programmer's work bench has been developed using X-Designer