Learning how to be robust: Deep polynomial regression

Polynomial regression is a recurrent problem with a large number of applications. In computer vision it often appears in motion analysis. Whatever the application, standard methods for regression of polynomial models tend to deliver biased results when the input data is heavily contaminated by outliers. Moreover, the problem is even harder when outliers have strong structure. Departing from problem-tailored heuristics for robust estimation of parametric models, we explore deep convolutional neural networks. Our work aims to find a generic approach for training deep regression models without the explicit need of supervised annotation. We bypass the need for a tailored loss function on the regression parameters by attaching to our model a differentiable hard-wired decoder corresponding to the polynomial operation at hand. We demonstrate the value of our findings by comparing with standard robust regression methods. Furthermore, we demonstrate how to use such models for a real computer vision problem, i.e., video stabilization. The qualitative and quantitative experiments show that neural networks are able to learn robustness for general polynomial regression, with results that well overpass scores of traditional robust estimation methods.Comment: 18 pages, conferenc

Image enhancement methods and applications in computational photography

Computational photography is currently a rapidly developing and cutting-edge topic in applied optics, image sensors and image processing fields to go beyond the limitations of traditional photography. The innovations of computational photography allow the photographer not only merely to take an image, but also, more importantly, to perform computations on the captured image data. Good examples of these innovations include high dynamic range imaging, focus stacking, super-resolution, motion deblurring and so on. Although extensive work has been done to explore image enhancement techniques in each subfield of computational photography, attention has seldom been given to study of the image enhancement technique of simultaneously extending depth of field and dynamic range of a scene. In my dissertation, I present an algorithm which combines focus stacking and high dynamic range (HDR) imaging in order to produce an image with both extended depth of field (DOF) and dynamic range than any of the input images. In this dissertation, I also investigate super-resolution image restoration from multiple images, which are possibly degraded by large motion blur. The proposed algorithm combines the super-resolution problem and blind image deblurring problem in a unified framework. The blur kernel for each input image is separately estimated. I also do not make any restrictions on the motion fields among images; that is, I estimate dense motion field without simplifications such as parametric motion. While the proposed super-resolution method uses multiple images to enhance spatial resolution from multiple regular images, single image super-resolution is related to techniques of denoising or removing blur from one single captured image. In my dissertation, space-varying point spread function (PSF) estimation and image deblurring for single image is also investigated. Regarding the PSF estimation, I do not make any restrictions on the type of blur or how the blur varies spatially. Once the space-varying PSF is estimated, space-varying image deblurring is performed, which produces good results even for regions where it is not clear what the correct PSF is at first. I also bring image enhancement applications to both personal computer (PC) and Android platform as computational photography applications

TFAW: wavelet-based signal reconstruction to reduce photometric noise in time-domain surveys

There have been many efforts to correct systematic effects in astronomical light curves to improve the detection and characterization of planetary transits and astrophysical variability. Algorithms like the Trend Filtering Algorithm (TFA) use simultaneously-observed stars to remove systematic effects, and binning is used to reduce high-frequency random noise. We present TFAW, a wavelet-based modified version of TFA. TFAW aims to increase the periodic signal detection and to return a detrended and denoised signal without modifying its intrinsic characteristics. We modify TFA's frequency analysis step adding a Stationary Wavelet Transform filter to perform an initial noise and outlier removal and increase the detection of variable signals. A wavelet filter is added to TFA's signal reconstruction to perform an adaptive characterization of the noise- and trend-free signal and the noise contribution at each iteration while preserving astrophysical signals. We carried out tests over simulated sinusoidal and transit-like signals to assess the effectiveness of the method and applied TFAW to real light curves from TFRM. We also studied TFAW's application to simulated multiperiodic signals, improving their characterization. TFAW improves the signal detection rate by increasing the signal detection efficiency (SDE) up to a factor ~2.5x for low SNR light curves. For simulated transits, the transit detection rate improves by a factor ~2-5x in the low-SNR regime compared to TFA. TFAW signal approximation performs up to a factor ~2x better than bin averaging for planetary transits. The standard deviations of simulated and real TFAW light curves are ~40x better than TFA. TFAW yields better MCMC posterior distributions and returns lower uncertainties, less biased transit parameters and narrower (~10x) credibility intervals for simulated transits. We present a newly-discovered variable star from TFRM.Comment: Accepted for publication by A&A. 13 pages, 16 figures and 5 table

A new adaptive algorithm for video super-resolution with improved outlier handling capability

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia Elétrica, Florianópolis, 2016.Abstract : Super resolution reconstruction (SRR) is a technique that consists basically in combining multiple low resolution images from a single scene in order to create an image with higher resolution. The main characteristics considered in the evaluation of SRR algorithms performance are the resulting image quality, its robustness to outliers and its computational cost. Among the super resolution algorithms present in the literature, the R-LMS has a very small computational cost, making it suitable for real-time operation. However, like many SRR techniques the R-LMS algorithm is also highly susceptible to outliers, which can lead the reconstructed image quality to be of lower quality than the low resolution observations. Although robust techniques have been proposed to mitigate this problem, the computational cost associated with even the simpler algorithms is not comparable to that of the R-LMS, making real-time operation impractical. It is therefore desirable to devise new algorithms that offer a better compromise between quality, robustness and computational cost. In this work, a new SRR technique based on the R-LMS algorithm is proposed. Based on the proximal-point cost function representation of the gradient descent iterative equation, an intuitive interpretation of the R-LMS algorithm behavior is obtained, both in ideal conditions and in the presence of outliers. Using a statistical model for the innovation outliers, a new regularization is then proposed to increase the algorithm robustness by allowing faster convergence on the subspace corresponding to the innovations while at the same time preserving the estimated image details. Two new algorithms are then derived. Computer simulations have shown that the new algorithms deliver a performance comparable to that of the R-LMS in the absence of outliers, and a significantly better performance in the presence of outliers, both quantitatively and visually. The computational cost of the proposed solution remained comparable to that of the R-LMS.Reconstrução com super resolução (SRR - Super resolution reconstruction) é uma técnica que consiste basicamente em combinar múltiplas imagens de baixa resolução a fim de formar uma única imagem com resolução superior. As principais características consideradas na avaliação de algoritmos de SRR são a qualidade da imagem reconstruída, sua robustez a outliers e o custo computacional associado. Uma maior qualidade nas imagens reconstruídas implica em um maior aumento efetivo na resolução das mesmas. Uma maior robustez, por outro lado, implica que um resultado de boa qualidade é obtido mesmo quando as imagens processadas não seguem fielmente o modelo matemático adotado. O custo computacional, por sua vez, é extremamente relevante em aplicações de SRR, dado que a dimensão do problema é extremamente grande. Uma das principais aplicações da SRR consiste na reconstrução de sequências de vídeo. De modo a facilitar o processamento em tempo real, o qual é um requisito frequente para aplicações de SRR de vídeo, algorítmos iterativos foram propostos, os quais processam apenas uma imagem a cada instante de tempo, utilizando informações presentes nas estimativas obtidas em instantes de tempo anteriores. Dentre os algoritmos de super resolução iterativos presentes na literatura, o R-LMS possui um custo computacional extremamente baixo, além de fornecer uma reconstrução com qualidade competitiva. Apesar disso, assim como grande parte das técnicas de SRR existentes o R-LMS é bastante suscetível a presença de outliers, os quais podem tornar a qualidade das imagens reconstruídas inferior àquela das observações de baixa resolução. A fim de mitigar esse problema, técnicas de SRR robusta foram propostas na literatura. Não obstante, mesmo o custo computacional dos algoritmos robustos mais simples não é comparável àquele do R-LMS, tornando o processamento em tempo real infactível. Deseja-se portanto desenvolver novos algoritmos que ofereçam um melhor compromisso entre qualidade, robustez e custo computacional. Neste trabalho uma nova técnica de SRR baseada no algoritmo R-LMS é proposta. Com base na representação da função custo do ponto proximal para a equação iterativa do método do gradiente, uma interpretação intuitiva para o comportamento do algoritmo R-LMS é obtida tanto para sua operação em condições ideais quanto na presença de outliers do tipo inovação, os quais representam variações significativas na cena entre frames adjacentes de uma sequência de vídeo. É demonstrado que o problema apresentado pelo R-LMS quanto a robustez à outliers de inovação se deve, principalmente, a sua baixa taxa de convergência. Além disso, um balanço direto pôde ser observado entre a rapidez da taxa de convergência e a preservação das informações estimadas em instantes de tempo anteriores. Desse modo, torna-se inviável obter, simultaneamente, uma boa qualidade no processamento de sequências bem comportadas e uma boa robustez na presença de inovações de grande porte. Desse modo, tem-se como objetivo projetar um algoritmo voltado à reconstrução de sequências de vídeo em tempo real que apresente uma maior robustez à outliers de grande porte, sem comprometer a preservação da informação estimada a partir da sequência de baixa resolução. Utilizando um modelo estatístico para os outliers provindos de inovações, uma nova regularização é proposta a fim de aumentar a robustez do algoritmo, permitindo simultaneamente uma convergência mais rápida no subespaço da imagem correspondente às inovações e a preservação dos detalhes previamente estimados. A partir disso dois novos algoritmos são então derivados. A nova regularização proposta penaliza variações entre estimativas adjacentes na sequência de vídeo em um subespaço aproximadamente ortogonal ao conteúdo das inovações. Verificou-se que o subespaço da imagem no qual a inovação contém menos energia é precisamente onde estão contidos os detalhes da imagem. Isso mostra que a regularização proposta, além de levar a uma maior robustez, também implica na preservação dos detalhes estimados na sequência de vídeo em instantes de tempo anteriores. Simulações computacionais mostram que apesar da solução proposta não levar a melhorias significativas no desempenho do algoritmo sob condições próximas às ideais, quando outliers estão presentes na sequência de imagens o método proposto superou consideravelmente o desempenho apresentado pelo R-LMS, tanto quantitativamente quanto visualmente. O custo computacional da solução proposta manteve-se comparável àquele do algoritmo R-LMS

Fast diffusion MRI based on sparse acquisition and reconstruction for long-term population imaging

Diffusion weighted magnetic resonance imaging (dMRI) is a unique MRI modality to probe the diffusive molecular transport in biological tissue. Due to its noninvasiveness and its ability to investigate the living human brain at submillimeter scale, dMRI is frequently performed in clinical and biomedical research to study the brain’s complex microstructural architecture. Over the last decades large prospective cohort studies have been set up with the aim to gain new insights into the development and progression of brain diseases across the life span and to discover biomarkers for disease prediction and potentially prevention. To allow for diverse brain imaging using different MRI modalities, stringent scan time limits are typically imposed in population imaging. Nevertheless, population studies aim to apply advanced and thereby time consuming dMRI protocols that deliver high quality data with great potential for future analysis. To allow for time-efficient but also versatile diffusion imaging, this thesis contributes to the investigation of accelerating diffusion spectrum imaging (DSI), an advanced dMRI technique that acquires imaging data with high intra-voxel resolution of tissue microstructure. Combining state-of-the-art parallel imaging and the theory of compressed sensing (CS) enables the acceleration of spatial encoding and diffusion encoding in dMRI. In this way, the otherwise long acquisition times in DSI can be reduced significantly. In this thesis, first, suitable q-space sampling strategies and basis functions are explored that fulfill the requirements of CS theory for accurate sparse DSI reconstruction. Novel 3D q-space sample distributions are investigated for CS-DSI. Moreover, conventional CS-DSI based on the discrete Fourier transform is compared for the first time to CS-DSI based on the continuous SHORE (simple harmonic oscillator based reconstruction and estimation) basis functions. Based on these findings, a CS-DSI protocol is proposed for application in a prospective cohort study, the Rhineland Study. A pilot study was designed and conducted to evaluate the CS-DSI protocol in comparison with state-of-the-art 3-shell dMRI and dedicated protocols for diffusion tensor imaging (DTI) and for the combined hindered and restricted model of diffusion (CHARMED). Population imaging requires processing techniques preferably with low computational cost to process and analyze the acquired big data within a reasonable time frame. Therefore, a pipeline for automated processing of CS-DSI acquisitions was implemented including both in-house developed and existing state-of-the-art processing tools. The last contribution of this thesis is a novel method for automatic detection and imputation of signal dropout due to fast bulk motion during the diffusion encoding in dMRI. Subject motion is a common source of artifacts, especially when conducting clinical or population studies with children, the elderly or patients. Related artifacts degrade image quality and adversely affect data analysis. It is, thus, highly desired to detect and then exclude or potentially impute defective measurements prior to dMRI analysis. Our proposed method applies dMRI signal modeling in the SHORE basis and determines outliers based on the weighted model residuals. Signal imputation reconstructs corrupted and therefore discarded measurements from the sparse set of inliers. This approach allows for fast and robust correction of imaging artifacts in dMRI which is essential to estimate accurate and precise model parameters that reflect the diffusive transport of water molecules and the underlying microstructural environment in brain tissue.Die diffusionsgewichtete Magnetresonanztomographie (dMRT) ist ein einzigartiges MRTBildgebungsverfahren, um die Diffusionsbewegung von Wassermolekülen in biologischem Gewebe zu messen. Aufgrund der Möglichkeit Schichtbilder nicht invasiv aufzunehmen und das lebende menschliche Gehirn im Submillimeter-Bereich zu untersuchen, ist die dMRT ein häufig verwendetes Bildgebungsverfahren in klinischen und biomedizinischen Studien zur Erforschung der komplexen mikrostrukturellen Architektur des Gehirns. In den letzten Jahrzehnten wurden große prospektive Kohortenstudien angelegt, um neue Einblicke in die Entwicklung und den Verlauf von Gehirnkrankheiten über die Lebenspanne zu erhalten und um Biomarker zur Krankheitserkennung und -vorbeugung zu bestimmen. Um durch die Verwendung unterschiedlicher MRT-Verfahren verschiedenartige Schichtbildaufnahmen des Gehirns zu ermöglich, müssen Scanzeiten typischerweise stark begrenzt werden. Dennoch streben Populationsstudien die Anwendung von fortschrittlichen und daher zeitintensiven dMRT-Protokollen an, um Bilddaten in hoher Qualität und mit großem Potential für zukünftige Analysen zu akquirieren. Um eine zeiteffizente und gleichzeitig vielseitige Diffusionsbildgebung zu ermöglichen, leistet diese Dissertation Beiträge zur Untersuchung von Beschleunigungsverfahren für die Bildgebung mittels diffusion spectrum imaging (DSI). DSI ist ein fortschrittliches dMRT-Verfahren, das Bilddaten mit hoher intra-voxel Auflösung der Gewebestruktur erhebt. Werden modernste Verfahren zur parallelen MRT-Bildgebung mit der compressed sensing (CS) Theorie kombiniert, ermöglicht dies eine Beschleunigung der räumliche Kodierung und der Diffusionskodierung in der dMRT. Dadurch können die ansonsten langen Aufnahmezeiten für DSI erheblich reduziert werden. In dieser Arbeit werden zuerst geeigenete Strategien zur Abtastung des q-space sowie Basisfunktionen untersucht, welche die Anforderungen der CS-Theorie für eine korrekte Signalrekonstruktion der dünnbesetzten DSI-Daten erfüllen. Neue 3D-Verteilungen von Messpunkten im q-space werden für die Verwendung in CS-DSI untersucht. Außerdem wird konventionell auf der diskreten Fourier-Transformation basierendes CS-DSI zum ersten Mal mit einem CS-DSI Verfahren verglichen, welches kontinuierliche SHORE (simple harmonic oscillator based reconstruction and estimation) Basisfunktionen verwendet. Aufbauend auf diesen Ergebnissen wird ein CS-DSI-Protokoll zur Anwendung in einer prospektiven Kohortenstudie, der Rheinland Studie, vorgestellt. Eine Pilotstudie wurde entworfen und durchgeführt, um das CS-DSI-Protokoll im Vergleich mit modernster 3-shell-dMRT und mit dedizierten Protokollen für diffusion tensor imaging (DTI) und für das combined hindered and restricted model of diffusion (CHARMED) zu evaluieren. Populationsbildgebung erfordert Prozessierungsverfahren mit möglichst geringem Rechenaufwand, um große akquirierte Datenmengen in einem angemessenen Zeitrahmen zu verarbeiten und zu analysieren. Dafür wurde eine Pipeline zur automatisierten Verarbeitung von CS-DSI-Daten implementiert, welche sowohl eigenentwickelte als auch bereits existierende moderene Verarbeitungsprogramme enthält. Der letzte Beitrag dieser Arbeit ist eine neue Methode zur automatischen Detektion und Imputation von Signalabfall, welcher durch schnelle Bewegungen während der Diffusionskodierung in der dMRT entsteht. Bewegungen der Probanden während der dMRT-Aufnahme sind eine häufige Ursache für Bildfehler, vor allem in klinischen oder Populationsstudien mit Kindern, alten Menschen oder Patienten. Diese Artefakte vermindern die Datenqualität und haben einen negativen Einfluss auf die Datenanalyse. Daher ist es das Ziel, fehlerhafte Messungen vor der dMRI-Analyse zu erkennen und dann auszuschließen oder wenn möglich zu ersetzen. Die vorgestellte Methode verwendet die SHORE-Basis zur dMRT-Signalmodellierung und bestimmt Ausreißer mit Hilfe von gewichteten Modellresidualen. Die Datenimputation rekonstruiert die unbrauchbaren und daher verworfenen Messungen mit Hilfe der verbleibenden, dünnbesetzten Menge an Messungen. Dieser Ansatz ermöglicht eine schnelle und robuste Korrektur von Bildartefakten in der dMRT, welche erforderlich ist, um korrekte und präzise Modellparameter zu schätzen, die die Diffusionsbewegung von Wassermolekülen und die zugrundeliegende Mikrostruktur des Gehirngewebes reflektieren