1,838 research outputs found
Deep speech inpainting of time-frequency masks
Transient loud intrusions, often occurring in noisy environments, can
completely overpower speech signal and lead to an inevitable loss of
information. While existing algorithms for noise suppression can yield
impressive results, their efficacy remains limited for very low signal-to-noise
ratios or when parts of the signal are missing. To address these limitations,
here we propose an end-to-end framework for speech inpainting, the
context-based retrieval of missing or severely distorted parts of
time-frequency representation of speech. The framework is based on a
convolutional U-Net trained via deep feature losses, obtained using speechVGG,
a deep speech feature extractor pre-trained on an auxiliary word classification
task. Our evaluation results demonstrate that the proposed framework can
recover large portions of missing or distorted time-frequency representation of
speech, up to 400 ms and 3.2 kHz in bandwidth. In particular, our approach
provided a substantial increase in STOI & PESQ objective metrics of the
initially corrupted speech samples. Notably, using deep feature losses to train
the framework led to the best results, as compared to conventional approaches.Comment: Accepted to InterSpeech202
Inpainting of long audio segments with similarity graphs
We present a novel method for the compensation of long duration data loss in
audio signals, in particular music. The concealment of such signal defects is
based on a graph that encodes signal structure in terms of time-persistent
spectral similarity. A suitable candidate segment for the substitution of the
lost content is proposed by an intuitive optimization scheme and smoothly
inserted into the gap, i.e. the lost or distorted signal region. Extensive
listening tests show that the proposed algorithm provides highly promising
results when applied to a variety of real-world music signals
Dealing with missing data: An inpainting application to the MICROSCOPE space mission
Missing data are a common problem in experimental and observational physics.
They can be caused by various sources, either an instrument's saturation, or a
contamination from an external event, or a data loss. In particular, they can
have a disastrous effect when one is seeking to characterize a
colored-noise-dominated signal in Fourier space, since they create a spectral
leakage that can artificially increase the noise. It is therefore important to
either take them into account or to correct for them prior to e.g. a
Least-Square fit of the signal to be characterized. In this paper, we present
an application of the {\it inpainting} algorithm to mock MICROSCOPE data; {\it
inpainting} is based on a sparsity assumption, and has already been used in
various astrophysical contexts; MICROSCOPE is a French Space Agency mission,
whose launch is expected in 2016, that aims to test the Weak Equivalence
Principle down to the level. We then explore the {\it inpainting}
dependence on the number of gaps and the total fraction of missing values. We
show that, in a worst-case scenario, after reconstructing missing values with
{\it inpainting}, a Least-Square fit may allow us to significantly measure a
Equivalence Principle violation signal, which is
sufficiently close to the MICROSCOPE requirements to implement {\it inpainting}
in the official MICROSCOPE data processing and analysis pipeline. Together with
the previously published KARMA method, {\it inpainting} will then allow us to
independently characterize and cross-check an Equivalence Principle violation
signal detection down to the level.Comment: Accepted for publication in Physical Review D. 12 pages, 6 figure
Audio Inpainting
(c) 2012 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Published version: IEEE Transactions on Audio, Speech and Language Processing 20(3): 922-932, Mar 2012. DOI: 10.1090/TASL.2011.2168211
Highly corrupted image inpainting through hypoelliptic diffusion
We present a new image inpainting algorithm, the Averaging and Hypoelliptic
Evolution (AHE) algorithm, inspired by the one presented in [SIAM J. Imaging
Sci., vol. 7, no. 2, pp. 669--695, 2014] and based upon a semi-discrete
variation of the Citti-Petitot-Sarti model of the primary visual cortex V1. The
AHE algorithm is based on a suitable combination of sub-Riemannian hypoelliptic
diffusion and ad-hoc local averaging techniques. In particular, we focus on
reconstructing highly corrupted images (i.e. where more than the 80% of the
image is missing), for which we obtain reconstructions comparable with the
state-of-the-art.Comment: 15 pages, 10 figure
A texture based approach to reconstruction of archaeological finds
Reconstruction of archaeological finds from fragments, is a tedious task requiring many hours of work from the archaeologists and restoration personnel. In this paper we present a framework for the full reconstruction of the original objects using texture and surface design information on the sherd. The texture of a band outside the border of pieces is predicted by inpainting and texture synthesis methods. The confidence of this process is also defined. Feature values are derived from these original and predicted images of pieces. A combination of the feature and confidence values is used to generate an affinity measure of corresponding pieces. The optimization of total affinity gives the best assembly of the piece. Experimental results are presented on real and artificial data
Analysis of Inpainting via Clustered Sparsity and Microlocal Analysis
Recently, compressed sensing techniques in combination with both wavelet and
directional representation systems have been very effectively applied to the
problem of image inpainting. However, a mathematical analysis of these
techniques which reveals the underlying geometrical content is completely
missing. In this paper, we provide the first comprehensive analysis in the
continuum domain utilizing the novel concept of clustered sparsity, which
besides leading to asymptotic error bounds also makes the superior behavior of
directional representation systems over wavelets precise. First, we propose an
abstract model for problems of data recovery and derive error bounds for two
different recovery schemes, namely l_1 minimization and thresholding. Second,
we set up a particular microlocal model for an image governed by edges inspired
by seismic data as well as a particular mask to model the missing data, namely
a linear singularity masked by a horizontal strip. Applying the abstract
estimate in the case of wavelets and of shearlets we prove that -- provided the
size of the missing part is asymptotically to the size of the analyzing
functions -- asymptotically precise inpainting can be obtained for this model.
Finally, we show that shearlets can fill strictly larger gaps than wavelets in
this model.Comment: 49 pages, 9 Figure
- âŠ