171 research outputs found
Scale Invariant Interest Points with Shearlets
Shearlets are a relatively new directional multi-scale framework for signal
analysis, which have been shown effective to enhance signal discontinuities
such as edges and corners at multiple scales. In this work we address the
problem of detecting and describing blob-like features in the shearlets
framework. We derive a measure which is very effective for blob detection and
closely related to the Laplacian of Gaussian. We demonstrate the measure
satisfies the perfect scale invariance property in the continuous case. In the
discrete setting, we derive algorithms for blob detection and keypoint
description. Finally, we provide qualitative justifications of our findings as
well as a quantitative evaluation on benchmark data. We also report an
experimental evidence that our method is very suitable to deal with compressed
and noisy images, thanks to the sparsity property of shearlets
ShearLab 3D: Faithful Digital Shearlet Transforms based on Compactly Supported Shearlets
Wavelets and their associated transforms are highly efficient when
approximating and analyzing one-dimensional signals. However, multivariate
signals such as images or videos typically exhibit curvilinear singularities,
which wavelets are provably deficient of sparsely approximating and also of
analyzing in the sense of, for instance, detecting their direction. Shearlets
are a directional representation system extending the wavelet framework, which
overcomes those deficiencies. Similar to wavelets, shearlets allow a faithful
implementation and fast associated transforms. In this paper, we will introduce
a comprehensive carefully documented software package coined ShearLab 3D
(www.ShearLab.org) and discuss its algorithmic details. This package provides
MATLAB code for a novel faithful algorithmic realization of the 2D and 3D
shearlet transform (and their inverses) associated with compactly supported
universal shearlet systems incorporating the option of using CUDA. We will
present extensive numerical experiments in 2D and 3D concerning denoising,
inpainting, and feature extraction, comparing the performance of ShearLab 3D
with similar transform-based algorithms such as curvelets, contourlets, or
surfacelets. In the spirit of reproducible reseaerch, all scripts are
accessible on www.ShearLab.org.Comment: There is another shearlet software package
(http://www.mathematik.uni-kl.de/imagepro/members/haeuser/ffst/) by S.
H\"auser and G. Steidl. We will include this in a revisio
Detecting spatio-temporally interest points using the shearlet transform
In this paper we address the problem of detecting spatio-temporal interest points in video sequences and we introduce a novel detection algorithm based on the three-dimensional shearlet transform. By evaluating our method on different application scenarios, we show we are able to extract meaningful spatio-temporal features from video sequences of human movements, including full body movements selected from benchmark datasets of human actions and human-machine interaction sequences where the goal is to segment drawing activities in smaller action primitives
Space-Time Signal Analysis and the 3D Shearlet Transform
In this work, we address the problem of analyzing video sequences by representing meaningful local space\ue2\u80\u93time neighborhoods. We propose a mathematical model to describe relevant points as local singularities of a 3D signal, and we show that these local patterns can be nicely highlighted by the 3D shearlet transform, which is at the root of our work. Based on this mathematical framework, we derive an algorithm to represent space\ue2\u80\u93time points which is very effective in analyzing video sequences. In particular, we show how points of the same nature have a very similar representation, allowing us to compute different space\ue2\u80\u93time primitives for a video sequence in an unsupervised way
Local Spatio-Temporal Representation Using the 3D Shearlet Transform (STSIP)
In this work we address the problem of analyzing video sequences and of representing meaningful space-time points of interest by using the 3D shearlet transform. We introduce a local representation based on shearlet coe cients of the video, regarded as 2D+T signal. This representation turns out to be informative to understand the local spatio-temporal characteristics, which can be easily detected by an unsupervised clustering algorithm
Spatio-Temporal Video Analysis and the 3D Shearlet Transform
Abstract
The automatic analysis of the content of a video sequence has captured the
attention of the computer vision community for a very long time. Indeed,
video understanding, which needs to incorporate both semantic and dynamic
cues, may be trivial for humans, but it turned out to be a very complex
task for a machine. Over the years the signal processing, computer vision,
and machine learning communities contributed with algorithms that are
today effective building blocks of more and more complex systems. In
the meanwhile, theoretical analysis has gained a better understanding of
this multifaceted type of data. Indeed, video sequences are not only high
dimensional data, but they are also very peculiar, as they include spatial as
well as temporal information which should be treated differently, but are
both important to the overall process. The work of this thesis builds a new
bridge between signal processing theory, and computer vision applications. It
considers a novel approach to multi resolution signal processing, the so-called
Shearlet Transform, as a reference framework for representing meaningful
space-time local information in a video signal. The Shearlet Transform
has been shown effective in analyzing multi-dimensional signals, ranging
from images to x-ray tomographic data. As a tool for signal denoising, has
also been applied to video data. However, to the best of our knowledge,
the Shearlet Transform has never been employed to design video analysis
algorithms. In this thesis, our broad objective is to explore the capabilities of
the Shearlet Transform to extract information from 2D+T-dimensional data.
We exploit the properties of the Shearlet decomposition to redesign a variety
of classical video processing techniques (including space-time interest point
detection and normal flow estimation) and to develop novel methods to better
understand the local behavior of video sequences. We provide experimental
evidence on the potential of our approach on synthetic as well as real data
drawn from publicly available benchmark datasets. The results we obtain
show the potential of our approach and encourages further investigations in
the near future
- …