20,444 research outputs found
Globally-Coordinated Locally-Linear Modeling of Multi-Dimensional Data
This thesis considers the problem of modeling and analysis of continuous, locally-linear, multi-dimensional spatio-temporal data. Our work extends the previously reported theoretical work on the global coordination model to temporal analysis of continuous, multi-dimensional data. We have developed algorithms for time-varying data analysis and used them in full-scale, real-world applications. The applications demonstrated in this thesis include tracking, synthesis, recognitions and retrieval of dynamic objects based on their shape, appearance and motion. The proposed approach in this thesis has advantages over existing approaches to analyzing complex spatio-temporal data. Experiments show that the new modeling features of our approach improve the performance of existing approaches in many applications. In object tracking, our approach is the first one to track nonlinear appearance variations by using low-dimensional representation of the appearance change in globally-coordinated linear subspaces. In dynamic texture synthesis, we are able to model non-stationary dynamic textures, which cannot be handled by any of the existing approaches. In human motion synthesis, we show that realistic synthesis can be performed without using specific transition points, or key frames
Biologically Inspired Dynamic Textures for Probing Motion Perception
Perception is often described as a predictive process based on an optimal
inference with respect to a generative model. We study here the principled
construction of a generative model specifically crafted to probe motion
perception. In that context, we first provide an axiomatic, biologically-driven
derivation of the model. This model synthesizes random dynamic textures which
are defined by stationary Gaussian distributions obtained by the random
aggregation of warped patterns. Importantly, we show that this model can
equivalently be described as a stochastic partial differential equation. Using
this characterization of motion in images, it allows us to recast motion-energy
models into a principled Bayesian inference framework. Finally, we apply these
textures in order to psychophysically probe speed perception in humans. In this
framework, while the likelihood is derived from the generative model, the prior
is estimated from the observed results and accounts for the perceptual bias in
a principled fashion.Comment: Twenty-ninth Annual Conference on Neural Information Processing
Systems (NIPS), Dec 2015, Montreal, Canad
Impressionistic techniques applied in sound art & design
Sound art and design collectively refer to the process of specifying, acquiring, manipulating or generating sonic elements to evoke emotion and environment. Sound is used to convey the intentions, emotions, spirit or aura of a story, performance, or sonic installation. Sound connects unique aural environments, creating an immersive experience via mood and atmosphere. Impressionistic techniques such as Impasto, Pointillism, Sgraffito, Stippling introduced by 19th-century painters captured the essence of their subject in more vivid compositions, exuding authentic movements and atmosphere. This thesis applied impressionistic techniques using sound art and design to project specific mood and atmosphere responses among listeners. Four unique sound textures, each representing a technique from Impressionism, and a fifth composite sound texture were created for this project. All five sound textures were validated as representative of their respective Impressionistic technique. Only sonic Pointillism matched its emotive intent. This outcome supports the research question that sound art and design can be used to direct listeners’ mood and atmosphere responses. Partnering Impressionistic principles with sound art and design offers a deeper palette to sonically deliver more robust, holistic soundscapes for amplifying an audience’s listening experience. This project provides a foundation for future explorations and studies in applying cross-disciplinary artistic techniques with sound art and design or other artistic endeavors
Convolutional Neural Network on Three Orthogonal Planes for Dynamic Texture Classification
Dynamic Textures (DTs) are sequences of images of moving scenes that exhibit
certain stationarity properties in time such as smoke, vegetation and fire. The
analysis of DT is important for recognition, segmentation, synthesis or
retrieval for a range of applications including surveillance, medical imaging
and remote sensing. Deep learning methods have shown impressive results and are
now the new state of the art for a wide range of computer vision tasks
including image and video recognition and segmentation. In particular,
Convolutional Neural Networks (CNNs) have recently proven to be well suited for
texture analysis with a design similar to a filter bank approach. In this
paper, we develop a new approach to DT analysis based on a CNN method applied
on three orthogonal planes x y , xt and y t . We train CNNs on spatial frames
and temporal slices extracted from the DT sequences and combine their outputs
to obtain a competitive DT classifier. Our results on a wide range of commonly
used DT classification benchmark datasets prove the robustness of our approach.
Significant improvement of the state of the art is shown on the larger
datasets.Comment: 19 pages, 10 figure
- …