2,179 research outputs found
The color of smiling: computational synaesthesia of facial expressions
This note gives a preliminary account of the transcoding or rechanneling
problem between different stimuli as it is of interest for the natural
interaction or affective computing fields. By the consideration of a simple
example, namely the color response of an affective lamp to a sensed facial
expression, we frame the problem within an information- theoretic perspective.
A full justification in terms of the Information Bottleneck principle promotes
a latent affective space, hitherto surmised as an appealing and intuitive
solution, as a suitable mediator between the different stimuli.Comment: Submitted to: 18th International Conference on Image Analysis and
Processing (ICIAP 2015), 7-11 September 2015, Genova, Ital
HeadOn: Real-time Reenactment of Human Portrait Videos
We propose HeadOn, the first real-time source-to-target reenactment approach
for complete human portrait videos that enables transfer of torso and head
motion, face expression, and eye gaze. Given a short RGB-D video of the target
actor, we automatically construct a personalized geometry proxy that embeds a
parametric head, eye, and kinematic torso model. A novel real-time reenactment
algorithm employs this proxy to photo-realistically map the captured motion
from the source actor to the target actor. On top of the coarse geometric
proxy, we propose a video-based rendering technique that composites the
modified target portrait video via view- and pose-dependent texturing, and
creates photo-realistic imagery of the target actor under novel torso and head
poses, facial expressions, and gaze directions. To this end, we propose a
robust tracking of the face and torso of the source actor. We extensively
evaluate our approach and show significant improvements in enabling much
greater flexibility in creating realistic reenacted output videos.Comment: Video: https://www.youtube.com/watch?v=7Dg49wv2c_g Presented at
Siggraph'1
Towards High-resolution Imaging from Underwater Vehicles
Large area mapping at high resolution underwater continues to be constrained by sensor-level environmental constraints and the mismatch between available navigation and sensor accuracy. In this paper, advances are presented that exploit aspects of the sensing modality, and consistency and redundancy within local sensor measurements to build high-resolution optical and acoustic maps that are a consistent representation of the environment. This work is presented in the context of real-world data acquired using autonomous underwater vehicles (AUVs) and remotely operated vehicles (ROVs) working in diverse applications including shallow water coral reef surveys with the Seabed AUV, a forensic survey of the RMS Titanic in the North Atlantic at a depth of 4100 m using the Hercules ROV, and a survey of the TAG hydrothermal vent area in the mid-Atlantic at a depth of 3600 m using the Jason II ROV. Specifically, the focus is on the related problems of structure from motion from underwater optical imagery assuming pose instrumented calibrated cameras. General wide baseline solutions are presented for these problems based on the extension of techniques from the simultaneous localization and mapping (SLAM), photogrammetric and the computer vision communities. It is also examined how such techniques can be extended for the very different sensing modality and scale associated with multi-beam bathymetric mapping. For both the optical and acoustic mapping cases it is also shown how the consistency in mapping can be used not only for better global mapping, but also to refine navigation estimates.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/86051/1/hsingh-21.pd
Spectral methods for multimodal data analysis
Spectral methods have proven themselves as an important and versatile tool in a wide range of problems in the fields of computer graphics, machine learning, pattern recognition, and computer vision, where many important problems boil down to constructing a Laplacian operator and finding a few of its eigenvalues and eigenfunctions. Classical examples include the computation of diffusion distances on manifolds in computer graphics, Laplacian eigenmaps, and spectral clustering in machine learning. In many cases, one has to deal with multiple data spaces simultaneously. For example, clustering multimedia data in machine learning applications involves various modalities or ``views'' (e.g., text and images), and finding correspondence between shapes in computer graphics problems is an operation performed between two or more modalities. In this thesis, we develop a generalization of spectral methods to deal with multiple data spaces and apply them to problems from the domains of computer graphics, machine learning, and image processing. Our main construction is based on simultaneous diagonalization of Laplacian operators. We present an efficient numerical technique for computing joint approximate eigenvectors of two or more Laplacians in challenging noisy scenarios, which also appears to be the first general non-smooth manifold optimization method. Finally, we use the relation between joint approximate diagonalizability and approximate commutativity of operators to define a structural similarity measure for images. We use this measure to perform structure-preserving color manipulations of a given image
Geometry Processing of Conventionally Produced Mouse Brain Slice Images
Brain mapping research in most neuroanatomical laboratories relies on
conventional processing techniques, which often introduce histological
artifacts such as tissue tears and tissue loss. In this paper we present
techniques and algorithms for automatic registration and 3D reconstruction of
conventionally produced mouse brain slices in a standardized atlas space. This
is achieved first by constructing a virtual 3D mouse brain model from annotated
slices of Allen Reference Atlas (ARA). Virtual re-slicing of the reconstructed
model generates ARA-based slice images corresponding to the microscopic images
of histological brain sections. These image pairs are aligned using a geometric
approach through contour images. Histological artifacts in the microscopic
images are detected and removed using Constrained Delaunay Triangulation before
performing global alignment. Finally, non-linear registration is performed by
solving Laplace's equation with Dirichlet boundary conditions. Our methods
provide significant improvements over previously reported registration
techniques for the tested slices in 3D space, especially on slices with
significant histological artifacts. Further, as an application we count the
number of neurons in various anatomical regions using a dataset of 51
microscopic slices from a single mouse brain. This work represents a
significant contribution to this subfield of neuroscience as it provides tools
to neuroanatomist for analyzing and processing histological data.Comment: 14 pages, 11 figure
Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images
We investigate the problem of learning to generate 3D parametric surface
representations for novel object instances, as seen from one or more views.
Previous work on learning shape reconstruction from multiple views uses
discrete representations such as point clouds or voxels, while continuous
surface generation approaches lack multi-view consistency. We address these
issues by designing neural networks capable of generating high-quality
parametric 3D surfaces which are also consistent between views. Furthermore,
the generated 3D surfaces preserve accurate image pixel to 3D surface point
correspondences, allowing us to lift texture information to reconstruct shapes
with rich geometry and appearance. Our method is supervised and trained on a
public dataset of shapes from common object categories. Quantitative results
indicate that our method significantly outperforms previous work, while
qualitative results demonstrate the high quality of our reconstructions.Comment: ECCV 202
- …