178 research outputs found
Context-Dependent Diffusion Network for Visual Relationship Detection
Visual relationship detection can bridge the gap between computer vision and
natural language for scene understanding of images. Different from pure object
recognition tasks, the relation triplets of subject-predicate-object lie on an
extreme diversity space, such as \textit{person-behind-person} and
\textit{car-behind-building}, while suffering from the problem of combinatorial
explosion. In this paper, we propose a context-dependent diffusion network
(CDDN) framework to deal with visual relationship detection. To capture the
interactions of different object instances, two types of graphs, word semantic
graph and visual scene graph, are constructed to encode global context
interdependency. The semantic graph is built through language priors to model
semantic correlations across objects, whilst the visual scene graph defines the
connections of scene objects so as to utilize the surrounding scene
information. For the graph-structured data, we design a diffusion network to
adaptively aggregate information from contexts, which can effectively learn
latent representations of visual relationships and well cater to visual
relationship detection in view of its isomorphic invariance to graphs.
Experiments on two widely-used datasets demonstrate that our proposed method is
more effective and achieves the state-of-the-art performance.Comment: 8 pages, 3 figures, 2018 ACM Multimedia Conference (MM'18
Real Time Animation of Virtual Humans: A Trade-off Between Naturalness and Control
Virtual humans are employed in many interactive applications using 3D virtual environments, including (serious) games. The motion of such virtual humans should look realistic (or ‘natural’) and allow interaction with the surroundings and other (virtual) humans. Current animation techniques differ in the trade-off they offer between motion naturalness and the control that can be exerted over the motion. We show mechanisms to parametrize, combine (on different body parts) and concatenate motions generated by different animation techniques. We discuss several aspects of motion naturalness and show how it can be evaluated. We conclude by showing the promise of combinations of different animation paradigms to enhance both naturalness and control
PointMCD: Boosting Deep Point Cloud Encoders via Multi-view Cross-modal Distillation for 3D Shape Recognition
As two fundamental representation modalities of 3D objects, 3D point clouds
and multi-view 2D images record shape information from different domains of
geometric structures and visual appearances. In the current deep learning era,
remarkable progress in processing such two data modalities has been achieved
through respectively customizing compatible 3D and 2D network architectures.
However, unlike multi-view image-based 2D visual modeling paradigms, which have
shown leading performance in several common 3D shape recognition benchmarks,
point cloud-based 3D geometric modeling paradigms are still highly limited by
insufficient learning capacity, due to the difficulty of extracting
discriminative features from irregular geometric signals. In this paper, we
explore the possibility of boosting deep 3D point cloud encoders by
transferring visual knowledge extracted from deep 2D image encoders under a
standard teacher-student distillation workflow. Generally, we propose PointMCD,
a unified multi-view cross-modal distillation architecture, including a
pretrained deep image encoder as the teacher and a deep point encoder as the
student. To perform heterogeneous feature alignment between 2D visual and 3D
geometric domains, we further investigate visibility-aware feature projection
(VAFP), by which point-wise embeddings are reasonably aggregated into
view-specific geometric descriptors. By pair-wisely aligning multi-view visual
and geometric descriptors, we can obtain more powerful deep point encoders
without exhausting and complicated network modification. Experiments on 3D
shape classification, part segmentation, and unsupervised learning strongly
validate the effectiveness of our method. The code and data will be publicly
available at https://github.com/keeganhk/PointMCD
- …