365 research outputs found
Embodied Scene-aware Human Pose Estimation
We propose embodied scene-aware human pose estimation where we estimate 3D
poses based on a simulated agent's proprioception and scene awareness, along
with external third-person observations. Unlike prior methods that often resort
to multistage optimization, non-causal inference, and complex contact modeling
to estimate human pose and human scene interactions, our method is one stage,
causal, and recovers global 3D human poses in a simulated environment. Since 2D
third-person observations are coupled with the camera pose, we propose to
disentangle the camera pose and use a multi-step projection gradient defined in
the global coordinate frame as the movement cue for our embodied agent.
Leveraging a physics simulation and prescanned scenes (e.g., 3D mesh), we
simulate our agent in everyday environments (libraries, offices, bedrooms,
etc.) and equip our agent with environmental sensors to intelligently navigate
and interact with scene geometries. Our method also relies only on 2D keypoints
and can be trained on synthetic datasets derived from popular human motion
databases. To evaluate, we use the popular H36M and PROX datasets and, for the
first time, achieve a success rate of 96.7% on the challenging PROX dataset
without ever using PROX motion sequences for training.Comment: Project website: https://embodiedscene.github.io/embodiedpose/
Zhengyi Luo and Shun Iwase contributed equall
Deep-MDS Framework for Recovering the 3D Shape of 2D Landmarks from a Single Image
In this paper, a low parameter deep learning framework utilizing the
Non-metric Multi-Dimensional scaling (NMDS) method, is proposed to recover the
3D shape of 2D landmarks on a human face, in a single input image. Hence, NMDS
approach is used for the first time to establish a mapping from a 2D landmark
space to the corresponding 3D shape space. A deep neural network learns the
pairwise dissimilarity among 2D landmarks, used by NMDS approach, whose
objective is to learn the pairwise 3D Euclidean distance of the corresponding
2D landmarks on the input image. This scheme results in a symmetric
dissimilarity matrix, with the rank larger than 2, leading the NMDS approach
toward appropriately recovering the 3D shape of corresponding 2D landmarks. In
the case of posed images and complex image formation processes like perspective
projection which causes occlusion in the input image, we consider an
autoencoder component in the proposed framework, as an occlusion removal part,
which turns different input views of the human face into a profile view. The
results of a performance evaluation using different synthetic and real-world
human face datasets, including Besel Face Model (BFM), CelebA, CoMA - FLAME,
and CASIA-3D, indicates the comparable performance of the proposed framework,
despite its small number of training parameters, with the related
state-of-the-art and powerful 3D reconstruction methods from the literature, in
terms of efficiency and accuracy
Utilization and experimental evaluation of occlusion aware kernel correlation filter tracker using RGB-D
Unlike deep-learning which requires large training datasets, correlation filter-based trackers like Kernelized Correlation Filter (KCF) uses implicit properties of tracked images (circulant matrices) for training in real-time. Despite their practical application in tracking, a need for a better understanding of the fundamentals associated with KCF in terms of theoretically, mathematically, and experimentally exists. This thesis first details the workings prototype of the tracker and investigates its effectiveness in real-time applications and supporting visualizations. We further address some of the drawbacks of the tracker in cases of occlusions, scale changes, object rotation, out-of-view and model drift with our novel RGB-D Kernel Correlation tracker. We also study the use of particle filter to improve trackers\u27 accuracy. Our results are experimentally evaluated using a) standard dataset and b) real-time using Microsoft Kinect V2 sensor. We believe this work will set the basis for better understanding the effectiveness of kernel-based correlation filter trackers and to further define some of its possible advantages in tracking
A Programmable Display-Layer Architecture for Virtual-Reality Applications
Two important technical objectives of virtual-reality systems are to provide compelling visuals and effective 3D user interaction. In this respect, modern virtual reality system architectures suffer from a number of short-comings. The reduction of end-to-end latency, crosstalk and judder are especially difficult challenges, each of which negatively affects visual quality or user interaction. In order to provide higher quality visuals, complex scenes consisting of large models are often used. Rendering such a complex scene is a time-consuming process resulting in high end-to-end latency, thereby hampering user interaction. Classic virtual-reality architectures can not adequately address these challenges due to their inherent design principles. In particular, the tight coupling between input devices, the rendering loop and the display system inhibits these systems from addressing all the aforementioned challenges simultaneously. In this thesis, a virtual-reality architecture design is introduced that is based on the addition of a new logical layer: the Programmable Display Layer (PDL). The governing idea is that an extra layer is inserted between the rendering system and the display. In this way, the display can be updated at a fast rate and in a custom manner independent of the other components in the architecture, including the rendering system. To generate intermediate display updates at a fast rate, the PDL performs per-pixel depth-image warping by utilizing the application data. Image warping is the process of computing a new image by transforming individual depth-pixels from a closely matching previous image to their updated locations. The PDL architecture can be used for a range of algorithms and to solve problems that are not easily solved using classic architectures. In particular, techniques to reduce crosstalk, judder and latency are examined using algorithms implemented on top of the PDL. Concerning user interaction techniques, several six-degrees-of-freedom input methods exists, of which optical tracking is a popular option. However, optical tracking methods also introduce several constraints that depend on the camera setup, such as line-of-sight requirements, the volume of the interaction space and the achieved tracking accuracy. These constraints generally cause a decline in the effectiveness of user interaction. To investigate the effectiveness of optical tracking methods, an optical tracker simulation framework has been developed, including a novel optical tracker to test this framework. In this way, different optical tracking algorithms can be simulated and quantitatively evaluated under a wide range of conditions. A common approach in virtual reality is to implement an algorithm and then to evaluate the efficacy of that algorithm by either subjective, qualitative metrics or quantitative user experiments, after which an updated version of the algorithm may be implemented and the cycle repeated. A different approach is followed here. Throughout this thesis, an attempt is made to automatically detect and quantify errors using completely objective and automated quantitative methods and to subsequently attempt to resolve these errors dynamically
Recommended from our members
Asymmetries of Leadership: Agency, Response and Reason
Drawing on empirical data from an action research project in policing, we propose that the power relations of leadership unfold in asymmetries of agency, response and reason: Leaders both expect and experience more responsibility than control; more blame than praise; and interpretations of failure - both their own and others’ - based more on personal fault than on situational or task complexity. We focus, therefore, on power asymmetry not in the sense of structural inequality between leaders and followers, but rather, as constellations of incongruity, imbalance and unevenness which circumscribe leaders’ actions, choices, relationships and feelings about their work. From this perspective, privilege and disadvantage are not polar opposites reflecting the powerful versus the powerless; instead, they are intimately interwoven within leadership experience. The asymmetries of police leadership involve an intermingling of the necessary and the impossible; a decoupling of failure from irresponsibility; resilience at the prospect of being blamed for success as readily as for failure; and containment of society’s unresolved crises of responsibility, anxiety and risk. We crystallise this as a paradox of transparency and occlusion - of openness and closedness - in which police leaders are scrutinised by, and answerable to, those whom they must also protect, including from having to bear the full burden of knowledge of the dangers of the world. We reflect on the implications of this not just within policing, but for critical understandings of the power of leadership more generally
- …