Vid2Game: Controllable Characters Extracted from Real-World Videos

We are given a video of a person performing a certain activity, from which we extract a controllable model. The model generates novel image sequences of that person, according to arbitrary user-defined control signals, typically marking the displacement of the moving body. The generated video can have an arbitrary background, and effectively capture both the dynamics and appearance of the person. The method is based on two networks. The first network maps a current pose, and a single-instance control signal to the next pose. The second network maps the current pose, the new pose, and a given background, to an output frame. Both networks include multiple novelties that enable high-quality performance. This is demonstrated on multiple characters extracted from various videos of dancers and athletes

Physics-based Scene-level Reasoning for Object Pose Estimation in Clutter

This paper focuses on vision-based pose estimation for multiple rigid objects placed in clutter, especially in cases involving occlusions and objects resting on each other. Progress has been achieved recently in object recognition given advancements in deep learning. Nevertheless, such tools typically require a large amount of training data and significant manual effort to label objects. This limits their applicability in robotics, where solutions must scale to a large number of objects and variety of conditions. Moreover, the combinatorial nature of the scenes that could arise from the placement of multiple objects is hard to capture in the training dataset. Thus, the learned models might not produce the desired level of precision required for tasks, such as robotic manipulation. This work proposes an autonomous process for pose estimation that spans from data generation to scene-level reasoning and self-learning. In particular, the proposed framework first generates a labeled dataset for training a Convolutional Neural Network (CNN) for object detection in clutter. These detections are used to guide a scene-level optimization process, which considers the interactions between the different objects present in the clutter to output pose estimates of high precision. Furthermore, confident estimates are used to label online real images from multiple views and re-train the process in a self-learning pipeline. Experimental results indicate that this process is quickly able to identify in cluttered scenes physically-consistent object poses that are more precise than the ones found by reasoning over individual instances of objects. Furthermore, the quality of pose estimates increases over time given the self-learning process.Comment: 18 pages, 13 figures, International Journal of Robotics Research (IJRR) 2019. arXiv admin note: text overlap with arXiv:1710.0857

The Normal Map Based on Area-Preserving Parameterization

In this paper, we present an approach to enhance and improve the current normal map rendering technique. Our algorithm is based on semi-discrete Optimal Mass Transportation (OMT) theory and has a solid theoretical base. The key difference from previous normal map method is that we preserve the local area when we unwrap a disk-like 3D surface onto 2D plane. Compared to the currently used techniques which is based on conformal parameterization, our method does not need to cut a surface into many small pieces to avoid the large area distortion. The following charts packing step is also unnecessary in our framework. Our method is practical and makes the normal map technique more robust and efficient.Comment: we need update i

Impedance control of a cable-driven SEA with mixed H2/H∞H_2/H_\infty synthesis

Purpose: This paper presents an impedance control method with mixed $H_2/H_\infty$ synthesis and relaxed passivity for a cable-driven series elastic actuator to be applied for physical human-robot interaction. Design/methodology/approach: To shape the system's impedance to match a desired dynamic model, the impedance control problem was reformulated into an impedance matching structure. The desired competing performance requirements as well as constraints from the physical system can be characterized with weighting functions for respective signals. Considering the frequency properties of human movements, the passivity constraint for stable human-robot interaction, which is required on the entire frequency spectrum and may bring conservative solutions, has been relaxed in such a way that it only restrains the low frequency band. Thus, impedance control became a mixed $H_2/H_\infty$ synthesis problem, and a dynamic output feedback controller can be obtained. Findings: The proposed impedance control strategy has been tested for various desired impedance with both simulation and experiments on the cable-driven series elastic actuator platform. The actual interaction torque tracked well the desired torque within the desired norm bounds, and the control input was regulated below the motor velocity limit. The closed loop system can guarantee relaxed passivity at low frequency. Both simulation and experimental results have validated the feasibility and efficacy of the proposed method. Originality/value: This impedance control strategy with mixed $H_2/H_\infty$ synthesis and relaxed passivity provides a novel, effective and less conservative method for physical human-robot interaction control.Comment: 11 pages, already published in Assembly Automatio

PhotoShape: Photorealistic Materials for Large-Scale Shape Collections

Existing online 3D shape repositories contain thousands of 3D models but lack photorealistic appearance. We present an approach to automatically assign high-quality, realistic appearance models to large scale 3D shape collections. The key idea is to jointly leverage three types of online data -- shape collections, material collections, and photo collections, using the photos as reference to guide assignment of materials to shapes. By generating a large number of synthetic renderings, we train a convolutional neural network to classify materials in real photos, and employ 3D-2D alignment techniques to transfer materials to different parts of each shape model. Our system produces photorealistic, relightable, 3D shapes (PhotoShapes).Comment: To be presented at SIGGRAPH Asia 2018. Project page: https://keunhong.com/publications/photoshape

Learning High Dynamic Range from Outdoor Panoramas

Outdoor lighting has extremely high dynamic range. This makes the process of capturing outdoor environment maps notoriously challenging since special equipment must be used. In this work, we propose an alternative approach. We first capture lighting with a regular, LDR omnidirectional camera, and aim to recover the HDR after the fact via a novel, learning-based inverse tonemapping method. We propose a deep autoencoder framework which regresses linear, high dynamic range data from non-linear, saturated, low dynamic range panoramas. We validate our method through a wide set of experiments on synthetic data, as well as on a novel dataset of real photographs with ground truth. Our approach finds applications in a variety of settings, ranging from outdoor light capture to image matching.Comment: 8 pages + 2 pages of citations, 10 figures. Accepted as an oral paper at ICCV 201

Toward Standardized Classification of Foveated Displays

Emergent in the field of head mounted display design is a desire to leverage the limitations of the human visual system to reduce the computation, communication, and display workload in power and form-factor constrained systems. Fundamental to this reduced workload is the ability to match display resolution to the acuity of the human visual system, along with a resulting need to follow the gaze of the eye as it moves, a process referred to as foveation. A display that moves its content along with the eye may be called a Foveated Display, though this term is also commonly used to describe displays with non-uniform resolution that attempt to mimic human visual acuity. We therefore recommend a definition for the term Foveated Display that accepts both of these interpretations. Furthermore, we include a simplified model for human visual Acuity Distribution Functions (ADFs) at various levels of visual acuity, across wide fields of view and propose comparison of this ADF with the Resolution Distribution Function of a foveated display for evaluation of its resolution at a particular gaze direction. We also provide a taxonomy to allow the field to meaningfully compare and contrast various aspects of foveated displays in a display and optical technology-agnostic manner.Comment: 9 pages, 8 figures, presented at IEEE VR 202

Isospin symmetry breaking

We discuss the separation of isospin-symmetric and isospin-breaking contributions in the hadronic observables within the framework of QCD plus QED. Further, we briefly review some recent work on the low-energy hadron phenomenology, in which the isospin-breaking effect plays a prominent role.Comment: Plenary talk at Sixth International Workshop on Chiral Dynamics, 6-10 July 2009, Bern (Switzerland

Computational Parquetry: Fabricated Style Transfer with Wood Pixels

Parquetry is the art and craft of decorating a surface with a pattern of differently colored veneers of wood, stone or other materials. Traditionally, the process of designing and making parquetry has been driven by color, using the texture found in real wood only for stylization or as a decorative effect. Here, we introduce a computational pipeline that draws from the rich natural structure of strongly textured real-world veneers as a source of detail in order to approximate a target image as faithfully as possible using a manageable number of parts. This challenge is closely related to the established problems of patch-based image synthesis and stylization in some ways, but fundamentally different in others. Most importantly, the limited availability of resources (any piece of wood can only be used once) turns the relatively simple problem of finding the right piece for the target location into the combinatorial problem of finding optimal parts while avoiding resource collisions. We introduce an algorithm that allows to efficiently solve an approximation to the problem. It further addresses challenges like gamut mapping, feature characterization and the search for fabricable cuts. We demonstrate the effectiveness of the system by fabricating a selection of "photo-realistic" pieces of parquetry from different kinds of unstained wood veneer

Strongly First-Order Electroweak Phase Transition and Classical Scale Invariance

In this work, we examine the possibility of realizing a strongly first-order electroweak phase transition within the minimal classically scale invariant extension of the standard model (SM), previously proposed and analyzed as a potential solution to the hierarchy problem. By introducing one complex singlet scalar and three right-handed Majorana neutrinos, the scenario was successfully capable of achieving a radiative breaking of the electroweak symmetry (Coleman-Weinberg Mechanism), inducing non-zero masses for the SM neutrinos (seesaw mechanism), presenting a pseudoscalar dark matter candidate, and predicting the existence of a second $CP$-even boson in addition to the 125 GeV scalar. We construct the full finite-temperature one-loop effective potential of the model, including the resummed thermal daisy loops, and demonstrate that finite-temperature effects induce a first-order electroweak phase transition. Requiring the thermally-driven first-order phase transition to be sufficiently strong further constrains the model's parameter space; in particular, an $\mathcal O(0.01)$ fraction of the dark matter in the universe may be simultaneously accommodated with a strongly first-order electroweak phase transition. Moreover, such a phase transition disfavors right-handed Majorana neutrino masses above several hundreds of GeV, confines the pseudoscalar dark matter masses to $\sim 1$-2 TeV, predicts the mass of the second $CP$-even scalar to be $\sim 100$-300 GeV, and requires the mixing angle between the $CP$-even components of the SM doublet and the complex singlet to lie within the range $0.2 \lesssim \sin\omega \lesssim 0.4$. The obtained results are displayed in comprehensive exclusion plots, identifying the viable regions of the parameter space. Many of these predictions lie within the reach of the next LHC run.Comment: 18 pages, 9 figures. Published version, typos corrected, references adde