Towards a Neural Graphics Pipeline for Controllable Image Generation

In this paper, we leverage advances in neural networks towards forming a neural rendering for controllable image generation, and thereby bypassing the need for detailed modeling in conventional graphics pipeline. To this end, we present Neural Graphics Pipeline (NGP), a hybrid generative model that brings together neural and traditional image formation models. NGP decomposes the image into a set of interpretable appearance feature maps, uncovering direct control handles for controllable image generation. To form an image, NGP generates coarse 3D models that are fed into neural rendering modules to produce view-specific interpretable 2D maps, which are then composited into the final output image using a traditional image formation model. Our approach offers control over image generation by providing direct handles controlling illumination and camera parameters, in addition to control over shape and appearance variations. The key challenge is to learn these controls through unsupervised training that links generated coarse 3D models with unpaired real images via neural and traditional (e.g., Blinn-Phong) rendering functions, without establishing an explicit correspondence between them. We demonstrate the effectiveness of our approach on controllable image generation of single-object scenes. We evaluate our hybrid modeling framework, compare with neural-only generation methods (namely, DCGAN, LSGAN, WGAN-GP, VON, and SRNs), report improvement in FID scores against real images, and demonstrate that NGP supports direct controls common in traditional forward rendering. Code is available at http://geometry.cs.ucl.ac.uk/projects/2021/ngp

Inferring object properties from human interaction and transferring them to new motions

Humans regularly interact with their surrounding objects. Such interactions often result in strongly correlated motions between humans and the interacting objects. We thus ask: “Is it possible to infer object properties from skeletal motion alone, even without seeing the interacting object itself?” In this paper, we present a fine-grained action recognition method that learns to infer such latent object properties from human interaction motion alone. This inference allows us to disentangle the motion from the object property and transfer object properties to a given motion. We collected a large number of videos and 3D skeletal motions of performing actors using an inertial motion capture device. We analyzed similar actions and learned subtle differences between them to reveal latent properties of the interacting objects. In particular, we learned to identify the interacting object, by estimating its weight, or its spillability. Our results clearly demonstrate that motions and interacting objects are highly correlated and that related object latent properties can be inferred from 3D skeleton sequences alone, leading to new synthesis possibilities for motions involving human interaction

Consistent Two-Flow Network for Tele-Registration of Point Clouds

Rigid registration of partial observations is a fundamental problem in various applied fields. In computer graphics, special attention has been given to the registration between two partial point clouds generated by scanning devices. State-of-the-art registration techniques still struggle when the overlap region between the two point clouds is small, and completely fail if there is no overlap between the scan pairs. In this paper, we present a learning-based technique that alleviates this problem, and allows registration between point clouds, presented in arbitrary poses, and having little or even no overlap, a setting that has been referred to as tele-registration. Our technique is based on a novel neural network design that learns a prior of a class of shapes and can complete a partial shape. The key idea is combining the registration and completion tasks in a way that reinforces each other. In particular, we simultaneously train the registration network and completion network using two coupled flows, one that register-and-complete, and one that complete-and-register, and encourage the two flows to produce a consistent result. We show that, compared with each separate flow, this two-flow training leads to robust and reliable tele-registration, and hence to a better point cloud prediction that completes the registered scans. It is also worth mentioning that each of the components in our neural network outperforms state-of-the-art methods in both completion and registration. We further analyze our network with several ablation studies and demonstrate its performance on a large number of partial point clouds

Voxelization of Free-Form Solids Represented by Catmull-Clark Subdivision Surfaces

www.cs.uky.edu/∼cheng Abstract. A voxelization technique and its applications for objects with arbitrary topology are presented. It converts a free-form object from its continuous geometric representation into a set of voxels that best approximates the geometry of the object. Unlike traditional 3D scan-conversion based methods, our voxelization method is performed by recursively subdividing the 2D parameter space and sampling 3D points from selected 2D parameter space points. Moreover, our voxelization of 3D closed objects is guaranteed to be leak-free when a 3D flooding operation is performed. This is ensured by proving that our voxelization results satisfy the properties of separability, accuracy and minimality.

Example-based image color and tone style enhancement

Color and tone adjustments are among the most frequent image enhancement operations. We define a color and tone style as a set of explicit or implicit rules governing color and tone adjustments. Our goal in this paper is to learn implicit color and tone adjustment rules from examples. That is, given a set of examples, each of which is a pair of corresponding images before and after adjustments, we would like to discover the underlying mathematical relationships optimally connecting the color and tone of corresponding pixels in all image pairs. We formally define tone and color adjustment rules as mappings, and propose to approximate complicated spatially varying nonlinear mappings in a piecewise manner. The reason behind this is that a very complicated mapping can still be locally approximated with a low-order polynomial model. Parameters within such low-order models are trained using data extracted from example image pairs. We successfully apply our framework in two scenarios, low-quality photo enhancement by transferring the style of a high-end camera, and photo enhancement using styles learned from photographers and designers. © 2011 ACM.postprin

Efficacy and safety of ozanimod in multiple sclerosis: Dose-blinded extension of a randomized phase II study

© The Author(s), 2018. Background: Ozanimod, an oral immunomodulator, selectively targets sphingosine 1-phosphate receptors 1 and 5. Objective: Evaluate efficacy, safety, and tolerability of ozanimod in relapsing multiple sclerosis. Methods: In the RADIANCE Part A phase II study (NCT01628393), participants with relapsing multiple sclerosis were randomized (1:1:1) to once-daily ozanimod hydrochloride (0.5 or 1 mg) or placebo. After 24 weeks, participants could enter a 2-year, dose-blinded extension. Ozanimod-treated participants continued their assigned dose; placebo participants were re-randomized (1:1) to ozanimod hydrochloride 0.5 or 1 mg (equivalent to ozanimod 0.46 and 0.92 mg). Results: A total of 223 (89.6%) of the 249 participants completed the blinded extension. At 2 years of the extension, the percentage of participants who were gadolinium-enhancing lesion-free ranged from 86.5% to 94.6%. Unadjusted annualized relapse rate during the blinded extension (week 24—end of treatment) was 0.32 for ozanimod hydrochloride 0.5 mg → ozanimod hydrochloride 0.5 mg, 0.18 for ozanimod hydrochloride 1 mg → ozanimod hydrochloride 1 mg, 0.30 for placebo → ozanimod hydrochloride 0.5 mg, and 0.18 for placebo → ozanimod hydrochloride 1 mg. No second-degree or higher atrioventricular block or serious opportunistic infection was reported. Conclusion: Ozanimod demonstrated sustained efficacy in participants continuing treatment up to 2 years and reached similar efficacy in participants who switched from placebo; no unexpected safety signals emerged

Build-to-Last: Strength to Weight 3D Printed Objects

The emergence of low-cost 3D printers steers the investigation of new geometric problems that control the quality of the fabricated object. In this paper, we present a method to reduce the material cost and weight of a given object while providing a durable printed model that is resistant to impact and external forces. We introduce a hollowing optimization algorithm based on the concept of honeycomb-cells structure. Honeycombs structures are known to be of minimal material cost while providing strength in tension. We utilize the Voronoi diagram to compute irregular honeycomb-like volume tessellations which define the inner structure. We formulate our problem as a strength–to–weight optimization and cast it as mutually finding an optimal interior tessellation and its maximal hollowing subject to relieve the interior stress. Thus, our system allows to build-to-last 3D printed objects with large control over their strength-to-weight ratio and easily model various interior structures. We demonstrate our method on a collection of 3D objects from different categories. Furthermore, we evaluate our method by printing our hollowed models and measure their stress and weights

Evidence for the different physiological significance of the 6- and 2-minute walk tests in multiple sclerosis

<p>Abstract</p> <p>Background</p> <p>Researchers have recently advocated for the 2-minute walk (2MW) as an alternative for the 6-minute walk (6MW) to assess long distance ambulation in persons with multiple sclerosis (MS). This recommendation has not been based on physiological considerations such as the rate of oxygen consumption (V·O₂) over the 6MW range.</p> <p>Objective</p> <p>This study examined the pattern of change in V·O₂over the range of the 6MW in a large sample of persons with MS who varied as a function of disability status.</p> <p>Method</p> <p>Ninety-five persons with clinically-definite MS underwent a neurological examination for generating an Expanded Disability Status Scale (EDSS) score, and then completion of the 6MW protocol while wearing a portable metabolic unit and an accelerometer.</p> <p>Results</p> <p>There was a time main effect on V·O₂during the 6MW (<it>p </it>= .0001) such that V·O₂increased significantly every 30 seconds over the first 3 minutes of the 6MW, and then remained stable over the second 3 minutes of the 6MW. This occurred despite no change in cadence across the 6MW (<it>p </it>= .84).</p> <p>Conclusions</p> <p>The pattern of change in V·O₂indicates that there are different metabolic systems providing energy for ambulation during the 6MW in MS subjects and steady state aerobic metabolism is reached during the last 3 minutes of the 6MW. By extension, the first 3 minutes would represent a test of mixed aerobic and anaerobic work, whereas the second 3 minutes would represent a test of aerobic work during walking.</p