Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3D

In recent years, there has been an explosion of 2D vision models for numerous tasks such as semantic segmentation, style transfer or scene editing, enabled by large-scale 2D image datasets. At the same time, there has been renewed interest in 3D scene representations such as neural radiance fields from multi-view images. However, the availability of 3D or multiview data is still substantially limited compared to 2D image datasets, making extending 2D vision models to 3D data highly desirable but also very challenging. Indeed, extending a single 2D vision operator like scene editing to 3D typically requires a highly creative method specialized to that task and often requires per-scene optimization. In this paper, we ask the question of whether any 2D vision model can be lifted to make 3D consistent predictions. We answer this question in the affirmative; our new Lift3D method trains to predict unseen views on feature spaces generated by a few visual models (i.e. DINO and CLIP), but then generalizes to novel vision operators and tasks, such as style transfer, super-resolution, open vocabulary segmentation and image colorization; for some of these tasks, there is no comparable previous 3D method. In many cases, we even outperform state-of-the-art methods specialized for the task in question. Moreover, Lift3D is a zero-shot method, in the sense that it requires no task-specific training, nor scene-specific optimization.Comment: Computer Vision and Pattern Recognition Conference (CVPR), 202

A Construct-Optimize Approach to Sparse View Synthesis without Camera Pose

Novel view synthesis from a sparse set of input images is a challenging problem of great practical interest, especially when camera poses are absent or inaccurate. Direct optimization of camera poses and usage of estimated depths in neural radiance field algorithms usually do not produce good results because of the coupling between poses and depths, and inaccuracies in monocular depth estimation. In this paper, we leverage the recent 3D Gaussian splatting method to develop a novel construct-and-optimize method for sparse view synthesis without camera poses. Specifically, we construct a solution progressively by using monocular depth and projecting pixels back into the 3D world. During construction, we optimize the solution by detecting 2D correspondences between training views and the corresponding rendered images. We develop a unified differentiable pipeline for camera registration and adjustment of both camera poses and depths, followed by back-projection. We also introduce a novel notion of an expected surface in Gaussian splatting, which is critical to our optimization. These steps enable a coarse solution, which can then be low-pass filtered and refined using standard optimization methods. We demonstrate results on the Tanks and Temples and Static Hikes datasets with as few as three widely-spaced views, showing significantly better quality than competing methods, including those with approximate camera pose information. Moreover, our results improve with more views and outperform previous InstantNGP and Gaussian Splatting algorithms even when using half the dataset. Project page: https://raymondjiangkw.github.io/cogs.github.io

One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization

Single image 3D reconstruction is an important but challenging task that requires extensive knowledge of our natural world. Many existing methods solve this problem by optimizing a neural radiance field under the guidance of 2D diffusion models but suffer from lengthy optimization time, 3D inconsistency results, and poor geometry. In this work, we propose a novel method that takes a single image of any object as input and generates a full 360-degree 3D textured mesh in a single feed-forward pass. Given a single image, we first use a view-conditioned 2D diffusion model, Zero123, to generate multi-view images for the input view, and then aim to lift them up to 3D space. Since traditional reconstruction methods struggle with inconsistent multi-view predictions, we build our 3D reconstruction module upon an SDF-based generalizable neural surface reconstruction method and propose several critical training strategies to enable the reconstruction of 360-degree meshes. Without costly optimizations, our method reconstructs 3D shapes in significantly less time than existing methods. Moreover, our method favors better geometry, generates more 3D consistent results, and adheres more closely to the input image. We evaluate our approach on both synthetic data and in-the-wild images and demonstrate its superiority in terms of both mesh quality and runtime. In addition, our approach can seamlessly support the text-to-3D task by integrating with off-the-shelf text-to-image diffusion models.Comment: project website: one-2-3-45.co

TMEM258 Is a Component of the Oligosaccharyltransferase Complex Controlling ER Stress and Intestinal Inflammation

Summary - Significant insights into disease pathogenesis have been gleaned from population-level genetic studies; however, many loci associated with complex genetic disease contain numerous genes, and phenotypic associations cannot be assigned unequivocally. In particular, a gene-dense locus on chromosome 11 (61.5–61.65 Mb) has been associated with inflammatory bowel disease, rheumatoid arthritis, and coronary artery disease. Here, we identify TMEM258 within this locus as a central regulator of intestinal inflammation. Strikingly, Tmem258 haploinsufficient mice exhibit severe intestinal inflammation in a model of colitis. At the mechanistic level, we demonstrate that TMEM258 is a required component of the oligosaccharyltransferase complex and is essential for N-linked protein glycosylation. Consequently, homozygous deficiency of Tmem258 in colonic organoids results in unresolved endoplasmic reticulum (ER) stress culminating in apoptosis. Collectively, our results demonstrate that TMEM258 is a central mediator of ER quality control and intestinal homeostasis.Leona M. and Harry B. Helmsley Charitable Trust (2014PG-IBD016)Crohn's and Colitis Foundation of AmericaNational Institutes of Health (U.S.) (grant DK043351)National Institutes of Health (U.S.) (grant DK097485

Succinate Dehydrogenase Supports Metabolic Repurposing of Mitochondria to Drive Inflammatory Macrophages.

Activated macrophages undergo metabolic reprogramming, which drives their pro-inflammatory phenotype, but the mechanistic basis for this remains obscure. Here, we demonstrate that upon lipopolysaccharide (LPS) stimulation, macrophages shift from producing ATP by oxidative phosphorylation to glycolysis while also increasing succinate levels. We show that increased mitochondrial oxidation of succinate via succinate dehydrogenase (SDH) and an elevation of mitochondrial membrane potential combine to drive mitochondrial reactive oxygen species (ROS) production. RNA sequencing reveals that this combination induces a pro-inflammatory gene expression profile, while an inhibitor of succinate oxidation, dimethyl malonate (DMM), promotes an anti-inflammatory outcome. Blocking ROS production with rotenone by uncoupling mitochondria or by expressing the alternative oxidase (AOX) inhibits this inflammatory phenotype, with AOX protecting mice from LPS lethality. The metabolic alterations that occur upon activation of macrophages therefore repurpose mitochondria from ATP synthesis to ROS production in order to promote a pro-inflammatory state

Search for third-generation scalar leptoquarks in the tτ channel in proton-proton collisions at √s = 8 TeV

A search for pair production of third-generation scalar leptoquarks decaying to top quark and τ lepton pairs is presented using proton-proton collision data at a center-of-mass energy of √s = 8 TeV collected with the CMS detector at the LHC and corresponding to an integrated luminosity of 19.7 fb[superscript −1]. The search is performed using events that contain an electron or a muon, a hadronically decaying τ lepton, and two or more jets. The observations are found to be consistent with the standard model predictions. Assuming that all leptoquarks decay to a top quark and a τ lepton, the existence of pair produced, charge −1/3, third-generation leptoquarks up to a mass of 685 GeV is excluded at 95% confidence level. This result constitutes the first direct limit for leptoquarks decaying into a top quark and a τ lepton, and may also be applied directly to the pair production of bottom squarks decaying predominantly via the R-parity violating coupling λ [subscript 333] ′.United States. Dept. of EnergyNational Science Foundation (U.S.)Alfred P. Sloan Foundatio

Measurement of the transverse momentum spectrum of the Higgs boson produced in pp collisions at √s=8 TeV using H → WW decays

The cross section for Higgs boson production in pp collisions is studied using the H → W[superscript +]W[superscript −] decay mode, followed by leptonic decays of the W bosons to an oppositely charged electron-muon pair in the final state. The measurements are performed using data collected by the CMS experiment at the LHC at a centre-of-mass energy of 8 TeV, corresponding to an integrated luminosity of 19.4 fb[superscript −1]. The Higgs boson transverse momentum (p[subscript T]) is reconstructed using the lepton pair p[subscript T] and missing p[subscript T]. The differential cross section times branching fraction is measured as a function of the Higgs boson pTin a fiducial phase space defined to match the experimental acceptance in terms of the lepton kinematics and event topology. The production cross section times branching fraction in the fiducial phase space is measured to be 39 ± 8 (stat) ± 9 (syst) fb. The measurements are found to agree, within experimental uncertainties, with theoretical calculations based on the standard model. Keywords: Hadron-Hadron scattering (experiments), Higgs physicsNational Science Foundation (U.S.)United States. Department of Energ

First narrow-band search for continuous gravitational waves from known pulsars in advanced detector data

Spinning neutron stars asymmetric with respect to their rotation axis are potential sources of continuous gravitational waves for ground-based interferometric detectors. In the case of known pulsars a fully coherent search, based on matched filtering, which uses the position and rotational parameters obtained from electromagnetic observations, can be carried out. Matched filtering maximizes the signalto- noise (SNR) ratio, but a large sensitivity loss is expected in case of even a very small mismatch between the assumed and the true signal parameters. For this reason, narrow-band analysis methods have been developed, allowing a fully coherent search for gravitational waves from known pulsars over a fraction of a hertz and several spin-down values. In this paper we describe a narrow-band search of 11 pulsars using data from Advanced LIGO’s first observing run. Although we have found several initial outliers, further studies show no significant evidence for the presence of a gravitational wave signal. Finally, we have placed upper limits on the signal strain amplitude lower than the spin-down limit for 5 of the 11 targets over the bands searched; in the case of J1813-1749 the spin-down limit has been beaten for the first time. For an additional 3 targets, the median upper limit across the search bands is below the spin-down limit. This is the most sensitive narrow-band search for continuous gravitational waves carried out so far

Charged-Particle Nuclear Modification Factors in PbPb and pPb Collisions at √sNN = 5.02 TeV

The spectra of charged particles produced within the pseudorapidity window |η| 20 GeV, R][subscript pA] exhibits weak momentum dependence and shows a moderate enhancement above unity.National Science Foundation (U.S.)United States. Department of Energ