SAM3D: Segment Anything in 3D Scenes

In this work, we propose SAM3D, a novel framework that is able to predict masks in 3D point clouds by leveraging the Segment-Anything Model (SAM) in RGB images without further training or finetuning. For a point cloud of a 3D scene with posed RGB images, we first predict segmentation masks of RGB images with SAM, and then project the 2D masks into the 3D points. Later, we merge the 3D masks iteratively with a bottom-up merging approach. At each step, we merge the point cloud masks of two adjacent frames with the bidirectional merging approach. In this way, the 3D masks predicted from different frames are gradually merged into the 3D masks of the whole 3D scene. Finally, we can optionally ensemble the result from our SAM3D with the over-segmentation results based on the geometric information of the 3D scenes. Our approach is experimented with ScanNet dataset and qualitative results demonstrate that our SAM3D achieves reasonable and fine-grained 3D segmentation results without any training or finetuning of SAM.Comment: Technical Report. The code is released at https://github.com/Pointcept/SegmentAnything3

GPT4Point: A Unified Framework for Point-Language Understanding and Generation

Multimodal Large Language Models (MLLMs) have excelled in 2D image-text comprehension and image generation, but their understanding of the 3D world is notably deficient, limiting progress in 3D language understanding and generation. To solve this problem, we introduce GPT4Point, an innovative groundbreaking point-language multimodal model designed specifically for unified 3D object understanding and generation within the MLLM framework. GPT4Point as a powerful 3D MLLM seamlessly can execute a variety of point-text reference tasks such as point-cloud captioning and Q&A. Additionally, GPT4Point is equipped with advanced capabilities for controllable 3D generation, it can get high-quality results through a low-quality point-text feature maintaining the geometric shapes and colors. To support the expansive needs of 3D object-text pairs, we develop Pyramid-XL, a point-language dataset annotation engine. It constructs a large-scale database over 1M objects of varied text granularity levels from the Objaverse-XL dataset, essential for training GPT4Point. A comprehensive benchmark has been proposed to evaluate 3D point-language understanding capabilities. In extensive evaluations, GPT4Point has demonstrated superior performance in understanding and generation

Pronounced Increases in Nitrogen Emissions and Deposition Due to the Historic 2020 Wildfires in the Western U.S.

Wildfire outbreaks can lead to extreme biomass burning (BB) emissions of both oxidized (e.g., nitrogen oxides; NOx= NO+NO2) and reduced form(e.g., ammonia; NH3) nitrogen (N) compounds. High N emissions aremajor concerns for air quality, atmospheric deposition, and consequential human and ecosystemhealth impacts. In this study, we use both satellite-based observations and modeling results to quantify the contribution of BB to the total emissions, and approximate the impact on total N deposition in the western U.S. Our results show that during the 2020 wildfire season of August–October, BB contributes significantly to the total emissions, with a satellite-derived fraction of NH3 to the total reactiveN emissions (median~40%) in the range of aircraft observations. During the peak of the western August Complex Fires in September, BB contributed to~55%(for the contiguous U.S.) and~83%(for thewestern U.S.) of the monthly total NOx and NH3 emissions. Overall, there is good model performance of the George Mason University- Wildfire Forecasting System(GMU-WFS) used in this work. The extreme BB emissions lead to significant contributions to the total N deposition for different ecosystems in California, with an average August – October 2020 relative increase of~78%(from7.1 to 12.6 kg ha−1 year−1) in deposition rate tomajor vegetation types (mixed forests+grasslands/ shrublands/savanna) compared to the GMU-WFS simulations without BB emissions. For mixed forest types only, the average N deposition rate increases (from 6.2 to 16.9 kg ha−1 year−1) are even larger at ~173%. Such large N deposition due to extreme BB emissions are much (~6-12 times) larger than low-end critical load thresholds for major vegetation types (e.g., forests at 1.5-3 kg ha−1 year−1), and thus may result in adverse N deposition effects across larger areas of lichen communities found in California\u27s mixed conifer forests

Advancements and future outlook in fundamental research and technological applications for ammonia co-firing with coal

Energy security and the “dual carbon” goals are impacting the global energy industry and supply chains, presenting some urgent needs for the secure, efficient, and green low-carbon transformation of energy and power systems in China. To ensure energy security, the coal-fired power plant will remain an important support for electricity supply security and the integration of renewable energy for a considerable period into the future in China. To achieve carbon neutrality, the energy system is gradually shifting from primarily relying on fossil fuels to primarily relying on renewable energy. However, the intermittency, randomness, and volatility of renewable energy make power system regulation more challenging, highlighting the issues of system balance and security. In this context, the zero-carbon fuels such as ammonia play an indispensable role in dealing with the intermittency of renewable energy. They can serve as carriers for long-term and large-scale energy storage to facilitate the large-scale integration of renewable energy and can also be flexibly used directly in power equipment such as boilers to replace fossil fuels. However, due to the differences in physical and chemical properties between zero-carbon fuels and fossil fuels, some challenges arise in the widespread utilization of ammonia fuel, such as poor flame stability and the generation of nitrogen oxides during combustion. Therefore, based on the feasibility, economy and necessity of ammoniac-coal co-combustion, this paper comprehensively reviews the chemical reaction kinetics, combustion characteristics (ignition, steady combustion) and pollutants characteristics (NOx, fly ash particles and soot) of ammoniac-coal co-combustion, and discusses the scaling laws of burner based on the dimensionless number. The efficient and stable combustion control strategy of existing industrial grade ammonia-coal burners is discussed in detail. The C—N fuel separation, air staged and their joint control technology can effectively reduce NOx emissions. In the future, artificial intelligence, big data and digital twin and other information technologies are integrated. It is expected to provide a scientific support and path reference for the research and development of the next generation of new green power generation system oriented to the dual carbon strategy from the source

PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm

In contrast to numerous NLP and 2D computer vision foundational models, the learning of a robust and highly generalized 3D foundational model poses considerably greater challenges. This is primarily due to the inherent data variability and the diversity of downstream tasks. In this paper, we introduce a comprehensive 3D pre-training framework designed to facilitate the acquisition of efficient 3D representations, thereby establishing a pathway to 3D foundational models. Motivated by the fact that informative 3D features should be able to encode rich geometry and appearance cues that can be utilized to render realistic images, we propose a novel universal paradigm to learn point cloud representations by differentiable neural rendering, serving as a bridge between 3D and 2D worlds. We train a point cloud encoder within a devised volumetric neural renderer by comparing the rendered images with the real images. Notably, our approach demonstrates the seamless integration of the learned 3D encoder into diverse downstream tasks. These tasks encompass not only high-level challenges such as 3D detection and segmentation but also low-level objectives like 3D reconstruction and image synthesis, spanning both indoor and outdoor scenarios. Besides, we also illustrate the capability of pre-training a 2D backbone using the proposed universal methodology, surpassing conventional pre-training methods by a large margin. For the first time, PonderV2 achieves state-of-the-art performance on 11 indoor and outdoor benchmarks. The consistent improvements in various settings imply the effectiveness of the proposed method. Code and models will be made available at https://github.com/OpenGVLab/PonderV2.Comment: arXiv admin note: text overlap with arXiv:2301.0015

KARR-seq reveals cellular higher-order RNA structures and RNA–RNA interactions

RNA fate and function are affected by their structures and interactomes. However, how RNA and RNA-binding proteins (RBPs) assemble into higher-order structures and how RNA molecules may interact with each other to facilitate functions remain largely unknown. Here we present KARR-seq, which uses N3-kethoxal labeling and multifunctional chemical crosslinkers to covalently trap and determine RNA–RNA interactions and higher-order RNA structures inside cells, independent of local protein binding to RNA. KARR-seq depicts higher-order RNA structure and detects widespread intermolecular RNA–RNA interactions with high sensitivity and accuracy. Using KARR-seq, we show that translation represses mRNA compaction under native and stress conditions. We determined the higher-order RNA structures of respiratory syncytial virus (RSV) and vesicular stomatitis virus (VSV) and identified RNA–RNA interactions between the viruses and the host RNAs that potentially regulate viral replication

Changes in glycosylated proteins in colostrum and mature milk and their implication

IntroductionGlycosylation is one of the essential post-translational modifications that influences the function of milk proteins.MethodsIn the present study, 998 proteins and 764 glycosylated sites from 402 glycoproteins were identified in human milk by TMT labeling proteomics. Compared to human milk proteins, the glycoproteins were mainly enriched in cell adhesion, proteolysis, and defense/immune process.ResultsThe abundance of 353 glycosylated sites and their 179 parent proteins was quantified. After normalization to their parent protein’s abundance, 78 glycosylated sites in 56 glycoproteins and 10 glycosylated sites in 10 glycoproteins were significantly higher in colostrum and mature milk, respectively. These changed glycoproteins were mainly related to host defense. Intriguingly, one glycosylated site (Asp144) in IgA and two glycosylated sites (Asp38 and Asp1079) in tenascin are significantly upregulated even though their protein abundance was downregulated during lactation.DiscussionThis study helps us figure out the critical glycosylated sites in proteins that might influence their biological function in an unbiased way

Improved Cellular Specificity of Plasmonic Nanobubbles versus Nanoparticles in Heterogeneous Cell Systems

The limited specificity of nanoparticle (NP) uptake by target cells associated with a disease is one of the principal challenges of nanomedicine. Using the threshold mechanism of plasmonic nanobubble (PNB) generation and enhanced accumulation and clustering of gold nanoparticles in target cells, we increased the specificity of PNB generation and detection in target versus non-target cells by more than one order of magnitude compared to the specificity of NP uptake by the same cells. This improved cellular specificity of PNBs was demonstrated in six different cell models representing diverse molecular targets such as epidermal growth factor receptor, CD3 receptor, prostate specific membrane antigen and mucin molecule MUC1. Thus PNBs may be a universal method and nano-agent that overcome the problem of non-specific uptake of NPs by non-target cells and improve the specificity of NP-based diagnostics, therapeutics and theranostics at the cell level