Surface-SOS:Self-Supervised Object Segmentation via Neural Surface Representation

Self-supervised Object Segmentation (SOS) aims to segment objects without any annotations. Under conditions of multi-camera inputs, the structural, textural and geometrical consistency among each view can be leveraged to achieve fine-grained object segmentation. To make better use of the above information, we propose Surface representation based Self-supervised Object Segmentation (Surface-SOS), a new framework to segment objects for each view by 3D surface representation from multi-view images of a scene. To model high-quality geometry surfaces for complex scenes, we design a novel scene representation scheme, which decomposes the scene into two complementary neural representation modules respectively with a Signed Distance Function (SDF). Moreover, Surface-SOS is able to refine single-view segmentation with multi-view unlabeled images, by introducing coarse segmentation masks as additional input. To the best of our knowledge, Surface-SOS is the first self-supervised approach that leverages neural surface representation to break the dependence on large amounts of annotated data and strong constraints. These constraints typically involve observing target objects against a static background or relying on temporal supervision in videos. Extensive experiments on standard benchmarks including LLFF, CO3D, BlendedMVS, TUM and several real-world scenes show that Surface-SOS always yields finer object masks than its NeRF-based counterparts and surpasses supervised single-view baselines remarkably.</p

Fusion of multi-scale hyperspectral and LiDAR features for tree species mapping

The added value of multiple data sources on tree species mapping has been widely analyzed. In particular, fusion of hyper-spectral (HS) and LiDAR sensors for forest applications is a very hot topic. In this paper, we exploit the use of multi-scale features to fuse HS and LiDAR data for tree species mapping. Hyperspectral data is obtained from the APEX sensor with 286 spectral bands. LiDAR data has been acquired with a TopoSys sensor Harrier 56 at full waveform. We generate multi-scale features on both HS and LiDAR data, by considering the diameter and the height layer of different tree species. Experimental results on a forested area in Belgium demonstrate the effectiveness of using multi-scale features for fusion of HS image and LiDAR data both visually and quantitatively

PKU-DyMVHumans:A Multi-View Video Benchmark for High-Fidelity Dynamic Human Modeling

High-quality human reconstruction and photo-realistic rendering of a dynamic scene is a long-standing problem in computer vision and graphics. Despite considerable efforts invested in developing various capture systems and reconstruction algorithms, recent advancements still struggle with loose or oversized clothing and overly complex poses. In part, this is due to the challenges of acquiring high-quality human datasets. To facilitate the development of these fields, in this paper, we present PKU-DyMVHumans, a versatile human-centric dataset for high-fidelity reconstruction and rendering of dynamic human scenarios from dense multi-view videos. It comprises 8.2 million frames captured by more than 56 synchronized cameras across diverse scenarios. These sequences comprise 32 human subjects across 45 different scenarios, each with a high-detailed appearance and realistic human motion. Inspired by recent advancements in neural radiance field (NeRF)-based scene representations, we carefully set up an off-the-shelf framework that is easy to provide those state-of-the-art NeRF-based implementations and benchmark on PKU-DyMVHumans dataset. It is paving the way for various applications like fine-grained foreground/background decomposition, high-quality human reconstruction and photo-realistic novel view synthesis of a dynamic scene. Extensive studies are performed on the benchmark, demonstrating new observations and challenges that emerge from using such high-fidelity dynamic data. The dataset is available at: https://pku-dymvhumans.github.io

Memory-Enhancing Effects of the Crude Extract of Polygala tenuifolia on Aged Mice.

Learning and memory disorders arise from distinct age-associated processes, and aging animals are often used as a model of memory impairment. The root of Polygala tenuifolia has been commonly used in some Asian countries as memory enhancer and its memory improvement has been reported in various animal models. However, there is less research to verify its effect on memory functions in aged animals. Herein, the memory-enhancing effects of the crude extract of Polygala tenuifolia (EPT) on normal aged mice were assessed by Morris water maze (MWM) and step-down passive avoidance tests. In MWM tests, the impaired spatial memory of the aged mice was partly reversed by EPT (100 and 200 mg/kg; P < 0.05) as compared with the aged control mice. In step-down tests, the nonspatial memory of the aged mice was improved by EPT (100 and 200 mg/kg; P < 0.05). Additionally, EPT could increase superoxide dismutase (SOD) and catalase (CAT) activities, inhibit monoamine oxidase (MAO) and acetyl cholinesterase (AChE) activities, and decrease the levels of malondialdehyde (MDA) in the brain tissue of the aged mice. The results showed that EPT improved memory functions of the aged mice probably via its antioxidant properties and via decreasing the activities of MAO and AChE

Deep learning for fusion of APEX hyperspectral and full-waveform LiDAR remote sensing data for tree species mapping

Deep learning has been widely used to fuse multi-sensor data for classification. However, current deep learning architecture for multi-sensor data fusion might not always perform better than single data source, especially for the fusion of hyperspectral and light detection and ranging (LiDAR) remote sensing data for tree species mapping in complex, closed forest canopies. In this paper, we propose a new deep fusion framework to integrate the complementary information from hyperspectral and LiDAR data for tree species mapping. We also investigate the fusion of either “single-band” or multi-band (i.e., full-waveform) LiDAR with hyperspectral data for tree species mapping. Additionally, we provide a solution to estimate the crown size of tree species by the fusion of multi-sensor data. Experimental results on fusing real APEX hyperspectral and LiDAR data demonstrate the effectiveness of the proposed deep fusion framework. Compared to using only single data source or current deep fusion architecture, our proposed method yields improvements in overall and average classification accuracies ranging from 82.21% to 87.10% and 76.71% to 83.45%, respectively

Possible Luttinger liquid behavior of edge transport in monolayer transition metal dichalcogenide crystals.

In atomically-thin two-dimensional (2D) semiconductors, the nonuniformity in current flow due to its edge states may alter and even dictate the charge transport properties of the entire device. However, the influence of the edge states on electrical transport in 2D materials has not been sufficiently explored to date. Here, we systematically quantify the edge state contribution to electrical transport in monolayer MoS2/WSe2 field-effect transistors, revealing that the charge transport at low temperature is dominated by the edge conduction with the nonlinear behavior. The metallic edge states are revealed by scanning probe microscopy, scanning Kelvin probe force microscopy and first-principle calculations. Further analyses demonstrate that the edge-state dominated nonlinear transport shows a universal power-law scaling relationship with both temperature and bias voltage, which can be well explained by the 1D Luttinger liquid theory. These findings demonstrate the Luttinger liquid behavior in 2D materials and offer important insights into designing 2D electronics

Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization

Recently, the remarkable advance of the Large Language Model (LLM) has inspired researchers to transfer its extraordinary reasoning capability to both vision and language data. However, the prevailing approaches primarily regard the visual input as a prompt and focus exclusively on optimizing the text generation process conditioned upon vision content by a frozen LLM. Such an inequitable treatment of vision and language heavily constrains the model's potential. In this paper, we break through this limitation by representing both vision and language in a unified form. Specifically, we introduce a well-designed visual tokenizer to translate the non-linguistic image into a sequence of discrete tokens like a foreign language that LLM can read. The resulting visual tokens encompass high-level semantics worthy of a word and also support dynamic sequence length varying from the image. Coped with this tokenizer, the presented foundation model called LaVIT can handle both image and text indiscriminately under the same generative learning paradigm. This unification empowers LaVIT to serve as an impressive generalist interface to understand and generate multi-modal content simultaneously. Extensive experiments further showcase that it outperforms the existing models by a large margin on massive vision-language tasks. Our code and models will be available at https://github.com/jy0205/LaVIT

Ciliary parathyroid hormone signaling activates transforming growth factor-β to maintain intervertebral disc homeostasis during aging

© 2018 The Author(s). Degenerative disc disease (DDD) is associated with intervertebral disc degeneration of spinal instability. Here, we report that the cilia of nucleus pulposus (NP) cells mediate mechanotransduction to maintain anabolic activity in the discs. We found that mechanical stress promotes transport of parathyroid hormone 1 receptor (PTH1R) to the cilia and enhances parathyroid hormone (PTH) signaling in NP cells. PTH induces transcription of integrin αvβ6 to activate the transforming growth factor (TGF)-β-connective tissue growth factor (CCN2)-matrix proteins signaling cascade. Intermittent injection of PTH (iPTH) effectively attenuates disc degeneration of aged mice by direct signaling through NP cells, specifically improving intervertebral disc height and volume by increasing levels of TGF-β activity, CCN2, and aggrecan. PTH1R is expressed in both mouse and human NP cells. Importantly, knockout PTH1R or cilia in the NP cells results in significant disc degeneration and blunts the effect of PTH on attenuation of aged discs. Thus, mechanical stress-induced transport of PTH1R to the cilia enhances PTH signaling, which helps maintain intervertebral disc homeostasis, particularly during aging, indicating therapeutic potential of iPTH for DDD