Instance Neural Radiance Field

This paper presents one of the first learning-based NeRF 3D instance segmentation pipelines, dubbed as Instance Neural Radiance Field, or Instance NeRF. Taking a NeRF pretrained from multi-view RGB images as input, Instance NeRF can learn 3D instance segmentation of a given scene, represented as an instance field component of the NeRF model. To this end, we adopt a 3D proposal-based mask prediction network on the sampled volumetric features from NeRF, which generates discrete 3D instance masks. The coarse 3D mask prediction is then projected to image space to match 2D segmentation masks from different views generated by existing panoptic segmentation models, which are used to supervise the training of the instance field. Notably, beyond generating consistent 2D segmentation maps from novel views, Instance NeRF can query instance information at any 3D point, which greatly enhances NeRF object segmentation and manipulation. Our method is also one of the first to achieve such results without ground-truth instance information during inference. Experimented on synthetic and real-world NeRF datasets with complex indoor scenes, Instance NeRF surpasses previous NeRF segmentation works and competitive 2D segmentation methods in segmentation performance on unseen views. See the demo video at https://youtu.be/wW9Bme73coI

Deep Reinforcement Learning for Flipper Control of Tracked Robots

The autonomous control of flippers plays an important role in enhancing the intelligent operation of tracked robots within complex environments. While existing methods mainly rely on hand-crafted control models, in this paper, we introduce a novel approach that leverages deep reinforcement learning (DRL) techniques for autonomous flipper control in complex terrains. Specifically, we propose a new DRL network named AT-D3QN, which ensures safe and smooth flipper control for tracked robots. It comprises two modules, a feature extraction and fusion module for extracting and integrating robot and environment state features, and a deep Q-Learning control generation module for incorporating expert knowledge to obtain a smooth and efficient control strategy. To train the network, a novel reward function is proposed, considering both learning efficiency and passing smoothness. A simulation environment is constructed using the Pymunk physics engine for training. We then directly apply the trained model to a more realistic Gazebo simulation for quantitative analysis. The consistently high performance of the proposed approach validates its superiority over manual teleoperation

NeRF-RPN: A general framework for object detection in NeRFs

This paper presents the first significant object detection framework, NeRF-RPN, which directly operates on NeRF. Given a pre-trained NeRF model, NeRF-RPN aims to detect all bounding boxes of objects in a scene. By exploiting a novel voxel representation that incorporates multi-scale 3D neural volumetric features, we demonstrate it is possible to regress the 3D bounding boxes of objects in NeRF directly without rendering the NeRF at any viewpoint. NeRF-RPN is a general framework and can be applied to detect objects without class labels. We experimented the NeRF-RPN with various backbone architectures, RPN head designs and loss functions. All of them can be trained in an end-to-end manner to estimate high quality 3D bounding boxes. To facilitate future research in object detection for NeRF, we built a new benchmark dataset which consists of both synthetic and real-world data with careful labeling and clean up. Please click https://youtu.be/M8_4Ih1CJjE for visualizing the 3D region proposals by our NeRF-RPN. Code and dataset will be made available

Causal Graph ODE: Continuous Treatment Effect Modeling in Multi-agent Dynamical Systems

Real-world multi-agent systems are often dynamic and continuous, where the agents co-evolve and undergo changes in their trajectories and interactions over time. For example, the COVID-19 transmission in the U.S. can be viewed as a multi-agent system, where states act as agents and daily population movements between them are interactions. Estimating the counterfactual outcomes in such systems enables accurate future predictions and effective decision-making, such as formulating COVID-19 policies. However, existing methods fail to model the continuous dynamic effects of treatments on the outcome, especially when multiple treatments (e.g., "stay-at-home" and "get-vaccine" policies) are applied simultaneously. To tackle this challenge, we propose Causal Graph Ordinary Differential Equations (CAG-ODE), a novel model that captures the continuous interaction among agents using a Graph Neural Network (GNN) as the ODE function. The key innovation of our model is to learn time-dependent representations of treatments and incorporate them into the ODE function, enabling precise predictions of potential outcomes. To mitigate confounding bias, we further propose two domain adversarial learning-based objectives, which enable our model to learn balanced continuous representations that are not affected by treatments or interference. Experiments on two datasets (i.e., COVID-19 and tumor growth) demonstrate the superior performance of our proposed model

Case report: A novel 5'-UTR-exon1-intron1 deletion in MLYCD in an IVF child with malonyl coenzyme A decarboxylase deficiency and literature review

The subject of the study is an 11-month old IVF baby girl with the typical clinical manifestation of malonyl coenzyme A decarboxylase deficiency, including developmental delay, limb weakness, cardiomyopathy, and excessive excretion of malonic acid and methylmalonic acid. Whole genome sequencing (WGS) revealed a novel heterozygous nonsense mutation (c.672delG, p.Trp224Ter) in the MLYCD gene of the proband and her father and a novel heterozygous deletion in 5'-UTR-exon1-intron1 of the MLYCD gene of the proband and her mother. The patient's cardiac function and limb weakness improved considerably after 3 months of a low-fat diet supplemented with L-carnitine. Furthermore, mapping of gene mutations and clinical manifestations was done by case collection

Molecular characterization of SARS-CoV-2 nucleocapsid protein

Corona Virus Disease 2019 (COVID-19) is a highly prevalent and potent infectious disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Until now, the world is still endeavoring to develop new ways to diagnose and treat COVID-19. At present, the clinical prevention and treatment of COVID-19 mainly targets the spike protein on the surface of SRAS-CoV-2. However, with the continuous emergence of SARS-CoV-2 Variants of concern (VOC), targeting the spike protein therapy shows a high degree of limitation. The Nucleocapsid Protein (N protein) of SARS-CoV-2 is highly conserved in virus evolution and is involved in the key process of viral infection and assembly. It is the most expressed viral structural protein after SARS-CoV-2 infection in humans and has high immunogenicity. Therefore, N protein as the key factor of virus infection and replication in basic research and clinical application has great potential research value. This article reviews the research progress on the structure and biological function of SARS-CoV-2 N protein, the diagnosis and drug research of targeting N protein, in order to promote researchers’ further understanding of SARS-CoV-2 N protein, and lay a theoretical foundation for the possible outbreak of new and sudden coronavirus infectious diseases in the future

Configurable, wearable sensing and vibrotactile feedback system for real-time postural balance and gait training: proof-of-concept

Abstract Background Postural balance and gait training is important for treating persons with functional impairments, however current systems are generally not portable and are unable to train different types of movements. Methods This paper describes a proof-of-concept design of a configurable, wearable sensing and feedback system for real-time postural balance and gait training targeted for home-based treatments and other portable usage. Sensing and vibrotactile feedback are performed via eight distributed, wireless nodes or “Dots” (size: 22.5 × 20.5 × 15.0 mm, weight: 12.0 g) that can each be configured for sensing and/or feedback according to movement training requirements. In the first experiment, four healthy older adults were trained to reduce medial-lateral (M/L) trunk tilt while performing balance exercises. When trunk tilt deviated too far from vertical (estimated via a sensing Dot on the lower spine), vibrotactile feedback (via feedback Dots placed on the left and right sides of the lower torso) cued participants to move away from the vibration and back toward the vertical no feedback zone to correct their posture. A second experiment was conducted with the same wearable system to train six healthy older adults to alter their foot progression angle in real-time by internally or externally rotating their feet while walking. Foot progression angle was estimated via a sensing Dot adhered to the dorsal side of the foot, and vibrotactile feedback was provided via feedback Dots placed on the medial and lateral sides of the mid-shank cued participants to internally or externally rotate their foot away from vibration. Results In the first experiment, the wearable system enabled participants to significantly reduce trunk tilt and increase the amount of time inside the no feedback zone. In the second experiment, all participants were able to adopt new gait patterns of internal and external foot rotation within two minutes of real-time training with the wearable system. Conclusion These results suggest that the configurable, wearable sensing and feedback system is portable and effective for different types of real-time human movement training and thus may be suitable for home-based or clinic-based rehabilitation applications.https://deepblue.lib.umich.edu/bitstream/2027.42/138819/1/12984_2017_Article_313.pd

On-site and visual detection of sorghum mosaic virus and rice stripe mosaic virus based on reverse transcription-recombinase-aided amplification and CRISPR/Cas12a

Rapid, sensitive and visual detection of plant viruses is conducive to effective prevention and control of plant viral diseases. Therefore, combined with reverse transcription and recombinase-aided amplification, we developed a CRISPR/Cas12a-based visual nucleic acid detection system targeting sorghum mosaic virus and rice stripe mosaic virus, which cause harm to crop production in field. When the RT-RAA products were recognized by crRNA and formed a complex with LbCas12a, the ssDNA labeled with a quenched green fluorescent molecule will be cleaved by LbCas12a, and then a significant green fluorescence signal will appear. The entire detection process can be completed within 30 min without using any sophisticated equipment and instruments. The detection system could detect samples at a dilution of 107, about 104-fold improvement over RT-PCR, so the system was successfully to detect rice stripe mosaic virus in a single leafhopper, which is the transmission vector of the virus. Finally, the CRISPR/Cas12a-based detection system was utilized to on-site detect the two viruses in the field, and the results were fully consistent with that we obtained by RT-PCR in laboratory, demonstrating that it has the application prospect of detecting important crop viruses in the field