mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar

Millimeter Wave (mmWave) Radar is gaining popularity as it can work in adverse environments like smoke, rain, snow, poor lighting, etc. Prior work has explored the possibility of reconstructing 3D skeletons or meshes from the noisy and sparse mmWave Radar signals. However, it is unclear how accurately we can reconstruct the 3D body from the mmWave signals across scenes and how it performs compared with cameras, which are important aspects needed to be considered when either using mmWave radars alone or combining them with cameras. To answer these questions, an automatic 3D body annotation system is first designed and built up with multiple sensors to collect a large-scale dataset. The dataset consists of synchronized and calibrated mmWave radar point clouds and RGB(D) images in different scenes and skeleton/mesh annotations for humans in the scenes. With this dataset, we train state-of-the-art methods with inputs from different sensors and test them in various scenarios. The results demonstrate that 1) despite the noise and sparsity of the generated point clouds, the mmWave radar can achieve better reconstruction accuracy than the RGB camera but worse than the depth camera; 2) the reconstruction from the mmWave radar is affected by adverse weather conditions moderately while the RGB(D) camera is severely affected. Further, analysis of the dataset and the results shadow insights on improving the reconstruction from the mmWave radar and the combination of signals from different sensors.Comment: ACM Multimedia 2022, Project Page: https://chen3110.github.io/mmbody/index.htm

NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction

Implicit neural representations have shown compelling results in offline 3D reconstruction and also recently demonstrated the potential for online SLAM systems. However, applying them to autonomous 3D reconstruction, where robots are required to explore a scene and plan a view path for the reconstruction, has not been studied. In this paper, we explore for the first time the possibility of using implicit neural representations for autonomous 3D scene reconstruction by addressing two key challenges: 1) seeking a criterion to measure the quality of the candidate viewpoints for the view planning based on the new representations, and 2) learning the criterion from data that can generalize to different scenes instead of hand-crafting one. For the first challenge, a proxy of Peak Signal-to-Noise Ratio (PSNR) is proposed to quantify a viewpoint quality. The proxy is acquired by treating the color of a spatial point in a scene as a random variable under a Gaussian distribution rather than a deterministic one; the variance of the distribution quantifies the uncertainty of the reconstruction and composes the proxy. For the second challenge, the proxy is optimized jointly with the parameters of an implicit neural network for the scene. With the proposed view quality criterion, we can then apply the new representations to autonomous 3D reconstruction. Our method demonstrates significant improvements on various metrics for the rendered image quality and the geometry quality of the reconstructed 3D models when compared with variants using TSDF or reconstruction without view planning.Comment: 8 pages, 6 figures, 2 table

Multiple target tracking under occlusions using modified Joint Probabilistic Data Association

International audienceThe size of target will induce a degradation of tracking performance, which has been neglected for simplicity in most previous studies. In multiple target tracking, occlusions will be caused by target size effect, one target can become a moving obstacle blocking the direct channel between the anchor and another target. In this paper, the data association problem in multiple target tracking is investigated. To reduce the computational complexity of traditional Joint Probabilistic Data Association (JPDA) algorithm, a modified JPDA algorithm is proposed to execute data association in multiple target tracking by utilizing the information of occlusion conditions, which is identified by a three-step algorithm. Simulation results show that the proposed algorithm is with good tracking performance and low computational complexity

ImmFusion: Robust mmWave-RGB Fusion for 3D Human Body Reconstruction in All Weather Conditions

3D human reconstruction from RGB images achieves decent results in good weather conditions but degrades dramatically in rough weather. Complementary, mmWave radars have been employed to reconstruct 3D human joints and meshes in rough weather. However, combining RGB and mmWave signals for robust all-weather 3D human reconstruction is still an open challenge, given the sparse nature of mmWave and the vulnerability of RGB images. In this paper, we present ImmFusion, the first mmWave-RGB fusion solution to reconstruct 3D human bodies in all weather conditions robustly. Specifically, our ImmFusion consists of image and point backbones for token feature extraction and a Transformer module for token fusion. The image and point backbones refine global and local features from original data, and the Fusion Transformer Module aims for effective information fusion of two modalities by dynamically selecting informative tokens. Extensive experiments on a large-scale dataset, mmBody, captured in various environments demonstrate that ImmFusion can efficiently utilize the information of two modalities to achieve a robust 3D human body reconstruction in all weather conditions. In addition, our method's accuracy is significantly superior to that of state-of-the-art Transformer-based LiDAR-camera fusion methods

Realtime characteristic of FF like centralized control fieldbus and its state-of-art

Colloque avec actes et comité de lecture. internationale.International audienceThe temporal property of MAC protocol of fieldbus is critical to meet real-time constraints of field devices in factory floor. Among various types of MAC protocols, the one using centralized strategy is characterized by providing feasible schedule to meet different temporal constraints of field devices online, but also providing schedulability analysis offline a priori. WorldFIP and FF, two popular international standards of fieldbus, both adapt centralized strategy, which is mainly implemented by schedule table (ST). This paper mainly discusses how to construct ST, including size of ST, schedule algorithm and schedulability analysis, to meet requirement of field devices on response time, jitter, synchronization, and its State-of-the Art

Decentralized Multi-Charger Coordination for Wireless Rechargeable Sensor Networks

International audienceWireless charging is a promising technology for provisioning dynamic power supply in wireless rechargeable sensor networks (WRSNs). The charging equipment can be carried by some mobile nodes to enhance the charging flexibility. With such mobile chargers (MCs), the charging process should simultaneously address the MC scheduling, the moving and charging time allocation, while saving the total energy consumption of MCs. However, the efficient solutions that jointly solve those challenges are generally lacking in the literature. First, we investigate the multi-MC coordination problem that minimizing the energy expenditure of MCs while guaranteeing the perpetual operation of WRSNs, and formulate this problem as a mixed-integer linear program (MILP). Second, to solve this problem efficiently, we propose a novel decentralized method which is based on Benders decomposition. The multi-MC coordination problem is then decomposed into a master problem (MP) and a slave problem (SP), with the MP for MC scheduling and the SP for MC moving and charging time allocation. The MP is being solved by the base station (BS), while the SP is further decomposed into several sub-SPs and being solved by the MCs in parallel. The BS and MCs coordinate themselves to decide an optimal charging strategy. The convergence of proposed method is analyzed theoretically. Simulation results demonstrate the effectiveness and scalability of the proposed method

Hepatitis B virus spliced variants are associated with an impaired response to interferon therapy

During hepatitis B virus (HBV) replication, spliced HBV genomes and splice-generated proteins have been widely described, however, their biological and clinical significance remains to be defined. Here, an elevation of the proportion of HBV spliced variants in the sera of patients with chronic hepatitis B (CHB) is shown to correlate with an impaired respond to interferon-α (IFN-α) therapy. Transfection of the constructs encoding the three most dominant species of spliced variants into cells or ectopic expression of the two major spliced protein including HBSP and N-terminal-truncated viral polymerase protein result in strong suppression of IFN-α signaling transduction, while mutation of the major splicing-related sites of HBV attenuates the viral anti-IFN activities in both cell and mouse models. These results have associated the productions of HBV spliced variants with the failure response to IFN therapy and illuminate a novel mechanism where spliced viral products are employed to resist IFN-mediated host defense. </p