mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar

Millimeter Wave (mmWave) Radar is gaining popularity as it can work in adverse environments like smoke, rain, snow, poor lighting, etc. Prior work has explored the possibility of reconstructing 3D skeletons or meshes from the noisy and sparse mmWave Radar signals. However, it is unclear how accurately we can reconstruct the 3D body from the mmWave signals across scenes and how it performs compared with cameras, which are important aspects needed to be considered when either using mmWave radars alone or combining them with cameras. To answer these questions, an automatic 3D body annotation system is first designed and built up with multiple sensors to collect a large-scale dataset. The dataset consists of synchronized and calibrated mmWave radar point clouds and RGB(D) images in different scenes and skeleton/mesh annotations for humans in the scenes. With this dataset, we train state-of-the-art methods with inputs from different sensors and test them in various scenarios. The results demonstrate that 1) despite the noise and sparsity of the generated point clouds, the mmWave radar can achieve better reconstruction accuracy than the RGB camera but worse than the depth camera; 2) the reconstruction from the mmWave radar is affected by adverse weather conditions moderately while the RGB(D) camera is severely affected. Further, analysis of the dataset and the results shadow insights on improving the reconstruction from the mmWave radar and the combination of signals from different sensors.Comment: ACM Multimedia 2022, Project Page: https://chen3110.github.io/mmbody/index.htm

ImmFusion: Robust mmWave-RGB Fusion for 3D Human Body Reconstruction in All Weather Conditions

3D human reconstruction from RGB images achieves decent results in good weather conditions but degrades dramatically in rough weather. Complementary, mmWave radars have been employed to reconstruct 3D human joints and meshes in rough weather. However, combining RGB and mmWave signals for robust all-weather 3D human reconstruction is still an open challenge, given the sparse nature of mmWave and the vulnerability of RGB images. In this paper, we present ImmFusion, the first mmWave-RGB fusion solution to reconstruct 3D human bodies in all weather conditions robustly. Specifically, our ImmFusion consists of image and point backbones for token feature extraction and a Transformer module for token fusion. The image and point backbones refine global and local features from original data, and the Fusion Transformer Module aims for effective information fusion of two modalities by dynamically selecting informative tokens. Extensive experiments on a large-scale dataset, mmBody, captured in various environments demonstrate that ImmFusion can efficiently utilize the information of two modalities to achieve a robust 3D human body reconstruction in all weather conditions. In addition, our method's accuracy is significantly superior to that of state-of-the-art Transformer-based LiDAR-camera fusion methods

Brain Storm Optimization Algorithm with an Adaptive Parameter Control Strategy for Finding Multiple Optimal Solutions

Abstract Real-world optimization problems often have multiple optimal solutions and simultaneously finding these optimal solutions is beneficial yet challenging. Brain storm optimization (BSO) is a relatively new paradigm of swarm intelligence algorithm that has been shown to be effective in solving global optimization problems, but it has not been fully exploited for multimodal optimization problems. A simple control strategy for the step size parameter in BSO cannot meet the need of optima finding task in multimodal landscapes and can possibly be refined and optimized. In this paper, we propose an adaptive BSO (ABSO) algorithm that adaptively adjusts the step size parameter according to the quality of newly created solutions. Extensive experiments are conducted on a set of multimodal optimization problems to evaluate the performance of ABSO and the experimental results show that ABSO outperforms existing BSO algorithms and some recently developed algorithms. BSO has great potential in multimodal optimization and is expected to be useful for solving real-world optimization problems that have multiple optimal solutions

Deep Reinforcement Learning Heterogeneous Channels for Poisson Multiple Access

This paper proposes a medium access control (MAC) protocol based on deep reinforcement learning (DRL), i.e., multi-channel transmit deep-reinforcement learning multi-channel access (MCT-DLMA) in heterogeneous wireless networks (HetNets). The work concerns practical unsaturated channel traffic that arrives following a Poisson distribution instead of saturated traffic that arrives before.By learning the access mode from historical information, MCT-DLMA can well fill the spectrum holes in the communication of existing users. In particular, it enables the cognitive user to multi-channel transmit at a time, e.g., via the multi-carrier technology. Thus, the spectrum resource can be fully utilized, and the sum throughput of the HetNet is maximized. Simulation results show that the proposed algorithm provides a much higher throughput than the conventional schemes, i.e., the whittle index policy and the DLMA algorithms for both the saturated and unsaturated traffic, respectively. In addition, it also achieves a near-optimal result in dynamic environments with changing primary users, which proves the enhanced robustness to time-varying communications

Dc field microscopy of Rydberg Li atoms

The Dc field microscopy of Rydberg Li atoms has been studied on the basis of the semiclassical theory for the first time. Especially we discuss the atomic core scattering effect in the ionization dynamics of the Rydberg Li atom. Unlike the case of the photoionization of Rydberg H atom in an electric field, where the photoionization microscopy interference patterns are mainly caused by the Coulomb scattering and the electric field potential, for the photoionization of Rydberg Li atom in an electric field, the influence of the atomic core scattering effect on the photoionization microscopy interference patterns plays an important role. In addition, the structure of the interference pattern, which contains the spatial component of the electronic wave function, evolves smoothly with the electron energy above the saddle point energy. The observed oscillatory patterns in the electron probability density distributions on the detector plane are interpreted within the framework of the semiclassical approximation, which can be considered as a manifestation of interference between various electron trajectories arriving at a given point from the atom to the detector plane. This study provides some reference values for future experimental research on photoionization microscopy of the non-hydrogen Rydberg atoms in the presence of external fields.The accepted manuscript in pdf format is listed with the files at the bottom of this page. The presentation of the authors' names and (or) special characters in the title of the manuscript may differ slightly between what is listed on this page and what is listed in the pdf file of the accepted manuscript; that in the pdf file of the accepted manuscript is what was submitted by the author