260 research outputs found
Motion and Context-Aware Audio-Visual Conditioned Video Prediction
Existing state-of-the-art method for audio-visual conditioned video
prediction uses the latent codes of the audio-visual frames from a multimodal
stochastic network and a frame encoder to predict the next visual frame.
However, a direct inference of per-pixel intensity for the next visual frame
from the latent codes is extremely challenging because of the high-dimensional
image space. To this end, we propose to decouple the audio-visual conditioned
video prediction into motion and appearance modeling. The first part is the
multimodal motion estimation module that learns motion information as optical
flow from the given audio-visual clip. The second part is the context-aware
refinement module that uses the predicted optical flow to warp the current
visual frame into the next visual frame and refines it base on the given
audio-visual context. Experimental results show that our method achieves
competitive results on existing benchmarks.Comment: under consideration at Computer Vision and Image Understandin
Carrier Dynamics in Submonolayer InGaAs/GaAs Quantum Dots
Carrier dynamics of submonolayer (SML) InGaAs/GaAs quantum dots (QDs) were
studied by micro-photoluminecence (MPL), selectively excited photoluminescence
(SEPL), and time-resolved photoluminescence (TRPL). MPL and SEPL show the
coexistence of localized and delocalized states, and different local phonon
modes. TRPL reveal shorter recombination lifetimes and longer capture times for
the QDs with higher emission energy. This suggests that the smallest SML QDs
are formed by perfectly vertically correlated 2D InAs islands, having the
highest In content and the lowest emission energy, while a slight deviation
from the perfectly vertical correlation produces larger QDs with lower In
content and higher emission energy.Comment: 12 pages, 5 figure
Towards Robust Few-shot Point Cloud Semantic Segmentation
Few-shot point cloud semantic segmentation aims to train a model to quickly
adapt to new unseen classes with only a handful of support set samples.
However, the noise-free assumption in the support set can be easily violated in
many practical real-world settings. In this paper, we focus on improving the
robustness of few-shot point cloud segmentation under the detrimental influence
of noisy support sets during testing time. To this end, we first propose a
Component-level Clean Noise Separation (CCNS) representation learning to learn
discriminative feature representations that separates the clean samples of the
target classes from the noisy samples. Leveraging the well separated clean and
noisy support samples from our CCNS, we further propose a Multi-scale
Degree-based Noise Suppression (MDNS) scheme to remove the noisy shots from the
support set. We conduct extensive experiments on various noise settings on two
benchmark datasets. Our results show that the combination of CCNS and MDNS
significantly improves the performance. Our code is available at
https://github.com/Pixie8888/R3DFSSeg.Comment: BMVC 202
Rethink Cross-Modal Fusion in Weakly-Supervised Audio-Visual Video Parsing
Existing works on weakly-supervised audio-visual video parsing adopt hybrid
attention network (HAN) as the multi-modal embedding to capture the cross-modal
context. It embeds the audio and visual modalities with a shared network, where
the cross-attention is performed at the input. However, such an early fusion
method highly entangles the two non-fully correlated modalities and leads to
sub-optimal performance in detecting single-modality events. To deal with this
problem, we propose the messenger-guided mid-fusion transformer to reduce the
uncorrelated cross-modal context in the fusion. The messengers condense the
full cross-modal context into a compact representation to only preserve useful
cross-modal information. Furthermore, due to the fact that microphones capture
audio events from all directions, while cameras only record visual events
within a restricted field of view, there is a more frequent occurrence of
unaligned cross-modal context from audio for visual event predictions. We thus
propose cross-audio prediction consistency to suppress the impact of irrelevant
audio information on visual event prediction. Experiments consistently
illustrate the superior performance of our framework compared to existing
state-of-the-art methods.Comment: WACV 202
A Natural Wind Defrosting, Nano-coated Antibacterial Self-cleaning Energy-saving Health Air-cooled Refrigerator
The air-cooled frost-free household refrigerator is popular in the market because of its large size and frost-free size. However, the evaporator defrost process consumes a large amount of electrical energy to limit the wide spread of this refrigerator, at the same time because of its structural problems, resulting in its evaporator, air duct can not be artificially cleaned, leading to the growth of bacteria, pollution of food storage. This research has developed a self-cleaning energy-saving health refrigerator that uses indoor natural wind defrosting, ultra-hydrophilic nano-titanium dioxide coating photocatalytic sterilization and sterilization. After experimental comparison, under the same operating time of the same operating conditions, the refrigeration mode saves 1.5%, the defrost process saves 95%, reduces the amount of frosting by 23%, the temperature changes of the freezer is less than 7 ℃ , and the desterilization rate of nano-coated reaches 80%
Lattice piecewise affine approximation of explicit nonlinear model predictive control with application to trajectory tracking of mobile robot
To promote the widespread use of mobile robots in diverse fields, the
performance of trajectory tracking must be ensured. To address the constraints
and nonlinear features associated with mobile robot systems, we apply nonlinear
model predictive control (MPC) to realize the trajectory tracking of mobile
robots. Specifically, to alleviate the online computational complexity of
nonlinear MPC, this paper devises a lattice piecewise affine (PWA)
approximation method that can approximate both the nonlinear system and control
law of explicit nonlinear MPC. The kinematic model of the mobile robot is
successively linearized along the trajectory to obtain a linear time-varying
description of the system, which is then expressed using a lattice PWA model.
Subsequently, the nonlinear MPC problem can be transformed into a series of
linear MPC problems. Furthermore, to reduce the complexity of online
calculation of multiple linear MPC problems, we approximate the optimal
solution of the linear MPC by using the lattice PWA model. That is, for
different sampling states, the optimal control inputs are obtained, and lattice
PWA approximations are constructed for the state control pairs. Simulations are
performed to evaluate the performance of our method in comparison with the
linear MPC and explicit linear MPC frameworks. The results show that compared
with the explicit linear MPC, our method has a higher online computing speed
and can decrease the offline computing time without significantly increasing
the tracking error
Generalized Few-Shot Point Cloud Segmentation Via Geometric Words
Existing fully-supervised point cloud segmentation methods suffer in the
dynamic testing environment with emerging new classes. Few-shot point cloud
segmentation algorithms address this problem by learning to adapt to new
classes at the sacrifice of segmentation accuracy for the base classes, which
severely impedes its practicality. This largely motivates us to present the
first attempt at a more practical paradigm of generalized few-shot point cloud
segmentation, which requires the model to generalize to new categories with
only a few support point clouds and simultaneously retain the capability to
segment base classes. We propose the geometric words to represent geometric
components shared between the base and novel classes, and incorporate them into
a novel geometric-aware semantic representation to facilitate better
generalization to the new classes without forgetting the old ones. Moreover, we
introduce geometric prototypes to guide the segmentation with geometric prior
knowledge. Extensive experiments on S3DIS and ScanNet consistently illustrate
the superior performance of our method over baseline methods. Our code is
available at: https://github.com/Pixie8888/GFS-3DSeg_GWs.Comment: Accepted by ICCV 202
- …