97 research outputs found
From 2D Images to 3D Model:Weakly Supervised Multi-View Face Reconstruction with Deep Fusion
We consider the problem of Multi-view 3D Face Reconstruction (MVR) with
weakly supervised learning that leverages a limited number of 2D face images
(e.g. 3) to generate a high-quality 3D face model with very light annotation.
Despite their encouraging performance, present MVR methods simply concatenate
multi-view image features and pay less attention to critical areas (e.g. eye,
brow, nose and mouth). To this end, we propose a novel model called Deep Fusion
MVR (DF-MVR) and design a multi-view encoding to a single decoding framework
with skip connections, able to extract, integrate, and compensate deep features
with attention from multi-view images. In addition, we develop a multi-view
face parse network to learn, identify, and emphasize the critical common face
area. Finally, though our model is trained with a few 2D images, it can
reconstruct an accurate 3D model even if one single 2D image is input. We
conduct extensive experiments to evaluate various multi-view 3D face
reconstruction methods. Our proposed model attains superior performance,
leading to 11.4% RMSE improvement over the existing best weakly supervised
MVRs. Source codes are available in the supplementary materials
Domain Adaptation with Incomplete Target Domains
Domain adaptation, as a task of reducing the annotation cost in a target
domain by exploiting the existing labeled data in an auxiliary source domain,
has received a lot of attention in the research community. However, the
standard domain adaptation has assumed perfectly observed data in both domains,
while in real world applications the existence of missing data can be
prevalent. In this paper, we tackle a more challenging domain adaptation
scenario where one has an incomplete target domain with partially observed
data. We propose an Incomplete Data Imputation based Adversarial Network
(IDIAN) model to address this new domain adaptation challenge. In the proposed
model, we design a data imputation module to fill the missing feature values
based on the partial observations in the target domain, while aligning the two
domains via deep adversarial adaption. We conduct experiments on both
cross-domain benchmark tasks and a real world adaptation task with imperfect
target domains. The experimental results demonstrate the effectiveness of the
proposed method
Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarization
Instance segmentation on point clouds is crucially important for 3D scene
understanding. Distance clustering is commonly used in state-of-the-art methods
(SOTAs), which is typically effective but does not perform well in segmenting
adjacent objects with the same semantic label (especially when they share
neighboring points). Due to the uneven distribution of offset points, these
existing methods can hardly cluster all instance points. To this end, we design
a novel divide and conquer strategy and propose an end-to-end network named
PBNet that binarizes each point and clusters them separately to segment
instances. PBNet divides offset instance points into two categories: high and
low density points (HPs vs.LPs), which are then conquered separately. Adjacent
objects can be clearly separated by removing LPs, and then be completed and
refined by assigning LPs via a neighbor voting method. To further reduce
clustering errors, we develop an iterative merging algorithm based on mean size
to aggregate fragment instances. Experiments on ScanNetV2 and S3DIS datasets
indicate the superiority of our model. In particular, PBNet achieves so far the
best AP50 and AP25 on the ScanNetV2 official benchmark challenge (Validation
Set) while demonstrating high efficiency
MathAttack: Attacking Large Language Models Towards Math Solving Ability
With the boom of Large Language Models (LLMs), the research of solving Math
Word Problem (MWP) has recently made great progress. However, there are few
studies to examine the security of LLMs in math solving ability. Instead of
attacking prompts in the use of LLMs, we propose a MathAttack model to attack
MWP samples which are closer to the essence of security in solving math
problems. Compared to traditional text adversarial attack, it is essential to
preserve the mathematical logic of original MWPs during the attacking. To this
end, we propose logical entity recognition to identify logical entries which
are then frozen. Subsequently, the remaining text are attacked by adopting a
word-level attacker. Furthermore, we propose a new dataset RobustMath to
evaluate the robustness of LLMs in math solving ability. Extensive experiments
on our RobustMath and two another math benchmark datasets GSM8K and MultiAirth
show that MathAttack could effectively attack the math solving ability of LLMs.
In the experiments, we observe that (1) Our adversarial samples from
higher-accuracy LLMs are also effective for attacking LLMs with lower accuracy
(e.g., transfer from larger to smaller-size LLMs, or from few-shot to zero-shot
prompts); (2) Complex MWPs (such as more solving steps, longer text, more
numbers) are more vulnerable to attack; (3) We can improve the robustness of
LLMs by using our adversarial samples in few-shot prompts. Finally, we hope our
practice and observation can serve as an important attempt towards enhancing
the robustness of LLMs in math solving ability. We will release our code and
dataset.Comment: 11 pages, 6 figure
SaliencyCut: Augmenting plausible anomalies for anomaly detection
Anomaly detection under the open-set scenario is a challenging task that requires learning discriminative features to detect anomalies that were even unseen during training. As a cheap yet effective approach, data augmentation has been widely used to create pseudo anomalies for better training of such models. Recent wisdom of augmentation methods focuses on generating random pseudo instances that may lead to a mixture of augmented instances with seen anomalies, or out of the typical range of anomalies. To address this issue, we propose a novel saliency-guided data augmentation method, SaliencyCut, to produce pseudo but more common anomalies that tend to stay in the plausible range of anomalies. Furthermore, we deploy a two-head learning strategy consisting of normal and anomaly learning heads to learn the anomaly score of each sample. Theoretical analyses show that this mechanism offers a more tractable and tighter lower bound of the data log-likelihood. We then design a novel patch-wise residual module in the anomaly learning head to extract and assess anomaly features from each sample, facilitating the learning of discriminative representations of anomaly instances. Extensive experiments conducted on six real-world anomaly detection datasets demonstrate the superiority of our method to competing methods under various settings. Codes are available at: https://github.com/yjnanan/SaliencyCut
Advances of deep learning in electrical impedance tomography image reconstruction
Electrical impedance tomography (EIT) has been widely used in biomedical research because of its advantages of real-time imaging and nature of being non-invasive and radiation-free. Additionally, it can reconstruct the distribution or changes in electrical properties in the sensing area. Recently, with the significant advancements in the use of deep learning in intelligent medical imaging, EIT image reconstruction based on deep learning has received considerable attention. This study introduces the basic principles of EIT and summarizes the application progress of deep learning in EIT image reconstruction with regards to three aspects: a single network reconstruction, deep learning combined with traditional algorithm reconstruction, and multiple network hybrid reconstruction. In future, optimizing the datasets may be the main challenge in applying deep learning for EIT image reconstruction. Adopting a better network structure, focusing on the joint reconstruction of EIT and traditional algorithms, and using multimodal deep learning-based EIT may be the solution to existing problems. In general, deep learning offers a fresh approach for improving the performance of EIT image reconstruction and could be the foundation for building an intelligent integrated EIT diagnostic system in the future
- …