A multi-view dataset for multimodal Anomaly Detection and Segmentation

Abstract

Anomaly detection is crucial in fields like industrial quality control and defect detection, helping identify deviations from expected behavior to ensure product reliability and safety. Traditional approaches typically rely on datasets, like Eyecandies, MVTec AD and MVTec 3D-AD, that focus on single-modal data or limited multimodal data, which may not be sufficient for detecting complex structural defects. To overcome these limitations, this thesis presents the development of a novel multi-view and multimodal dataset for anomaly detection and segmentation, integrating grayscale images, depth maps and 3D point clouds. Designed as a benchmark for multimodal anomaly detection models, the dataset addresses existing limitations while enabling more robust and generalizable methods. It includes diverse object categories, representing various materials and manufacturing conditions, providing a strong foundation for evaluating anomaly detection algorithms in practical settings. The dataset was collected using a high-precision ATOS Q structured-light 3D scanner in combination with a FANUC robotic arm, ensuring consistency and accuracy across multiple viewpoints. The dataset comprises both real and synthetic samples, where real-world acquisitions, part of this thesis, provide high-fidelity ground truth data and synthetic samples complement the dataset under controlled conditions to improve variability and generalization. This thesis also establishes a standardized data acquisition pipeline for extracting images, STL files and positional data relative to the camera, incorporating calibration processes and manual labeling techniques, ensuring reproducibility. Furthermore, it explores anomaly detection techniques, focusing on PatchCore, a memory-based method for image anomaly detection, and an HHA-based approach that enhances depth features, with a baseline evaluation conducted to validate the dataset and assess performance through AUROC and AUPRO metrics