This paper proposes a novel zero-shot edge detection with SCESAME, which
stands for Spectral Clustering-based Ensemble for Segment Anything Model
Estimation, based on the recently proposed Segment Anything Model (SAM). SAM is
a foundation model for segmentation tasks, and one of the interesting
applications of SAM is Automatic Mask Generation (AMG), which generates
zero-shot segmentation masks of an entire image. AMG can be applied to edge
detection, but suffers from the problem of overdetecting edges. Edge detection
with SCESAME overcomes this problem by three steps: (1) eliminating small
generated masks, (2) combining masks by spectral clustering, taking into
account mask positions and overlaps, and (3) removing artifacts after edge
detection. We performed edge detection experiments on two datasets, BSDS500 and
NYUDv2. Although our zero-shot approach is simple, the experimental results on
BSDS500 showed almost identical performance to human performance and CNN-based
methods from seven years ago. In the NYUDv2 experiments, it performed almost as
well as recent CNN-based methods. These results indicate that our method
effectively enhances the utility of SAM and can be a new direction in zero-shot
edge detection methods.Comment: 11 pages, accepted to WACV 2024 Worksho