Most image retrieval research focuses on improving predictive performance,
but they may fall short in scenarios where the reliability of the prediction is
crucial. Though uncertainty quantification can help by assessing uncertainty
for query and database images, this method can provide only a heuristic
estimate rather than an guarantee. To address these limitations, we present
Risk Controlled Image Retrieval (RCIR), which generates retrieval sets that are
guaranteed to contain the ground truth samples with a predefined probability.
RCIR can be easily plugged into any image retrieval method, agnostic to data
distribution and model selection. To the best of our knowledge, this is the
first work that provides coverage guarantees for image retrieval. The validity
and efficiency of RCIR is demonstrated on four real-world image retrieval
datasets, including the Stanford CAR-196 (Krause et al. 2013), CUB-200 (Wah et
al. 2011), the Pittsburgh dataset (Torii et al. 2013) and the ChestX-Det
dataset (Lian et al. 2021)