In this vision paper, we propose a shift in perspective for improving the
effectiveness of similarity search. Rather than focusing solely on enhancing
the data quality, particularly machine learning-generated embeddings, we
advocate for a more comprehensive approach that also enhances the underpinning
search mechanisms. We highlight three novel avenues that call for a
redefinition of the similarity search problem: exploiting implicit data
structures and distributions, engaging users in an iterative feedback loop, and
moving beyond a single query vector. These novel pathways have gained relevance
in emerging applications such as large-scale language models, video clip
retrieval, and data labeling. We discuss the corresponding research challenges
posed by these new problem areas and share insights from our preliminary
discoveries