2 research outputs found
Dynamicity and Durability in Scalable Visual Instance Search.
Visual instance search involves retrieving from a collection of images the
ones that contain an instance of a visual query. Systems designed for visual
instance search face the major challenge of scalability: a collection of a few
million images used for instance search typically creates a few billion
features that must be indexed. Furthermore, as real image collections grow
rapidly, systems must also provide dynamicity, i.e., be able to handle on-line
insertions while concurrently serving retrieval operations. Durability, which
is the ability to recover correctly from software and hardware crashes, is the
natural complement of dynamicity. Durability, however, has rarely been
integrated within scalable and dynamic high-dimensional indexing solutions.
This article addresses the issue of dynamicity and durability for scalable
indexing of very large and rapidly growing collections of local features for
instance retrieval. By extending the NV-tree, a scalable disk-based
high-dimensional index, we show how to implement the ACID properties of
transactions which ensure both dynamicity and durability. We present a detailed
performance evaluation of the transactional NV-tree: (i) We show that the
insertion throughput is excellent despite the overhead for enforcing the ACID
properties; (ii) We also show that this transactional index is truly scalable
using a standard image benchmark embedded in collections of up to 28.5 billion
high-dimensional vectors; the largest single-server evaluations reported in the
literature
The NV-network : a distributed architecture for high-throughput image retrieval
Recently, the computer vision community has started a trend towards advanced image description schemes, where an image yields many local descriptors, each describing a small area of the image. These new schemes are not satisfactorily served by traditional multi-dimensional indexing methods and call for new and advanced database support techniques.
In previous work, we have proposed a new indexing structure, the NV-tree, which repeatedly segments the descriptor collection based on projections to random lines. Although using the NV-tree yields performance which is orders of magnitude faster than previous approaches, it is still unsuitable for high-throughput environments. Therefore, we present the NV-Network which is a distributed architecture built around the NV-tree. The NV-Network mirrors the NV-tree across many worker machines and uses a coordinator machine to manage the system and balance the workers. Our system is designed to be scalable, effective and reliable to hardware failures. The system also has moderate hardware requirements, keeping the hardware cost to a minimum. Experiments show that using the NV-Network results in a very high throughput image retrieval, without any effect on result quality.Rannsóknir á sviði tölvusjónar hafa nýlega lagt áherslu á víðværar myndlýsingar þar sem hverri mynd er lýst af mörgum lýsingum. Fyrri vísar styðja ekki þennan fjölda af lýsingum og því er mikilvægt að næsta kynslóð vísa styðji þennan háa fjölda.
Í fyrri verkum kynntum við NV-tréð, sem endurkvæmt varpar lýsingunum í hólf byggt á vörpunum á slembi línur. Þó að NV-tréð sé margfalt hraðvirkara en fyrri vísar er það engu að síður of hægvirkt til að nota í afkastamiklum kerfum.
Í ljósi þess, kynnum við NV-Netið sem er dreift kerfi smíðað utan um NV-tréð. NVNetið speglar NV-tréð og notar samhæfistjórnun til þess að stjórna kerfinu og dreifa álagi. NV-Netið er hannað til þess að vera skalanlegt, hraðvirkt og áreiðanlegt gagnvart vélbúnaðarbilunum. Kerfið gerir einnig lágar vélbúnaðarkröfur sem halda kostnaði í lágmarki. Mælingar sýna að NV-Netið gefur af sér hámarks afköst án þess að fórna gæði niðurstaðna