Beyond standard benchmarks: Parameterizing performance evaluation in visual object tracking

Object-to-camera motion produces a variety of apparent motion patterns that significantly affect performance of short-term visual trackers. Despite being crucial for designing robust trackers, their influence is poorly explored in standard benchmarks due to weakly defined, biased and overlapping attribute annotations. In this paper we propose to go beyond pre-recorded benchmarks with post-hoc annotations by presenting an approach that utilizes omnidirectional videos to generate realistic, consistently annotated, short-term tracking scenarios with exactly parameterized motion patterns. We have created an evaluation system, constructed a fully annotated dataset of omnidirectional videos and the generators for typical motion patterns. We provide an in-depth analysis of major tracking paradigms which is complementary to the standard benchmarks and confirms the expressiveness of our evaluation approach

The Ninth Visual Object Tracking VOT2021 Challenge Results

acceptedVersionPeer reviewe

A hierarchical adaptive model for robust short-term visual tracking

Vizualno sledenje je področje v okviru računalniškega vida, katerega rezultate je mogoče uporabiti na mnogih, tako novih kot tudi že uveljavljenih, področjih, kot so npr. robotika, video-nadzorni sistemi, interakcija med človekom in računalnikom, avtonomna vozila ter analiza športa. Glavno vprašanje vizualnega sledenja je razvoj algoritmov (sledilnikov), ki določajo stanja enega ali več objektov v toku slik ob upoštevanju časovne soslednosti le-teh. V tej doktorski disertaciji naslavljamo dve raziskovalni temi iz področja kratkoročnega vizualnega sledenja. Prvi sklop predstavljenih raziskav naslavlja konstrukcijo vizualnega modela, ki ga sledilnik uporablja za opis izgleda objekta. Vprašanje modeliranja ter osveževanje vizualnega modela je eno izmed ključnih vprašanj vizualnega sledenja. V okviru dela najprej predstavimo hierarhični vizualni model, ki izgled strukturira v več plasti. Najnižja plast vsebuje najbolj specifične informacije o izgledu, višje plasti pa opisujejo izgled v bolj posplošeni obliki. Hierarhična urejenost se odraža tudi v posodabljanju vizualnega modela, kjer višje plasti vodijo posodabljanje nižjih plasti, le-te pa v primeru lastne zanesljivosti služijo kot vir informacij za osveževanje višjih plasti. Koristi hierarhičnega modela sta predstavljeni z dvema implementacijama, ki sta primarno namenjeni sledenju netogih in artikuliranih objektov, kot tiste kategorije objektov, ki predstavlja velik problem za marsikateri vizualni sledilnik. Prvi predlagani model združuje lokalno in globalno predstavitev izgleda v sklopljenem vizualnem modelu. Spodnja plast je sestavljena iz več med seboj povezanih delov, ki so se sposobni prilagajati geometrijskim spremembam netogih objektov, zgornja plast pa vsebuje večmodalno globalno predstavitev izgleda, ki vodi proces posodobitve spodnje plasti. V okviru eksperimentalne analize smo pokazali, da se tak sklopljeni model izgleda izkaže v robustnosti, klub dejstvu, da smo za opis izgleda uporabili sorazmerno preproste opisnike. Analiza razkrije tudi nekaj pomanjkljivosti modela, ki se kažejo v znižani natančnosti sledenja. Zato naš drugi predstavljeni model razširja hierarhijo s tretjo plastjo in konceptom sidrnih predlog. Prvi dve plasti drugega vizualnega modela sta konceptualno zelo podobni osnovnemu sklopljenemu vizualnemu modelu, tretja plast pa vsebuje spominski sistem statičnih predlog, ki vizualnemu modelu nudijo močno informacijo o položaju in velikosti objekta v primeru dobrega ujemanja ene izmed predlog s sliko. Na ta način tretja plast pripomore k hitremu okrevanju celotnega vizualnega modela. Predstavljena eksperimentalna analiza koristi tretje plasti potrdi, saj sledilnik s tem modelom izgleda izboljša natančnost, pa tudi splošno kvaliteto sledenja. Drugo vprašanje, ki ga naslavljamo v tej doktorski disertaciji, je ocenjevanje performans kartkoročnih sledilnikov. V nasprotju s prevladujočimi trendi v zadnjih desetletjih trdimo, da je vizualno sledenje kompleksen proces, katerega lastnosti ni mogoče opisati z eno samo mero uspešnosti, po drugi strani pa tudi ne smemo uporabiti poljubne množice mer, za katere ne poznamo medsebojnih odnosov. V naši raziskavi smo zato pregledali in analizirali pogosto uporabljene mere performans in pokazali, da nekatere izmed njih merijo iste kvalitete ali pa so celo teoretično ekvivalentne. Na temelju te analize smo predlagali par dveh šibko koreliranih mer, ki odražata natančnost in robustnost sledilnega algoritma, ustrezen prikaz takih rezultatov ter analizo celotne metodologije s pomočjo predlaganih teoretičnih sledilnikov, ki izražajo ekstremno obnašanje sledilnih algoritmov. Vse to smo nadgradili še z metodologijo rangiranja večjega števila sledilnikov, ki upošteva morebitno stohastično naravo sledilnikov ter preveri statistično značilnost razlike med njihovimi rezultati. Celotno metodologijo smo implementirali v odprtokodnem programskem orodju, razvili pa smo tudi preprost komunikacijski protokol, ki omogoča preprosto integracijo obstoječih implementacij sledilnikov v sistem. Z uporabo razvitega orodja se predlagana metodologija sedaj uporablja tudi v okviru Visual Object Tracking (VOT) challenge delavnic in tekmovanj.Visual tracking is a topic in computer vision with applications in many emerging as well as established technological areas, such as robotics, video surveillance, human-computer interaction, autonomous vehicles, and sport analytics. The main question of visual tracking is how to design an algorithm (visual tracker) that determines the state of one or more objects in a stream of images by accounting for their sequential nature. In this doctoral thesis we address two important topics in single-target short-term visual tracking. The first topic is related to construction of an object appearance model for visual tracking. The modeling and updating of the appearance model is crucial for successful tracking. We introduce a hierarchical appearance model which structures object appearance in multiple layers. The bottom layer contains the most specific information and each higher layer models the appearance information in a more general way. The hierarchical relations are also reflected in the update process where the higher layers guide the lower layers in their update while the lower layers provide a source for adaptation to higher layers if their information is reliable. The benefits of hierarchical appearance models are demonstrated with two implementations, primarily designed to tackle tracking of non-rigid and articulated objects that present a challenge for many existing trackers. The first example of appearance model combines local and global visual information in a coupled-layer appearance model. The bottom layer contains a part-based appearance description that is able to adapt to the geometrical deformations of non-rigid targets and the top layer is a multi-modal global object appearance model that guides the model during object appearance changes. The experimental evaluation shows that the proposed coupled-layer appearance model excels in robustness despite the fact that is uses relatively simple appearance descriptors. Our evaluation also exposed several weaknesses that were reflected in a decreased accuracy. Our second presented appearance model extends the hierarchy by introducing the third layer and a concept of template anchors. The first two layers are conceptually similar to the original two-layer appearance model, while the third layer is a memory system that is composed of static templates that provide a strong spatial cue when one of the templates is matched to the image reliably, thus assisting in quick recovery of the entire appearance model. In the experimental evaluation we show that this addition indeed improves the accuracy, as well as the overall performance of a tracker. The second question that we are addressing is the performance evaluation of single-target short-term visual tracking algorithms. In contrast to the dominant trend in the past decades, we claim that visual tracking is a complex process and that the performance of visual trackers cannot be reduced to a single performance measure, nor should it be described by an arbitrary set of measures where the relationship between measures is not well understood. In our research we investigate performance measures that are traditionally used in performance evaluation of single-target short-term visual trackers, through theoretical and empirical analysis, and show that some of them are measuring the same aspect of tracking performance. Based on our analysis we propose a pair of two weakly correlated measures to measure the accuracy and robustness of a tracker, propose a visualization of the results as well as the analysis of the entire methodology using the theoretical trackers that exhibit extreme tracking behaviors. This is followed by an extension of the methodology on ranking of multiple trackers where we also take into account the potentially stochastic nature of visual trackers and test the statistical significance of performance differences. To support the proposed evaluation methodology we have developed an open-source software tool that implements the methodology and a simple communication protocol that enables a straightforward integration of trackers. The proposed evaluation methodology and the evaluation system have been adopted by several Visual Object Tracking (VOT) challenges

A modular toolkit for visual tracking performance evaluation

We present a modular software package for conducting single-target visual object tracking experiments and analyzing results. Our software supports many of the common usage patterns in visual tracking evaluation out of the box, but is also modular and allows various extensions. Users are able to integrate existing implementations of visual tracking algorithms with little additional effort using a standardized and flexible communication protocol. The software has been the technical backbone of the VOT Challenge initiative for many years and has grown and evolved with the competitions that it supported. We present its current state and the capabilities of the package and conclude with some plans for future development

Application of temporal convolutional neural network for the classification of crops on Sentinel-2 time series

The recent development of Earth observation systems - like the Copernicus Sentinels - has provided access to satellite data with high spatial and temporal resolution. This is a key component for the accurate monitoring of state and changes in land use and land cover. In this research, the crops classification was performed by implementing two deep neural networks based on structured data. Despite the wide availability of optical satellite imagery, such as Landsat and Sentinel-2, the limitations of high quality tagged data make the training of machine learning methods very difficult. For this purpose, we have created and labeled a dataset of the crops in Slovenia for the year 2017. With the selected methods we are able to correctly classify 87% of all cultures. Similar studies have already been carried out in the past, but are limited to smaller regions or a smaller number of crop types

Stališča do genetsko modificiranih organizmov v Sloveniji

Objective: Because existing studies examining the impact of knowledge on peopleʼs attitudes towards genetically modified organisms (GMOs) have had contradictory results, the goal of this study was to explore the attitudes that the population of Slovenia has towards GMOs and how knowledge affects their attitudes. Methods: In January 2012, a telephone survey was conducted researching attitudes towards GMOs and knowledge about them on a representative sample of the population of Slovenia (N=446). Results: The results revealed a predominantly negative attitude towards GMOs, regardless of their type, application and geographical distanceperceptions of the negative impact of GMOs on an individualʼs health were particularly strong. The majority of respondents (59.5%) had moderate knowledge about GMOs, while a largeshare (30.4%) had poor knowledge of the topic. They had better objective knowledge about topics linked to formal education or legislation and a weaker understanding of mass media myths. Correlation analysis and one-way analysis of variance showed a statistically significant correlation between knowledge and attitudes towards GMOs. The respondents with better objective knowledge (who gave the correct answers to test questions) had a less firm and a more positive attitude towards GMOs and vice versa. The respondents who lacked objective knowledge but expressed subjective knowledge (they were convinced that their answers were correct) on average had a more negative attitude towards GMOs compared to those who lacked subjective knowledge. Conclusions: This finding leads to the conclusion that knowledge, particularly relating to media myths about GMOs, has an important role in forming attitudes towards the impact of GMOs on an individualʼs health.Namen: Zaradi nasprotujočih si izsledkov obstoječih raziskav o vplivu znanja na stališča o GSO je bil namen študije ugotoviti, kakšna so stališča prebivalcev Slovenije do gensko spremenjenih organizmov (GSO) in kako znanje vpliva na stališča o GSO. Metode: V januarju 2012 je bila izvedena telefonska anketa o stališčih in znanju o GSO na reprezentativnem vzorcu med prebivalci Slovenije (N = 446). Rezultati: Izsledki so pokazali prevladujoče negativno stališče do GSO ne glede na vrsto, uporabo in na zemljepisno oddaljenostpri tem posebej izstopa percepcija vpliva GSO na posameznikovo zdravje. Večina anketiranih (59,5 %) ima o GSO srednje dobro znanjevisok delež (30,4 %) je takih, katerih znanje je slabo. Boljše objektivno znanje imajo o temah iz formalnega izobraževanja ali spremljanja zakonodaje, slabše pa o medijskih mitih. Korelacijska analiza in enosmerna analiza variance sta pokazali, da medznanjem in stališči o GSO obstaja statistično značilna povezanost. Anketiranci z boljšim objektivnim znanjem (pravilni odgovori na testna vprašanja) imajo manj trdno in bolj pozitivno stališče do GSO in nasprotno. Anketiranci brez objektivnega znanja, a z izraženim subjektivnim znanjem (prepričanost o pravilnosti svojih odgovorov) imajo v povprečju bolj negativna stališča do GSO kot tisti, ki nimajo subjektivnega znanja. Zaključki:To pomeni, da ima znanje, še posebej pa medijski miti o GSO, pomembno vlogo pri oblikovanju stališča o vplivu GSO na posameznikovo zdravje

The Visual Object Tracking VOT2017 challenge results

International audienceThe Visual Object Tracking challenge VOT2017 is the fifth annual tracker benchmarking activity organized by the VOT initiative. Results of 51 trackers are presented; many are state-of-the-art published at major computer vision conferences or journals in recent years. The evaluation included the standard VOT and other popular methodologies and a new "real-time" experiment simulating a situation where a tracker processes images as if provided by a continuously running sensor. Performance of the tested trackers typically by far exceeds standard baselines. The source code for most of the trackers is publicly available from the VOT page. The VOT2017 goes beyond its predecessors by (i) improving the VOT public dataset and introducing a separate VOT2017 sequestered dataset, (ii) introducing a realtime tracking experiment and (iii) releasing a redesigned toolkit that supports complex experiments. The dataset, the evaluation kit and the results are publicly available at the challenge websit