Transformer for Object Re-Identification: A Survey

Object Re-Identification (Re-ID) aims to identify and retrieve specific objects from varying viewpoints. For a prolonged period, this field has been predominantly driven by deep convolutional neural networks. In recent years, the Transformer has witnessed remarkable advancements in computer vision, prompting an increasing body of research to delve into the application of Transformer in Re-ID. This paper provides a comprehensive review and in-depth analysis of the Transformer-based Re-ID. In categorizing existing works into Image/Video-Based Re-ID, Re-ID with limited data/annotations, Cross-Modal Re-ID, and Special Re-ID Scenarios, we thoroughly elucidate the advantages demonstrated by the Transformer in addressing a multitude of challenges across these domains. Considering the trending unsupervised Re-ID, we propose a new Transformer baseline, UntransReID, achieving state-of-the-art performance on both single-/cross modal tasks. Besides, this survey also covers a wide range of Re-ID research objects, including progress in animal Re-ID. Given the diversity of species in animal Re-ID, we devise a standardized experimental benchmark and conduct extensive experiments to explore the applicability of Transformer for this task to facilitate future research. Finally, we discuss some important yet under-investigated open issues in the big foundation model era, we believe it will serve as a new handbook for researchers in this field

Combining feature aggregation and geometric similarity for re-identification of patterned animals

Image-based re-identification of animal individuals allows gathering of information such as migration patterns of the animals over time. This, together with large image volumes collected using camera traps and crowdsourcing, opens novel possibilities to study animal populations. For many species, the re-identification can be done by analyzing the permanent fur, feather, or skin patterns that are unique to each individual. In this paper, we address the re-identification by combining two types of pattern similarity metrics: 1) pattern appearance similarity obtained by pattern feature aggregation and 2) geometric pattern similarity obtained by analyzing the geometric consistency of pattern similarities. The proposed combination allows to efficiently utilize both the local and global pattern features, providing a general re-identification approach that can be applied to a wide variety of different pattern types. In the experimental part of the work, we demonstrate that the method achieves promising re-identification accuracies for Saimaa ringed seals and whale sharks.Comment: Camera traps, AI, and Ecology, 3rd International Worksho

Extracting Accurate Long-Term Behavior Changes from a Large Pig Dataset

Visual observation of uncontrolled real-world behavior leads to noisy observations, complicated by occlusions, ambiguity, variable motion rates, detection and tracking errors, slow transitions between behaviors, etc. We show in this paper that reliable estimates of long-term trends can be extracted given enough data, even though estimates from individual frames may be noisy. We validate this concept using a new public dataset of approximately 20+ million daytime pig observations over 6 weeks of their main growth stage, and we provide annotations for various tasks including 5 individual behaviors. Our pipeline chains detection, tracking and behavior classification combining deep and shallow computer vision techniques. While individual detections may be noisy, we show that long-term behavior changes can still be extracted reliably, and we validate these results qualitatively on the full dataset. Eventually, starting from raw RGB video data we are able to both tell what pigs main daily activities are, and how these change through time

Pre-trained models are not enough: active and lifelong learning is important for long-term visual monitoring of mammals in biodiversity research—Individual identification and attribute prediction with image features from deep neural networks and decoupled decision models applied to elephants and great apes

Animal re-identification based on image data, either recorded manually by photographers or automatically with camera traps, is an important task for ecological studies about biodiversity and conservation that can be highly automatized with algorithms from computer vision and machine learning. However, fixed identification models only trained with standard datasets before their application will quickly reach their limits, especially for long-term monitoring with changing environmental conditions, varying visual appearances of individuals over time that differ a lot from those in the training data, and new occurring individuals that have not been observed before. Hence, we believe that active learning with human-in-the-loop and continuous lifelong learning is important to tackle these challenges and to obtain high-performance recognition systems when dealing with huge amounts of additional data that become available during the application. Our general approach with image features from deep neural networks and decoupled decision models can be applied to many different mammalian species and is perfectly suited for continuous improvements of the recognition systems via lifelong learning. In our identification experiments, we consider four different taxa, namely two elephant species: African forest elephants and Asian elephants, as well as two species of great apes: gorillas and chimpanzees. Going beyond classical re-identification, our decoupled approach can also be used for predicting attributes of individuals such as gender or age using classification or regression methods. Although applicable for small datasets of individuals as well, we argue that even better recognition performance will be achieved by improving decision models gradually via lifelong learning to exploit huge datasets and continuous recordings from long-term applications. We highlight that algorithms for deploying lifelong learning in real observational studies exist and are ready for use. Hence, lifelong learning might become a valuable concept that supports practitioners when analyzing large-scale image data during long-term monitoring of mammals

Solving Computer Vision Challenges with Synthetic Data

Computer vision researchers spent a lot of time creating large datasets, yet there is still much information that is difficult to label. Detailed annotations like part segmentation and dense keypoint are expensive to annotate. 3D information requires extra hardware to capture. Besides the labeling cost, an image dataset also lacks the ability to allow an intelligent agent to interact with the world. As a human, we learn through interaction, rather than per-pixel labeled images. To fill in the gap of existing datasets, we propose to build virtual worlds using computer graphics and use generated synthetic data to solve these challenges. In this dissertation, I demonstrate cases where computer vision challenges can be solved with synthetic data. The first part describes our engineering effort about building a simulation pipeline. The second and third part describes using synthetic data to train better models and diagnose trained models. The major challenge for using synthetic data is the domain gap between real and synthetic. In the model training part, I present two cases, which have different characteristics in terms of domain gap. Two domain adaptation methods are proposed, respectively. Synthetic data saves enormous labeling effort by providing detailed ground truth. In the model diagnosis part, I present how to control nuisance factors to analyze model robustness. Finally, I summarize future research directions that can benefit from synthetic data

Conflict and conservation : sharing the costs and benefits of tiger (Panthera tigris) conservation in communities adjacent to tiger reserves in Nepal

Human wildlife conflict (HWC) is a highly studied but unresolved aspect of wildlife management. To further knowledge on HWC, this study used a multidisciplinary approach to investigate HWC implications for local people living adjacent to two key tiger conservation reserves in Nepal. The study also assessed the potential to redistribute financial benefits accruing from predator conservation to those bearing costs through associated HWC. Secondary data and anecdotal reports suggest that local people experience significant direct and indirect costs from predator conservation through livestock losses following attacks by common leopards and Bengal tigers, and additionally, crop losses due to their prey species plus two mega herbivores (elephant and one-horned rhinoceros). To investigate this situation, data regarding HWC incidents and costs were sourced through interviews with 422 local households, direct observations, and stakeholder interviews. Collected data included livestock loss (5-year time-period) and crop loss (1-year time-period). Complementary direct observation data collated livestock loss and crop damage for 12 months. Interviews were conducted also with park visitors (N=387) and tourism business owners (N=74). Results showed that tigers are involved in significantly fewer depredation events compared to leopards. Leopards predominantly killed small to medium livestock whereas tigers selected both small to medium and large sized livestock. Livestock depredation events occurred more frequently in livestock corrals relative to forest zones or crop fields. Rates of livestock losses per household per year self-reported during interviews with local people were found higher when compared to those observed by direct measurement. Prey species of tigers and leopards (most often wild boar and chital) were involved in more frequently in crop raiding events, and caused more crop damage, when compared that caused by mega herbivores. Quantities of crops lost per household were lowest in communities where effective physical barriers to wildlife were present. Park visitors and tourism business owners indicated willingness to pay for conservation of tigers and for compensation of farmers for the losses caused by tigers and their prey species. Study findings support several key recommendations proposed to mitigate negative HWC effects in the study area. These include financial support for local communities to build predator proof livestock corrals and establishment of effective physical barriers at the park borders. A dedicated tariff for park visitors and a levy for tourism business owners are also recommended to fund ongoing predator conservation and support financial compensation for local farmers affected by HWC.Doctor of Philosoph

Inter-sexual and inter-seasonal differences in the chemical signalling strategies of brown bears

The brown bear (Ursus arctos) is a species which, due to its solitary, dominance hierarchy social system and large home range, is thought to rely heavily on chemical signals as a means of communication. Through camera traps orientated towards bear ‘rub trees’ over a two-year period, we assessed the proportional contribution of scent marking in different seasons by different age sex classes, and gained insights into the role of chemical signalling in maintaining social structure. We found, during the breeding season (June-July), that both adult males (n=38 P1 year (n=11 P=0.003) scent marked trees significantly more often than expected, whereas lone adult females (n=7) and subadults (n=3) marked less than expected. Outside of the breeding season (August-October), adult males (n=70) marked in an expected proportion, females with cubs (all ages) marked significantly more than expected (n=71 P<0.001), and lone adult females (n=11) and subadults (n=15) marked less than expected. During both the breeding season (n=7 P=0.026) and the fall (n=11 P<0.001), adult females marked trees significantly less than their occurrence on bear trails would expect, as did subadults during the breeding season (n=3 P=0.026) but not during the fall (n=15). Adult males marked at significantly high frequencies both during and outside of the breeding season, potentially to communicate dominance between males. Supported by the low frequency of scent marking by subadults. We observed a total avoidance of bear trails containing active rub trees by females with cubs <1 year during the breeding season, a possible counterstrategy to sexually selected infanticide due to the strong male bias in scent marking during the breeding season. We hypothesize that scent marking in brown bears is taught by the mother, beginning with cubs <1 year outside of the breeding season at a relatively ‘safe’ time of year