Computer Vision for Multimedia Geolocation in Human Trafficking Investigation: A Systematic Literature Review

The task of multimedia geolocation is becoming an increasingly essential component of the digital forensics toolkit to effectively combat human trafficking, child sexual exploitation, and other illegal acts. Typically, metadata-based geolocation information is stripped when multimedia content is shared via instant messaging and social media. The intricacy of geolocating, geotagging, or finding geographical clues in this content is often overly burdensome for investigators. Recent research has shown that contemporary advancements in artificial intelligence, specifically computer vision and deep learning, show significant promise towards expediting the multimedia geolocation task. This systematic literature review thoroughly examines the state-of-the-art leveraging computer vision techniques for multimedia geolocation and assesses their potential to expedite human trafficking investigation. This includes a comprehensive overview of the application of computer vision-based approaches to multimedia geolocation, identifies their applicability in combating human trafficking, and highlights the potential implications of enhanced multimedia geolocation for prosecuting human trafficking. 123 articles inform this systematic literature review. The findings suggest numerous potential paths for future impactful research on the subject

Visual navigation in ants

Les remarquables capacités de navigation des insectes nous prouvent à quel point ces " mini-cerveaux " peuvent produire des comportements admirablement robustes et efficaces dans des environnements complexes. En effet, être capable de naviguer de façon efficace et autonome dans un environnement parfois hostile (désert, forêt tropicale) sollicite l'intervention de nombreux processus cognitifs impliquant l'extraction, la mémorisation et le traitement de l'information spatiale préalables à une prise de décision locomotrice orientée dans l'espace. Lors de leurs excursions hors du nid, les insectes tels que les abeilles, guêpes ou fourmis, se fient à un processus d'intégration du trajet, mais également à des indices visuels qui leur permettent de mémoriser des routes et de retrouver certains sites alimentaires familiers et leur nid. L'étude des mécanismes d'intégration du trajet a fait l'objet de nombreux travaux, par contre, nos connaissances à propos de l'utilisation d'indices visuels sont beaucoup plus limitées et proviennent principalement d'études menées dans des environnements artificiellement simplifiés, dont les conclusions sont parfois difficilement transposables aux conditions naturelles. Cette thèse propose une approche intégrative, combinant 1- des études de terrains et de laboratoire conduites sur deux espèces de fourmis spécialistes de la navigation visuelle (Melophorus bagoti et Gigantiops destructor) et 2- des analyses de photos panoramiques prisent aux endroits où les fourmis naviguent qui permettent de quantifier objectivement l'information visuelle accessible à l'insecte. Les résultats convergents obtenus sur le terrain et au laboratoire permettent de montrer que, chez ces deux espèces, les fourmis ne fragmentent pas leur monde visuel en multiples objets indépendants, et donc ne mémorisent pas de 'repères visuels' ou de balises particuliers comme le ferait un être humain. En fait, l'efficacité de leur navigation émergerait de l'utilisation de paramètres visuels étendus sur l'ensemble de leur champ visuel panoramique, incluant repères proximaux comme distaux, sans les individualiser. Contre-intuitivement, de telles images panoramiques, même à basse résolution, fournissent une information spatiale précise et non ambiguë dans les environnements naturels. Plutôt qu'une focalisation sur des repères isolés, l'utilisation de vues dans leur globalité semble être plus efficace pour représenter la complexité des scènes naturelles et être mieux adaptée à la basse résolution du système visuel des insectes. Les photos panoramiques enregistrées peuvent également servir à l'élaboration de modèles navigationnels. Les prédictions de ces modèles sont ici directement comparées au comportement des fourmis, permettant ainsi de tester et d'améliorer les différentes hypothèses envisagées. Cette approche m'a conduit à la conclusion selon laquelle les fourmis utilisent leurs vues panoramiques de façons différentes suivant qu'elles se déplacent en terrain familier ou non. Par exemple, aligner son corps de manière à ce que la vue perçue reproduise au mieux l'information mémorisée est une stratégie très efficace pour naviguer le long d'une route bien connue ; mais n'est d'aucune efficacité si l'insecte se retrouve en territoire nouveau, écarté du chemin familier. Dans ces cas critiques, les fourmis semblent recourir à une seconde stratégie qui consiste à se déplacer vers les régions présentant une ligne d'horizon plus basse que celle mémorisée, ce qui généralement conduit vers le terrain familier. Afin de choisir parmi ces deux différentes stratégies, les fourmis semblent tout simplement se fier au degré de familiarisation avec le panorama perçu. Cette thèse soulève aussi la question de la nature de l'information visuelle mémorisée par les insectes. Le modèle du " snapshot " qui prédomine dans la littérature suppose que les fourmis mémorisent une séquence d'instantanés photographiques placés à différents points le long de leurs routes. A l'inverse, les résultats obtenus dans le présent travail montrent que l'information visuelle mémorisée au bout d'une route (15 mètres) modifie l'information mémorisée à l'autre extrémité de cette même route, ce qui suggère que la connaissance visuelle de l'ensemble de la route soit compactée en une seule et même représentation mémorisée. Cette hypothèse s'accorde aussi avec d'autres de nos résultats montrant que la mémoire visuelle ne s'acquiert pas instantanément, mais se développe et s'affine avec l'expérience répétée. Lorsqu'une fourmi navigue le long de sa route, ses récepteurs visuels sont stimulés de façon continue par une scène évoluant doucement et régulièrement au fur et à mesure du déplacement. Mémoriser un pattern général de stimulations, plutôt qu'une série de " snapshots " indépendants et très ressemblants les uns aux autres, constitue une hypothèse parcimonieuse. Cette hypothèse s'applique en outre particulièrement bien aux modèles en réseaux de neurones, suggérant sa pertinence biologique. Dans l'ensemble, cette thèse s'intéresse à la nature des perceptions et de la mémoire visuelle des fourmis, ainsi qu'à la manière dont elles sont intégrées et traitées afin de produire une réponse navigationnelle appropriée. Nos résultats sont aussi discutés dans le cadre de la cognition comparée. Insectes comme vertébrés ont résolu le même problème qui consiste à naviguer de façon efficace sur terre. A la lumière de la théorie de l'évolution de Darwin, il n'y a 'a priori' aucune raison de penser qu'il existe une forme de transition brutale entre les mécanismes cognitifs des différentes espèces animales. Le fossé marqué entre insectes et vertébrés au sein des sciences cognitives pourrait bien être dû à des approches différentes plutôt qu'à de vraies différences ontologiques. Historiquement, l'étude de la navigation de l'insecte a suivi une approche de type 'bottom-up' qui recherche comment des comportements apparemment complexes peuvent découler de mécanismes simples. Ces solutions parcimonieuses, comme celles explorées dans cette thèse, peuvent fournir de remarquables hypothèses de base pour expliquer la navigation chez d'autres espèces animales aux cerveaux et comportements apparemment plus complexes, contribuant ainsi à une véritable cognition comparée.Navigating efficiently in the outside world requires many cognitive abilities like extracting, memorising, and processing information. The remarkable navigational abilities of insects are an existence proof of how small brains can produce exquisitely efficient, robust behaviour in complex environments. During their foraging trips, insects, like ants or bees, are known to rely on both path integration and learnt visual cues to recapitulate a route or reach familiar places like the nest. The strategy of path integration is well understood, but much less is known about how insects acquire and use visual information. Field studies give good descriptions of visually guided routes, but our understanding of the underlying mechanisms comes mainly from simplified laboratory conditions using artificial, geometrically simple landmarks. My thesis proposes an integrative approach that combines 1- field and lab experiments on two visually guided ant species (Melophorus bagoti and Gigantiops destructor) and 2- an analysis of panoramic pictures recorded along the animal's route. The use of panoramic pictures allows an objective quantification of the visual information available to the animal. Results from both species, in the lab and the field, converged, showing that ants do not segregate their visual world into objects, such as landmarks or discrete features, as a human observers might assume. Instead, efficient navigation seems to arise from the use of cues widespread on the ants' panoramic visual field, encompassing both proximal and distal objects together. Such relatively unprocessed panoramic views, even at low resolution, provide remarkably unambiguous spatial information in natural environment. Using such a simple but efficient panoramic visual input, rather than focusing on isolated landmarks, seems an appropriate strategy to cope with the complexity of natural scenes and the poor resolution of insects' eyes. Also, panoramic pictures can serve as a basis for running analytical models of navigation. The predictions of these models can be directly compared with the actual behaviour of real ants, allowing the iterative tuning and testing of different hypotheses. This integrative approach led me to the conclusion that ants do not rely on a single navigational technique, but might switch between strategies according to whether they are on or off their familiar terrain. For example, ants can recapitulate robustly a familiar route by simply aligning their body in a way that the current view matches best their memory. However, this strategy becomes ineffective when displaced away from the familiar route. In such a case, ants appear to head instead towards the regions where the skyline appears lower than the height recorded in their memory, which generally leads them closer to a familiar location. How ants choose between strategies at a given time might be simply based on the degree of familiarity of the panoramic scene currently perceived. Finally, this thesis raises questions about the nature of ant memories. Past studies proposed that ants memorise a succession of discrete 2D 'snapshots' of their surroundings. Contrastingly, results obtained here show that knowledge from the end of a foraging route (15 m) impacts strongly on the behaviour at the beginning of the route, suggesting that the visual knowledge of a whole foraging route may be compacted into a single holistic memory. Accordingly, repetitive training on the exact same route clearly affects the ants' behaviour, suggesting that the memorised information is processed and not 'obtained at once'. While navigating along their familiar route, ants' visual system is continually stimulated by a slowly evolving scene, and learning a general pattern of stimulation rather than storing independent but very similar snapshots appears a reasonable hypothesis to explain navigation on a natural scale; such learning works remarkably well with neural networks. Nonetheless, what the precise nature of ants' visual memories is and how elaborated they are remain wide open question. Overall, my thesis tackles the nature of ants' perception and memory as well as how both are processed together to output an appropriate navigational response. These results are discussed in the light of comparative cognition. Both vertebrates and insects have resolved the same problem of navigating efficiently in the world. In light of Darwin's theory of evolution, there is no a priori reason to think that there is a clear division between cognitive mechanisms of different species. The actual gap between insect and vertebrate cognitive sciences may result more from different approaches rather than real differences. Research on insect navigation has been approached with a bottom-up philosophy, one that examines how simple mechanisms can produce seemingly complex behaviour. Such parsimonious solutions, like the ones explored in the present thesis, can provide useful baseline hypotheses for navigation in other larger-brained animals, and thus contribute to a more truly comparative cognition

IDEAS-1997-2021-Final-Programs

This document records the final program for each of the 26 meetings of the International Database and Engineering Application Symposium from 1997 through 2021. These meetings were organized in various locations on three continents. Most of the papers published during these years are in the digital libraries of IEEE(1997-2007) or ACM(2008-2021)

Mechanisms of place recognition and path integration based on the insect visual system

Animals are often able to solve complex navigational tasks in very challenging terrain, despite using low resolution sensors and minimal computational power, providing inspiration for robots. In particular, many species of insect are known to solve complex navigation problems, often combining an array of different behaviours (Wehner et al., 1996; Collett, 1996). Their nervous system is also comparatively simple, relative to that of mammals and other vertebrates. In the first part of this thesis, the visual input of a navigating desert ant, Cataglyphis velox, was mimicked by capturing images in ultraviolet (UV) at similar wavelengths to the ant’s compound eye. The natural segmentation of ground and sky lead to the hypothesis that skyline contours could be used by ants as features for navigation. As proof of concept, sky-segmented binary images were used as input for an established localisation algorithm SeqSLAM (Milford and Wyeth, 2012), validating the plausibility of this claim (Stone et al., 2014). A follow-up investigation sought to determine whether using the sky as a feature would help overcome image matching problems that the ant often faced, such as variance in tilt and yaw rotation. A robotic localisation study showed that using spherical harmonics (SH), a representation in the frequency domain, combined with extracted sky can greatly help robots localise on uneven terrain. Results showed improved performance to state of the art point feature localisation methods on fast bumpy tracks (Stone et al., 2016a). In the second part, an approach to understand how insects perform a navigational task called path integration was attempted by modelling part of the brain of the sweat bee Megalopta genalis. A recent discovery that two populations of cells act as a celestial compass and visual odometer, respectively, led to the hypothesis that circuitry at their point of convergence in the central complex (CX) could give rise to path integration. A firing rate-based model was developed with connectivity derived from the overlap of observed neural arborisations of individual cells and successfully used to build up a home vector and steer an agent back to the nest (Stone et al., 2016b). This approach has the appeal that neural circuitry is highly conserved across insects, so findings here could have wide implications for insect navigation in general. The developed model is the first functioning path integrator that is based on individual cellular connections

Visual homing in field crickets and desert ants: a comparative behavioural and modelling study

Visually guided navigation represents a long standing goal in robotics. Insights may be drawn from various insect species for which visual information has been shown sufficient for navigation in complex environments, however the generality of visual homing abilities across insect species remains unclear. Furthermore variousmodels have been proposed as strategies employed by navigating insects yet comparative studies across models and species are lacking. This work addresses these questions in two insect species not previously studied: the field cricket Gryllus bimaculatus for which almost no navigational data is available; and the European desert ant Cataglyphis velox, a relation of the African desert ant Cataglyphis bicolor which has become a model species for insect navigation studies. The ability of crickets to return to a hidden target using surrounding visual cues was tested using an analogue of the Morris water-maze, a standard paradigm for spatial memory testing in rodents. Crickets learned to re-locate the hidden target using the provided visual cues, with the best performance recorded when a natural image was provided as stimulus rather than clearly identifiable landmarks. The role of vision in navigation was also observed for desert ants within their natural habitat. Foraging ants formed individual, idiosyncratic, visually guided routes through their cluttered surroundings as has been reported in other ant species inhabiting similar environments. In the absence of other cues ants recalled their route even when displaced along their path indicating that ants recall previously visited places rather than a sequence of manoeuvres. Image databases were collected within the environments experienced by the insects using custompanoramic cameras that approximated the insect eye viewof the world. Six biologically plausible visual homing models were implemented and their performance assessed across experimental conditions. The models were first assessed on their ability to replicate the relative performance across the various visual surrounds in which crickets were tested. That is, best performance was sought with the natural scene, followed by blank walls and then the distinct landmarks. Only two models were able to reproduce the pattern of results observed in crickets: pixel-wise image difference with RunDown and the centre of mass average landmark vector. The efficacy of models was then assessed across locations in the ant habitat. A 3D world was generated from the captured images providing noise free and high spatial resolution images asmodel input. Best performancewas found for optic flow and image difference based models. However in many locations the centre of mass average landmark vector failed to provide reliable guidance. This work shows that two previously unstudied insect species can navigate using surrounding visual cues alone. Moreover six biologically plausible models of visual navigation were assessed in the same environments as the insects and only an image difference based model succeeded in all experimental conditions

CompoundRay, an open-source tool for high-speed and high-fidelity rendering of compound eyes

Revealing the functioning of compound eyes is of interest to biologists and engineers alike who wish to understand how visually complex behaviours (e.g. detection, tracking, and navigation) arise in nature, and to abstract concepts to develop novel artificial sensory systems. A key investigative method is to replicate the sensory apparatus using artificial systems, allowing for investigation of the visual information that drives animal behaviour when exposed to environmental cues. To date, ‘compound eye models’ (CEMs) have largely explored features such as field of view and angular resolution, but the role of shape and overall structure have been largely overlooked due to modelling complexity. Modern real-time ray-tracing technologies are enabling the construction of a new generation of computationally fast, high-fidelity CEMs. This work introduces a new open-source CEM software (CompoundRay) that is capable of accurately rendering the visual perspective of bees (6000 individual ommatidia arranged on 2 realistic eye surfaces) at over 3000 frames per second. We show how the speed and accuracy facilitated by this software can be used to investigate pressing research questions (e.g. how low resolution compound eyes can localise small objects) using modern methods (e.g. machine learning-based information exploration)

How is an ant navigation algorithm affected by visual parameters and ego-motion?

Ants typically use path integration and vision for navigation when the environment precludes the use of pheromones for trails. Recent simulations have been able to accurately mimic the retinotopic navigation behaviour of these ants using simple models of movement and memory of unprocessed visual images. Naturally it is interesting to test these navigation algorithms in more realistic circumstances, particularly with actual route data from the ant, in an accurate facsimile of the ant world and with visual input that draws on the characteristics of the animal. While increasing the complexity of the visual processing to include skyline extraction, inhomogeneous sampling and motion processing was conjectured to improve the performance of the simulations, the reverse appears to be the case. Examining closely the assumptions about motion, analysis of ants in the field shows that they experience considerable displacement of the head which when applied to the simulation leads to significant degradation in performance. The family of simulations rely upon continuous visual monitoring of the scene to determine heading and it was decided to test whether the animals were similarly dependent on this input. A field study demonstrated that ants with only visual navigation cues can return the nest when largely facing away from the direction of travel (moving backwards) and so it appears that ant visual navigation is not a process of continuous retinotopic image matching. We conclude ants may use vision to determine an initial heading by image matching and then continue to follow this direction using their celestial compass, or they may use a rotationally invariant form of the visual world for continuous course correction

Applications of a Graph Theoretic Based Clustering Framework in Computer Vision and Pattern Recognition

Recently, several clustering algorithms have been used to solve variety of problems from different discipline. This dissertation aims to address different challenging tasks in computer vision and pattern recognition by casting the problems as a clustering problem. We proposed novel approaches to solve multi-target tracking, visual geo-localization and outlier detection problems using a unified underlining clustering framework, i.e., dominant set clustering and its extensions, and presented a superior result over several state-of-the-art approaches.Comment: doctoral dissertatio