Compréhension de contenus visuels par analyse conjointe du contenu et des usages

Dans cette thèse, nous traitons de la compréhension de contenus visuels, qu’il s’agisse d’images, de vidéos ou encore de contenus 3D. On entend par compréhension la capacité à inférer des informations sémantiques sur le contenu visuel. L’objectif de ce travail est d’étudier des méthodes combinant deux approches : 1) l’analyse automatique des contenus et 2) l’analyse des interactions liées à l’utilisation de ces contenus (analyse des usages, en plus bref). Dans un premier temps, nous étudions l’état de l’art issu des communautés de la vision par ordinateur et du multimédia. Il y a 20 ans, l’approche dominante visait une compréhension complètement automatique des images. Cette approche laisse aujourd’hui plus de place à différentes formes d’interventions humaines. Ces dernières peuvent se traduire par la constitution d’une base d’apprentissage annotée, par la résolution interactive de problèmes (par exemple de détection ou de segmentation) ou encore par la collecte d’informations implicites issues des usages du contenu. Il existe des liens riches et complexes entre supervision humaine d’algorithmes automatiques et adaptation des contributions humaines via la mise en œuvre d’algorithmes automatiques. Ces liens sont à l’origine de questions de recherche modernes : comment motiver des intervenants humains ? Comment concevoir des scénarii interactifs pour lesquels les interactions contribuent à comprendre le contenu manipulé ? Comment vérifier la qualité des traces collectées ? Comment agréger les données d’usage ? Comment fusionner les données d’usage avec celles, plus classiques, issues d’une analyse automatique ? Notre revue de la littérature aborde ces questions et permet de positionner les contributions de cette thèse. Celles-ci s’articulent en deux grandes parties. La première partie de nos travaux revisite la détection de régions importantes ou saillantes au travers de retours implicites d’utilisateurs qui visualisent ou acquièrent des con- tenus visuels. En 2D d’abord, plusieurs interfaces de vidéos interactives (en particulier la vidéo zoomable) sont conçues pour coordonner des analyses basées sur le contenu avec celles basées sur l’usage. On généralise ces résultats en 3D avec l’introduction d’un nouveau détecteur de régions saillantes déduit de la capture simultanée de vidéos de la même performance artistique publique (spectacles de danse, de chant etc.) par de nombreux utilisateurs. La seconde contribution de notre travail vise une compréhension sémantique d’images fixes. Nous exploitons les données récoltées à travers un jeu, Ask’nSeek, que nous avons créé. Les interactions élémentaires (comme les clics) et les données textuelles saisies par les joueurs sont, comme précédemment, rapprochées d’analyses automatiques des images. Nous montrons en particulier l’intérêt d’interactions révélatrices des relations spatiales entre différents objets détectables dans une même scène. Après la détection des objets d’intérêt dans une scène, nous abordons aussi le problème, plus ambitieux, de la segmentation. ABSTRACT : This thesis focuses on the problem of understanding visual contents, which can be images, videos or 3D contents. Understanding means that we aim at inferring semantic information about the visual content. The goal of our work is to study methods that combine two types of approaches: 1) automatic content analysis and 2) an analysis of how humans interact with the content (in other words, usage analysis). We start by reviewing the state of the art from both Computer Vision and Multimedia communities. Twenty years ago, the main approach was aiming at a fully automatic understanding of images. This approach today gives way to different forms of human intervention, whether it is through the constitution of annotated datasets, or by solving problems interactively (e.g. detection or segmentation), or by the implicit collection of information gathered from content usages. These different types of human intervention are at the heart of modern research questions: how to motivate human contributors? How to design interactive scenarii that will generate interactions that contribute to content understanding? How to check or ensure the quality of human contributions? How to aggregate human contributions? How to fuse inputs obtained from usage analysis with traditional outputs from content analysis? Our literature review addresses these questions and allows us to position the contributions of this thesis. In our first set of contributions we revisit the detection of important (or salient) regions through implicit feedback from users that either consume or produce visual contents. In 2D, we develop several interfaces of interactive video (e.g. zoomable video) in order to coordinate content analysis and usage analysis. We also generalize these results to 3D by introducing a new detector of salient regions that builds upon simultaneous video recordings of the same public artistic performance (dance show, chant, etc.) by multiple users. The second contribution of our work aims at a semantic understanding of fixed images. With this goal in mind, we use data gathered through a game, Ask’nSeek, that we created. Elementary interactions (such as clicks) together with textual input data from players are, as before, mixed with automatic analysis of images. In particular, we show the usefulness of interactions that help revealing spatial relations between different objects in a scene. After studying the problem of detecting objects on a scene, we also adress the more ambitious problem of segmentation

Combining content analysis with usage analysis to better understand visual contents

This thesis focuses on the problem of understanding visual contents, which can be images, videos or 3D contents. Understanding means that we aim at inferring semantic information about the visual content. The goal of our work is to study methods that combine two types of approaches: 1) automatic content analysis and 2) an analysis of how humans interact with the content (in other words, usage analysis). We start by reviewing the state of the art from both Computer Vision and Multimedia communities. Twenty years ago, the main approach was aiming at a fully automatic understanding of images. This approach today gives way to different forms of human intervention, whether it is through the constitution of annotated datasets, or by solving problems interactively (e.g. detection or segmentation), or by the implicit collection of information gathered from content usages. These different types of human intervention are at the heart of modern research questions: how to motivate human contributors? How to design interactive scenarii that will generate interactions that contribute to content understanding? How to check or ensure the quality of human contributions? How to aggregate human contributions? How to fuse inputs obtained from usage analysis with traditional outputs from content analysis? Our literature review addresses these questions and allows us to position the contributions of this thesis. In our first set of contributions we revisit the detection of important (or salient) regions through implicit feedback from users that either consume or produce visual contents. In 2D, we develop several interfaces of interactive video (e.g. zoomable video) in order to coordinate content analysis and usage analysis. We also generalize these results to 3D by introducing a new detector of salient regions that builds upon simultaneous video recordings of the same public artistic performance (dance show, chant, etc.) by multiple users. The second contribution of our work aims at a semantic understanding of fixed images. With this goal in mind, we use data gathered through a game, Ask’nSeek, that we created. Elementary interactions (such as clicks) together with textual input data from players are, as before, mixed with automatic analysis of images. In particular, we show the usefulness of interactions that help revealing spatial relations between different objects in a scene. After studying the problem of detecting objects on a scene, we also adress the more ambitious problem of segmentation

Grim Grimoires: Pragmatic Ritual in the Magic Tantras

Magic tantras, despite their ubiquity in South Asia, have been woefully ignored by Historians of Religions in South Asia. Magic operations or pragmatic rituals, as opposed to transcendental rituals, reveal the lives and anxieties of medieval South Asians. I identify three categories of rituals that constitute a typology of magic in the tantras: (1) six-results rituals (ṣaṭkarman), (2) fantastic feats and enchanted items (kautukakarman, indrajāla), and (3) conjuring (yakṣinīsādhana). I focus primarily on the six-results in this dissertation: tranquilizing (śānti), subjugation (vaśīkaraṇa), immobilization (sthambhana), bewildering (mohana), dissension (vidveṣaṇa), eradication (uccāṭana), attraction (ākarṣaṇa), and murder (māraṇa). No matter having six, nine, or ninety-nine constituents, these are called the six-results. Three scholars have written on the six results previously: Teun Goudriaan, Hans-Georg Türstig, and Gudrun Bühnemann, though only Bühnemann describes the six results without proposing a universal structure and system of magic throughout South Asia, a proposal that is ultimately non-existent. Magic existed prior to the tantras in the form of (1) aggressive lethal magic (abhicāra); (2) ritual enhancements, often child-bearing (abhicāra); (3) conjuring dreadful female witches (kṛtyā, kṛtyābhicāra); and (4) herbal magic pertaining to erotics and sorcery (mulakarman, auṣadhi). I explore these techniques in the Atharvaveda, Manusṃṛṭi, Arthaśāstra, Kāmasūtra, and the Mahābhārata. While the language and many of the techniques found in these pre-medievel sources are echoed in the magic tantras, there is no coherent, unified ritual system that stretches from earlier ritual cultures into the magic tantras. I located a battery of six results rituals in the Uḍḍīśatantra edited and glossed by Tripathī that is reproduced in two other Uḍḍīśatantras; those other two are edited, glossed, and commented upon by Śivadatta and by Śrivāstava. I present each category as described by Tripathī, translating the full operation including mantras, ingredients, ritual actions, and results. I add to Triapthīs description any unique rituals from the other two tantras. I conclude each treatment of the six results rituals by presenting Śrivāstava's lively Hindi-language interpretations of these techniques in contemporary times. A full translation of Tripathī's Uḍḍīśatantra is appended to this dissertation. But magic was not unique to Śaiva sources. Two circa tenth-century Digambara Jain tantras from Karnataka describe all the six results procedures, namely the Bhairavapadmāvatīkalpa and the Jvālāmālinīkalpa. The two Jain tantras represent a curious backwater of medieval Jain ritual culture in which pragmatic rituals for aggressive, martial, and even erotic ends are prescribed without concern for normative Jain ideology on non-violence and asceticism. Titular goddesses Padmāvatī and Jvālāmālinī are Jain deities suited for the agonistic, medieval world in which Śaivas, Buddhists, and Jains competed to secure royal patronage and vied for popularity in contentious religious marketplaces. Presenting six results lore in these texts uncovers a Jain tradition of magic that has never been thoroughly studied and demonstrates contiguity with Śaiva tantra traditions, especially Śrividyā. A full translation of the Bhairavpadmāvatīkalpa is appended to this dissertation. Finally I depart from Śaivism and Jainism and turn to the Buddhist Bhūtaḍāmaratantra, whose main ritual concern is conjuring, the third constituent in my definition of magic. The tantra opens by describing Buddhist Vajradhara/Vajrapāṇi dominating Maheśvara, forcing to Śaiva god to convert to Buddhism and to support Buddhism, to support Buddhists, and to protect all those who practice the rituals in this text. After the conversion narrative, the text describes numerous sequences of goddess spirits who are dominated via spells, hand gestures, and ritual exertions. When presenting conjuring materials in this Buddhist text I note parallels deities and practices, especially yakṣinī spirits and yoginī sādhana practices, that are found throughout the magic tantras. A partial critical edition of the Bhūtaḍāmaratantra created from Nepali manuscripts is appended to this dissertation; its contents are either translated or glossed in the body of this final chapter