Ontology-based annotation of paintings with artistic concepts

Ph.DDOCTOR OF PHILOSOPH

A multi-resolution multi-source and multi-modal (M3) transductive framework for concept detection in news video

Ph.DDOCTOR OF PHILOSOPH

IMAGE RETRIEVAL BASED ON COMPLEX DESCRIPTIVE QUERIES

The amount of visual data such as images and videos available over web has increased exponentially over the last few years. In order to efficiently organize and exploit these massive collections, a system, apart from being able to answer simple classification based questions such as whether a specific object is present (or absent) in an image, should also be capable of searching images and videos based on more complex descriptive questions. There is also a considerable amount of structure present in the visual world which, if effectively utilized, can help achieve this goal. To this end, we first present an approach for image ranking and retrieval based on queries consisting of multiple semantic attributes. We further show that there are significant correlations present between these attributes and accounting for them can lead to superior performance. Next, we extend this by proposing an image retrieval framework for descriptive queries composed of object categories, semantic attributes and spatial relationships. The proposed framework also includes a unique multi-view hashing technique, which enables query specification in three different modalities - image, sketch and text. We also demonstrate the effectiveness of leveraging contextual information to reduce the supervision requirements for learning object and scene recognition models. We present an active learning framework to simultaneously learn appearance and contextual models for scene understanding. Within this framework we introduce new kinds of labeling questions that are designed to collect appearance as well as contextual information and which mimic the way in which humans actively learn about their environment. Furthermore we explicitly model the contextual interactions between the regions within an image and select the question which leads to the maximum reduction in the combined entropy of all the regions in the image (image entropy)

Computer aided techniques for the attribution of Attic black-figure vase-paintings using the Princeton painter as a model.

Thesis (Ph.D.)-University of KwaZulu-Natal, Durban, 2009.Because of their abundance and because of the insight into the ancient world offered by the depictions on their decorated surfaces, Attic painted ceramics are an extremely valuable source of material evidence. Knowing the identities and personalities of the artists who painted them not only helps us understand the paintings, but also helps in the process of dating them and, in the case of sherds, reconstructing them. However, few of the artists signed their wares, and the identities of the artists have to be revealed through a close analysis of the style in a process called attribution. The vast majority of the attributions of archaic Attic vases are due to John Beazley whose monumental works set the stage for the dominance of attribution studies in the scholarship of Greek ceramics for most of the 20th century. However, the number of new scholars trained in this arcane art is dwindling as new avenues of archaeological research have gained ascendency. A computer-aided technique for attribution may preserve the benefits of the art while allowing new scholars to explore previously ignored areas of research. To this end, the present study provides a theoretical framework for computer-aided attribution, and using the corpus of the Princeton Painter - a painter active in the 6th century BCE - demonstrates the principal that, by employing pattern recognition techniques, computers may be trained to serve as an aid in the attribution process. Three different techniques are presented that are capable of distinguishing between paintings of the Princeton Painter and some of his contemporaries with reasonable accuracy. The first uses shape descriptors to distinguish between the methods employed by respective artists to render minor anatomical details. The second shows that the relative positions of cranial features of the male figures on black-figure paintings is an indicator of style and may also be used as part of the attribution process. Finally a novel technique is presented that can distinguish between pots constructed by different potters based on their shape profiles. This technique may offer valuable clues for attribution when artists are known to work mostly with a single potter

Transductive inference using multiple experts for brushwork annotation in paintings domain

10.1145/1180639.1180684Proceedings of the 14th Annual ACM International Conference on Multimedia, MM 2006157-16

Sensing the Cultural Significance with AI for Social Inclusion

Social Inclusion has been growing as a goal in heritage management. Whereas the 2011 UNESCO Recommendation on the Historic Urban Landscape (HUL) called for tools of knowledge documentation, social media already functions as a platform for online communities to actively involve themselves in heritage-related discussions. Such discussions happen both in “baseline scenarios” when people calmly share their experiences about the cities they live in or travel to, and in “activated scenarios” when radical events trigger their emotions. To organize, process, and analyse the massive unstructured multi-modal (mainly images and texts) user-generated data from social media efficiently and systematically, Artificial Intelligence (AI) is shown to be indispensable. This thesis explores the use of AI in a methodological framework to include the contribution of a larger and more diverse group of participants with user-generated data. It is an interdisciplinary study integrating methods and knowledge from heritage studies, computer science, social sciences, network science, and spatial analysis. AI models were applied, nurtured, and tested, helping to analyse the massive information content to derive the knowledge of cultural significance perceived by online communities. The framework was tested in case study cities including Venice, Paris, Suzhou, Amsterdam, and Rome for the baseline and/or activated scenarios. The AI-based methodological framework proposed in this thesis is shown to be able to collect information in cities and map the knowledge of the communities about cultural significance, fulfilling the expectation and requirement of HUL, useful and informative for future socially inclusive heritage management processes