Transferring Neural Representations for Low-dimensional Indexing of Maya Hieroglyphic Art

We analyze the performance of deep neural architectures for extracting shape representations of binary images, and for generating low-dimensional representations of them. In particular, we focus on indexing binary images exhibiting compounds of Maya hieroglyphic signs, referred to as glyph-blocks, which constitute a very challenging dataset of arts given their visual complexity and large stylistic variety. More precisely, we demonstrate empirically that intermediate outputs of convolutional neural networks can be used as representations for complex shapes, even when their parameters are trained on gray-scale images, and that these representations can be more robust than traditional handcrafted features. We also show that it is possible to compress such representations up to only three dimensions without harming much of their discriminative structure, such that effective visualization of Maya hieroglyphs can be rendered for subsequent epigraphic analysis

Visual Analysis of Maya Glyphs via Crowdsourcing and Deep Learning

In this dissertation, we study visual analysis methods for complex ancient Maya writings. The unit sign of a Maya text is called glyph, and may have either semantic or syllabic significance. There are over 800 identified glyph categories, and over 1400 variations across these categories. To enable fast manipulation of data by scholars in Humanities, it is desirable to have automatic visual analysis tools such as glyph categorization, localization, and visualization. Analysis and recognition of glyphs are challenging problems. The same patterns may be observed in different signs but with different compositions. The inter-class variance can thus be significantly low. On the opposite, the intra-class variance can be high, as the visual variants within the same semantic category may differ to a large extent except for some patterns specific to the category. Another related challenge of Maya writings is the lack of a large dataset to study the glyph patterns. Consequently, we study local shape representations, both knowledge-driven and data-driven, over a set of frequent syllabic glyphs as well as other binary shapes, i.e. sketches. This comparative study indicates that a large data corpus and a deep network architecture are needed to learn data-driven representations that can capture the complex compositions of local patterns. To build a large glyph dataset in a short period of time, we study a crowdsourcing approach as an alternative to time-consuming data preparation of experts. Specifically, we work on individual glyph segmentation out of glyph-blocks from the three remaining codices (i.e. folded bark pages painted with a brush). With gradual steps in our crowdsourcing approach, we observe that providing supervision and careful task design are key aspects for non-experts to generate high-quality annotations. This way, we obtain a large dataset (over 9000) of individual Maya glyphs. We analyze this crowdsourced glyph dataset with both knowledge-driven and data-driven visual representations. First, we evaluate two competitive knowledge-driven representations, namely Histogram of Oriented Shape Context and Histogram of Oriented Gradients. Secondly, thanks to the large size of the crowdsourced dataset, we study visual representation learning with deep Convolutional Neural Networks. We adopt three data-driven approaches: assess- ing representations from pretrained networks, fine-tuning the last convolutional block of a pretrained network, and training a network from scratch. Finally, we investigate different glyph visualization tasks based on the studied representations. First, we explore the visual structure of several glyph corpora by applying a non-linear dimensionality reduction method, namely t-distributed Stochastic Neighborhood Embedding, Secondly, we propose a way to inspect the discriminative parts of individual glyphs according to the trained deep networks. For this purpose, we use the Gradient-weighted Class Activation Mapping method and highlight the network activations as a heatmap visualization over an input image. We assess whether the highlighted parts correspond to distinguishing parts of glyphs in a perceptual crowdsourcing study. Overall, this thesis presents a promising crowdsourcing approach, competitive data-driven visual representations, and interpretable visualization methods that can be applied to explore various other Digital Humanities datasets

4th International Workshop on Multimodal Affect and Aesthetic Experience

"Aesthetic experience"corresponds to the inner state of a person exposed to the form and content of artistic objects. Quantifying and interpreting the aesthetic experience of people in various contexts contribute towards a) creating context, and b) better understanding people's affective reactions to aesthetic stimuli. Focusing on different types of artistic content, such as movie, music, literature, urban art, ancient artwork, and modern interactive technology, the 4th international workshop on Multimodal Affect and Aesthetic Experience (MAAE) aims to enhance interdisciplinary collaboration among researchers from affective computing, aesthetics, human-robot/computer interaction, digital archaeology and art, culture, ethics, and addictive games

Maya Codical Glyph Segmentation: A Crowdsourcing Approach

This paper focuses on the crowd-annotation of an ancient Maya glyph dataset derived from the three ancient codices that survived up to date. More precisely, non-expert annotators are asked to segment glyph-blocks into their constituent glyph entities. As a means of supervision, available glyph variants are provided to the annotators during the crowdsourcing task. Compared to object recognition in natural images or handwriting transcription tasks, designing an engaging task and dealing with crowd behavior is challenging in our case. This challenge originates from the inherent complexity of Maya writing and an incomplete understanding of the signs and semantics in the existing catalogs. We elaborate on the evolution of the crowdsourcing task design, and discuss the choices for providing supervision during the task. We analyze the distributions of similarity and task difficulty scores, and the segmentation performance of the crowd. A unique dataset of over 9000 Maya glyphs from 291 categories individually segmented from the three codices was created and will be made publicly available thanks to this process. This dataset lends itself to automatic glyph classification tasks. We provide baseline methods for glyph classification using traditional shape descriptors and convolutional neural networks

Electronic Imaging & the Visual Arts. EVA 2013 Florence

Important Information Technology topics are presented: multimedia systems, data-bases, protection of data, access to the content. Particular reference is reserved to digital images (2D, 3D) regarding Cultural Institutions (Museums, Libraries, Palace – Monuments, Archaeological Sites). The main parts of the Conference Proceedings regard: Strategic Issues, EC Projects and Related Networks & Initiatives, International Forum on “Culture & Technology”, 2D – 3D Technologies & Applications, Virtual Galleries – Museums and Related Initiatives, Access to the Culture Information. Three Workshops are related to: International Cooperation, Innovation and Enterprise, Creative Industries and Cultural Tourism

European Avant-Garde: Art, Borders and Culture in Relationship to Mainstream Cinema and New Media

This research analyses the impact of transformation and hybridization processes at the intersection of art, science and technology. These forms of transformation and hybridization are the result of contemporary interactions between classic and digital media. It discusses the concept of 'remediation' presented by Bolter and proposes the concept of 'digital ekphrasis,' which is based on Manovich' s analyses of the interactions between classic and digital media. This is a model which, borrowed from semiotic structures, encompasses the technical as well as aesthetic and philosophical transformations of contemporary media. The thesis rejects Baudrillard's and Virilio's proposed concepts of 'digital black hole' as the only possible form of evolution of contemporary digital media. It proposes a different concept for the evolutionary model of contemporary hybridization processes based on contemporary forms of hybridizations that are rooted in aesthetic, philosophical and technological developments. This concept is argued as emancipated from the 'religious' idea of a 'divine originated' perfect image that Baudrillard and Virilio consider to be deteriorated from contemporary hybridization experimentation. The thesis proposes, through historical examples in the fine arts, the importance of transmedia migrations and experimentations as the framework for a philosophical, aesthetic and technological evolutionary concept of humanity freed from the restrictions of religious imperatives

Deity and Divine Agency in the Hebrew Bible: Cognitive Perspectives

This thesis interrogates the conceptualization of deity and divine agency in the Hebrew Bible, focusing particularly on the problem of the relationship of divine images and representatives to their patron deities. In order to move beyond the tendentiousness of previous scholarship that addresses this problem, I employ an interdisciplinary approach that will center cognitive linguistics and the cognitive science of religion, and also include biblical criticism, archaeology, anthropology, materiality studies, and other disciplines. I begin in Part One with a methodological discussion that describes the approaches being taken and interrogates some of the conceptual frameworks that have governed the previous scholarship on the question, such as “religion” and the practice of definition. It will then move on to discuss the concepts of agency and personhood, and how contemporary anthropological research on both can help inform our interrogation of the ancient world. Part Two begins the interrogation of the generic concept of deity, demonstrating that such concepts are products of the engagement of our intuitive and reflective reasoning with our cognitive ecologies, and that they build on our everyday conceptualizations of agency and personhood. These dynamics facilitate a view of divine agency as separable and communicable, which will be demonstrated to undergird the unique relationships understood to be shared by deities and their divine images. Chapter 4 employs a cognitive linguistic lens to propose semantic bases, domains, and profiles for the generic concept of deity in the Hebrew Bible. Part Three applies the models developed in Chapters 3 and 4 to an interrogation of YHWH as a deity and of YHWH’s divine agents, such as the ark of the covenant, the messenger of YHWH, and the very text of the Torah itself. The Conclusion summarizes findings and discusses implications for further research