Individuality and the collective in AI agents: Explorations of shared consciousness and digital homunculi in the metaverse for cultural heritage

The confluence of extended reality (XR) technologies, including augmented and virtual reality, with large language models (LLM) marks a significant advancement in the field of digital humanities, opening uncharted avenues for the representation of cultural heritage within the burgeoning metaverse. This paper undertakes an examination of the potentialities and intricacies of such a convergence, focusing particularly on the creation of digital homunculi or changelings. These virtual beings, remarkable for their sentience and individuality, are also part of a collective consciousness, a notion explored through a thematic comparison in science fiction with the Borg and the Changelings in the Star Trek universe. Such a comparison offers a metaphorical framework for discussing complex phenomena such as shared consciousness and individuality, illuminating their bearing on perceptions of self and awareness. Further, the paper considers the ethical implications of these concepts, including potential loss of individuality and the challenges inherent to accurate representation of historical figures and cultures. The latter necessitates collaboration with cultural experts, underscoring the intersectionality of technological innovation and cultural sensitivity. Ultimately, this chapter contributes to a deeper understanding of the technical aspects of integrating large language models with immersive technologies and situates these developments within a nuanced cultural and ethical discourse. By offering a comprehensive overview and proposing clear recommendations, the paper lays the groundwork for future research and development in the application of these technologies within the unique context of cultural heritage representation in the metaverse

Методологічні аспекти осмислення людини в культурологічній науковій парадигмі

The purpose of the work is to substantiate the methodological possibilities of cultural studies as post-non-classical knowledge in the study of anthropological problems, as well as to identify methodological approaches that correspond to the modern strategy of scientific comprehension of man. The methodology of the research is based on paradigm as a meta-methodology of cultural knowledge, the manifestation of which is discursive variability and interdisciplinary integrity. Structural-functional and phenomenological methods have become key ones to the study of this problem. To change the interpretation of stable semantic constants from positions of post-non-classical knowledge, methods of semiotic analysis and deconstruction are involved. The scientific novelty of the work lies in the definition and disclosure of the essence of methodological approaches, that reveal the specificity of human cognition in the context of the culturological scientific paradigm. Conclusions. Culturological comprehension of a person can be carried out from the standpoint of the following approaches - universalist, individualized and activity-oriented. The approaches identified in the research are an effective methodological tool for cultural science. Synthesis of the proposed approaches makes it possible to identify the relationship between the corresponding levels of development of culture and humanity, culture and man of a certain historical epoch, culture and personality.Цель работы заключается в обосновании методологических возможностей культурологии как постнекласичного знания в исследовании антропологической проблематики, а также выявлении методологических подходов, отвечающих современной стратегии научного осмысления человека. Методология исследования основана на парадигмальности как метаметодологии культурологического знания, проявлением которой является дискурсивная вариативность и междисциплинарная интегративность. Ключевыми для исследования данной проблематики стали структурно-функциональный и феноменологический методы. Для изменения трактовки устойчивых смысловых констант с позиций постнеклассического знания задействованы методы семиотического анализа и деконструкции. Научная новизна работы заключается в определении и раскрытии сущности методологических подходов, раскрывающих специфику познания человека в контексте культурологической научной парадигмы. Выводы. Культурологическое осмысление человека может осуществляться с позиций следующих подходов – универсалистского, индивидуализированного, деятельностного. Определенные в исследовании подходы выступают эффективным методологическим инструментарием культурологической науки. Синтез предложенных подходов позволяет выявить связь между соответствующими уровнями развития культуры и человечества, культуры и человека определенной исторической эпохи, культуры и личности.Мета роботи полягає в обґрунтуванні методологічних можливостей культурології як постнекласичного знання в дослідженні антропологічної проблематики, а також виявленні методологічних підходів, що відповідають сучасній стратегії наукового осмислення людини. Методологія дослідження ґрунтується на парадигмальності як метаметодології культурологічного знання, виявом якої є дискурсивна варіативність та міждисциплінарна інтегративність. Ключовими для дослідження даної проблематики стали структурно-функціональний та феноменологічний методи. Для перетрактування стійких смислових констант з позицій постнекласичного знання задіяні методи семіотичного аналізу та деконструкції. Наукова новизна роботи полягає у визначенні та розкритті сутності методологічних підходів, що розкривають специфіку пізнання людини в контексті культурологічної наукової парадигми. Висновки. Культурологічне осмислення людини може здійснюватися з позицій наступних підходів – універсалістського, індивідуалізованого, діяльнісного. Визначені підходи виступають ефективним методологічним інструментарієм культурологічної науки. Синтез запропонованих підходів дозволяє виявити зв’язок між відповідними рівнями розвитку культури та людства, культури та людини певної історичної епохи, культури та особистості

A Framework for Psychophysiological Classification within a Cultural Heritage Context Using Interest

This article presents a psychophysiological construct of interest as a knowledge emotion and illustrates the importance of interest detection in a cultural heritage context. The objective of this work is to measure and classify psychophysiological reactivity in response to cultural heritage material presented as visual and audio. We present a data processing and classification framework for the classification of interest. Two studies are reported, adopting a subject-dependent approach to classify psychophysiological signals using mobile physiological sensors and the support vector machine learning algorithm. The results show that it is possible to reliably infer a state of interest from cultural heritage material using psychophysiological feature data and a machine learning approach, informing future work for the development of a real-time physiological computing system for use within an adaptive cultural heritage experience designed to adapt the provision of information to sustain the interest of the visitor

An interdisciplinary design of an interactive cultural heritage visit for in-situ, mixed reality and affective experiences

Interactive technologies, such as mixed-reality and natural interactions with avatars, can enhance cultural heritage and the experience of visiting a museum. In this paper, we present the design rationale of an interactive experience for a cultural heritage place in the church of Roncesvalles at the beginning of Camino de Santiago. We followed a participatory design with a multidisciplinary team which resulted in the design of a spatial augmented reality system that employs 3D projection mapping and a conversational agent acting as the storyteller. Multiple features were identified as desirable for an interactive experience: interdisciplinary design team; in-situ; mixed reality; interactive digital storytelling; avatar; tangible objects; gestures; emotions and groups. The findings from a workshop are presented for guiding other interactive cultural heritage experiences. © 2022 by the authors.This research was funded by the R & D projects of the Government of Navarra under grant agreement No 0011-1365-2021-000063

A survey of cultural aspects in Human Computer Interaction Research

Considering culture in human computer interaction research is an important issue since culture has a strong impact on many cognitive and affective processes, closed related to the design and evaluation of interactive systems. Also, people with different cultural backgrounds develop alternative interpretations and strategies and do not value their environment in the same way, and this reflects in their interactions and satisfaction with interactive technologies. In this survey we summarize some concepts of the cultural aspects related to human computer interaction research. After we discuss how HCI practices could address these cultural issues. Our intention is to establish background and some basic concepts for helping researchers incorporating cultural issues in their design and evaluation processes

Mapping the Current Landscape of Research Library Engagement with Emerging Technologies in Research and Learning: Final Report

The generation, dissemination, and analysis of digital information is a significant driver, and consequence, of technological change. As data and information stewards in physical and virtual space, research libraries are thoroughly entangled in the challenges presented by the Fourth Industrial Revolution:1 a societal shift powered not by steam or electricity, but by data, and characterized by a fusion of the physical and digital worlds.2 Organizing, structuring, preserving, and providing access to growing volumes of the digital data generated and required by research and industry will become a critically important function. As partners with the community of researchers and scholars, research libraries are also recognizing and adapting to the consequences of technological change in the practices of scholarship and scholarly communication. Technologies that have emerged or become ubiquitous within the last decade have accelerated information production and have catalyzed profound changes in the ways scholars, students, and the general public create and engage with information. The production of an unprecedented volume and diversity of digital artifacts, the proliferation of machine learning (ML) technologies,3 and the emergence of data as the “world’s most valuable resource,”4 among other trends, present compelling opportunities for research libraries to contribute in new and significant ways to the research and learning enterprise. Librarians are all too familiar with predictions of the research library’s demise in an era when researchers have so much information at their fingertips. A growing body of evidence provides a resounding counterpoint: that the skills, experience, and values of librarians, and the persistence of libraries as an institution, will become more important than ever as researchers contend with the data deluge and the ephemerality and fragility of much digital content. This report identifies strategic opportunities for research libraries to adopt and engage with emerging technologies,5 with a roughly fiveyear time horizon. It considers the ways in which research library values and professional expertise inform and shape this engagement, the ways library and library worker roles will be reconceptualized, and the implication of a range of technologies on how the library fulfills its mission. The report builds on a literature review covering the last five years of published scholarship, primarily North American information science literature, and interviews with a dozen library field experts, completed in fall 2019. It begins with a discussion of four cross-cutting opportunities that permeate many or all aspects of research library services. Next, specific opportunities are identified in each of five core research library service areas: facilitating information discovery, stewarding the scholarly and cultural record, advancing digital scholarship, furthering student learning and success, and creating learning and collaboration spaces. Each section identifies key technologies shaping user behaviors and library services, and highlights exemplary initiatives. Underlying much of the discussion in this report is the idea that “digital transformation is increasingly about change management”6 —that adoption of or engagement with emerging technologies must be part of a broader strategy for organizational change, for “moving emerging work from the periphery to the core,”7 and a broader shift in conceptualizing the research library and its services. Above all, libraries are benefitting from the ways in which emerging technologies offer opportunities to center users and move from a centralized and often siloed service model to embedded, collaborative engagement with the research and learning enterprise

Interactive Experience Design: Integrated and Tangible Storytelling with Maritime Museum Artefacts

Museums play the role of intermediary between cultural heritage and visitors, and are often described as places and environments for education and enjoyment. The European Union also encourages innovative uses of museums to support education through the cultural heritage resources. However, the importance of visitors’ active role in museums as places for education and entertainment, on the one hand, and the growing and indispensable presence of technology in the cultural heritage domain, on the other hand, provided the initial ideas to develop the research. This thesis, presents the study and design for an interactive storytelling installation for a maritime museum. The installation is designed to integrate different museum artefacts into the storytelling system to enrich the visitors experience through tangible storytelling. The project was conducted in collaboration with another PhD student, Luca Ciotoli. His contribution was mainly focused on the narrative and storytelling features of the research, while my contribution was focused on the interaction- and technology-related features, including the design and implementation of the prototype. The research is deployed using a four-phase iterative approach. The first phase of the research, Study, deals with literature review and different studies to identify the requirements. The second phase, Design, determines the broad outlines of the project i.e., an interactive storytelling installation. The design phase includes interaction and museum experience design. We investigated different design approaches, e.g., interaction and museum experience design, to develop a conceptual design. The third phase, prototype, allows us to determine how to fulfill the tasks and meet the requirements that are established for the research. Prototyping involves content creation, storyboarding, integrating augmented artefacts into the storytelling system. Th final phase, test, refers to the evaluations that are conducted during the aforementioned phases e.g., formative and the final usability testing with users. The outcome of the research confirms previous results in the literature about how digital narratives can be enriched with the tangible dimension, moreover it shows how this dimension can enable to communicate stories and knowledge of the past that are complex, such as the art of navigating in the past, by integrating tangible objects that play different roles in the storytelling process

A User-Centered Perspective on Information Technologies in Museums

Information Technology (IT) has been put forth as a reasonable way to sustain visitor interest and encourage visit repetition in museums. Therefore, IT is becoming more common in museum settings and professionals express their need for more information about how their visitors interact with these systems. This dissertation is an attempt to answer this call. We propose three essays that deal with different aspects of museums and IT from a user-centered perspective. The first essay is an attempt to determine with a free simulation experiment how IT and more particularly websites can arouse interest for museological content. The second essay relies on a field study to analyze the influence of IT on affective and cognitive reactions during a museum visit, namely perceived enjoyment, perceived authenticity and learning. In the third essay, we use focus groups and questionnaires to explore visitor expectations towards a phenomenological experience and the role played by IT in visitor experience of the past. This dissertation contributes to research by (1) advancing our knowledge of IT dedicated to the cultural heritage area, and (2) identifying and understanding visitor perceptions of hedonic systems. By proposing a set of key dimensions that could be used for IT evaluation in the cultural heritage, this dissertation also offers actionable advices to museum professionals

Semantically-enhanced recommendations in cultural heritage

In the Web 2.0 environment, institutes and organizations are starting to open up their previously isolated and heterogeneous collections in order to provide visitors with maximal access. Semantic Web technologies act as instrumental in integrating these rich collections of metadata by defining ontologies which accommodate different representation schemata and inconsistent naming conventions over the various vocabularies. Facing the large amount of metadata with complex semantic structures, it is becoming more and more important to support visitors with a proper selection and presentation of information. In this context, the Dutch Science Foundation (NWO) funded the Cultural Heritage Information Personalization (CHIP) project in early 2005, as part of the Continuous Access to Cultural Heritage (CATCH) program in the Netherlands. It is a collaborative project between the Rijksmuseum Amsterdam, the Eindhoven University of Technology and the Telematica Instituut. The problem statement that guides the research of this thesis is as follows: Can we support visitors with personalized access to semantically-enriched collections? To study this question, we chose cultural heritage (museums) as an application domain, and the semantically rich background knowledge about the museum collection provides a basis to our research. On top of it, we deployed user modeling and recommendation technologies in order to provide personalized services for museum visitors. Our main contributions are: (i) we developed an interactive rating dialog of artworks and art concepts for a quick instantiation of the CHIP user model, which is built as a specialization of FOAF and mapped to an existing event model ontology SEM; (ii) we proposed a hybrid recommendation algorithm, combining both explicit and implicit relations from the semantic structure of the collection. On the presentation level, we developed three tools for end-users: Art Recommender, Tour Wizard and Mobile Tour Guide. Following a user-centered design cycle, we performed a series of evaluations with museum visitors to test the effectiveness of recommendations using the rating dialog, different ways to build an optimal user model and the prediction accuracy of the hybrid algorithm. Chapter 1 introduces the research questions, our approaches and the outline of this thesis. Chapter 2 gives an overview of our work at the first stage. It includes (i) the semantic enrichment of the Rijksmuseum collection, which is mapped to three Getty vocabularies (ULAN, AAT, TGN) and the Iconclass thesaurus; (ii) the minimal user model ontology defined as a specialization of FOAF, which only stores user ratings at that time, (iii) the first implementation of the content-based recommendation algorithm in our first tool, the CHIP Art Recommender. Chapter 3 presents two other tools: Tour Wizard and Mobile Tour Guide. Based on the user's ratings, the Web-based Tour Wizard recommends museum tours consisting of recommended artworks that are currently available for museum exhibitions. The Mobile Tour Guide converts recommended tours to mobile devices (e.g. PDA) that can be used in the physical museum space. To connect users' various interactions with these tools, we made a conversion of the online user model stored in RDF into XML format which the mobile guide can parse, and in this way we keep the online and on-site user models dynamically synchronized. Chapter 4 presents the second generation of the Mobile Tour Guide with a real time routing system on different mobile devices (e.g. iPod). Compared with the first generation, it can adapt museum tours based on the user's ratings artworks and concepts, her/his current location in the physical museum and the coordinates of the artworks and rooms in the museum. In addition, we mapped the CHIP user model to an existing event model ontology SEM. Besides ratings, it can store additional user activities, such as following a tour and viewing artworks. Chapter 5 identifies a number of semantic relations within one vocabulary (e.g. a concept has a broader/narrower concept) and across multiple vocabularies (e.g. an artist is associated to an art style). We applied all these relations as well as the basic artwork features in content-based recommendations and compared all of them in terms of usefulness. This investigation also enables us to look at the combined use of artwork features and semantic relations in sequence and derive user navigation patterns. Chapter 6 defines the task of personalized recommendations and decomposes the task into a number of inference steps for ontology-based recommender systems, from a perspective of knowledge engineering. We proposed a hybrid approach combining both explicit and implicit recommendations. The explicit relations include artworks features and semantic relations with preliminary weights which are derived from the evaluation in Chapter 5. The implicit relations are built between art concepts based on instance-based ontology matching. Chapter 7 gives an example of reusing user interaction data generated by one application into another one for providing cross-application recommendations. In this example, user tagging about cultural events, gathered by iCITY, is used to enrich the user model for generating content-based recommendations in the CHIP Art Recommender. To realize full tagging interoperability, we investigated the problems that arise in mapping user tags to domain ontologies, and proposed additional mechanisms, such as the use of SKOS matching operators to deal with the possible mis-alignment of tags and domain-specific ontologies. We summarized to what extent the problem statement and each of the research questions are answered in Chapter 8. We also discussed a number of limitations in our research and looked ahead at what may follow as future work