Summarization from Multiple User Generated Videos in Geo-Space

Ph.DDOCTOR OF PHILOSOPH

CICHMKG: a large-scale and comprehensive Chinese intangible cultural heritage multimodal knowledge graph

Intangible Cultural Heritage (ICH) witnesses human creativity and wisdom in long histories, composed of a variety of immaterial manifestations. The rapid development of digital technologies accelerates the record of ICH, generating a sheer number of heterogenous data but in a state of fragmentation. To resolve that, existing studies mainly adopt approaches of knowledge graphs (KGs) which can provide rich knowledge representation. However, most KGs are text-based and text-derived, and incapable to give related images and empower downstream multimodal tasks, which is also unbeneficial for the public to establish the visual perception and comprehend ICH completely especially when they do not have the related ICH knowledge. Hence, aimed at that, we propose to, taking the Chinese nation-level ICH list as an example, construct a large-scale and comprehensive Multimodal Knowledge Graph (CICHMKG) combining text and image entities from multiple data sources and give a practical construction framework. Additionally, in this paper, to select representative images for ICH entities, we propose a method composed of the denoising algorithm (CNIFA) and a series of criteria, utilizing global and local visual features of images and textual features of captions. Extensive empirical experiments demonstrate its effectiveness. Lastly, we construct the CICHMKG, consisting of 1,774,005 triples, and visualize it to facilitate the interactions and help the public dive into ICH deeply

SPATIAL SENSOR DATA PROCESSING AND ANALYSIS FOR MOBILE MEDIA APPLICATIONS

Ph.DDOCTOR OF PHILOSOPH

GEO-REFERENCED VIDEO RETRIEVAL: TEXT ANNOTATION AND SIMILARITY SEARCH

Ph.DDOCTOR OF PHILOSOPH

Confluence of Vision and Natural Language Processing for Cross-media Semantic Relations Extraction

In this dissertation, we focus on extracting and understanding semantically meaningful relationships between data items of various modalities; especially relations between images and natural language. We explore the ideas and techniques to integrate such cross-media semantic relations for machine understanding of large heterogeneous datasets, made available through the expansion of the World Wide Web. The datasets collected from social media websites, news media outlets and blogging platforms usually contain multiple modalities of data. Intelligent systems are needed to automatically make sense out of these datasets and present them in such a way that humans can find the relevant pieces of information or get a summary of the available material. Such systems have to process multiple modalities of data such as images, text, linguistic features, and structured data in reference to each other. For example, image and video search and retrieval engines are required to understand the relations between visual and textual data so that they can provide relevant answers in the form of images and videos to the users\u27 queries presented in the form of text. We emphasize the automatic extraction of semantic topics or concepts from the data available in any form such as images, free-flowing text or metadata. These semantic concepts/topics become the basis of semantic relations across heterogeneous data types, e.g., visual and textual data. A classic problem involving image-text relations is the automatic generation of textual descriptions of images. This problem is the main focus of our work. In many cases, large amount of text is associated with images. Deep exploration of linguistic features of such text is required to fully utilize the semantic information encoded in it. A news dataset involving images and news articles is an example of this scenario. We devise frameworks for automatic news image description generation based on the semantic relations of images, as well as semantic understanding of linguistic features of the news articles

Efficient location-based spatial keyword query processing

Ph.DDOCTOR OF PHILOSOPH

THE DEVELOPMENT OF GUIDELINES FOR DESIGNING DIGITAL MEDIA TO ENGAGE VISITORS WITH NON-VISIBLE OUTDOOR HERITAGE

This PhD investigates the role of digital media in optimising visitor engagement with non-visible outdoor heritage. Motivated by concerns that digital media products developed for the heritage sector might not be reaching their potential to enrich the visit experience and concerned about a lack of clarity as to what constitutes visitor engagement; this thesis proposes guidance for the production of interpretive digital media and a framework for visitor engagement. Cultural heritage sites featured in this study are characteristically outdoor locations; frequently non-stewarded with very little tangible evidence of the historical or cultural relevance of the site. The unique potential of digital media products to address the specific challenges of engaging visitors with invisible heritage in these locations is discussed within this thesis. The practice of interpreting heritage is investigated to identify the processes, stages, experiences and behavioural states associated with a high level of engagement. Visitor engagement is defined in this study as being a transformational experience in which the visitor’s emotional and/or cognitive relationship with the heritage is altered. This is achieved when the visitor sufficiently experiences appropriate states of engagement across all stages of the visitor engagement framework. This study proposes guidance to advise and support heritage professionals and their associated designers in the design, development and implementation of interpretive digital media products. Within this guide sits the engagement framework which proposes a framework for engagement, defining the stages (process) and the states (experiences and behaviours) of visitor engagement with cultural heritage. In using this resource the cultural heritage practitioner can be confident of their capacity to run and deliver interpretive digital media projects regardless of their expertise in design or technology. This thesis proposes that well designed interpretive digital media can optimise the engagement of visitors in ways which cannot be achieved by any other single method of interpretation. This PhD contributes a design guide and an engagement framework to the existing field of knowledge regarding interpretive digital design

Popular culture as a powerful destination marketing tool: an Australian study

This thesis is concerned with the marketing possibilities of popular culture tourism (PCT). There is promise in developing alternative forms of cultural communication and cultural representation in tourism. Specifically, PCT is explored as a way to enhance and reshape the current approach to destination promotion in Australia. Through the arrival of new and diversified cultural experiences, Australia can improve the existing tourism portfolio. Although there have been many studies which describe the marketing practice of using elements of popular culture in destination promotion, few, if any, effectively address the issue of usability of such methods in Australia. To understand the nature of the challenge, it is important to acknowledge the diversity inherent within popular culture, as well as the huge diversity of individual experiences and responses to such cultural practices. This work is concerned with the richness of individual experience, the multi-form qualities of interpersonal encounters with popular culture in Australia. This thesis uses pragmatism as the main interpretive framework, with its powerful ability to disambiguate and clarify the research questions. To study the phenomenon the researcher uses a combination of three research methods: survey research, descriptive study, and exploratory study. Each study contributes a unique perspective to the literature on popular culture tourism. To answer the research questions considerable data comprising 253 detailed questionnaires, 20 unstructured interviews, 648 blogs and social media posts, and marketing materials of over 50 DMOs were collected and analysed. The thesis has six chapters in total. The first chapter introduces the concept of PCT. It discusses how popular media and tourism, and thoughtful engagement of these forces, have created a phenomenon with great potential and strong commercial and popular impact. PCT is an umbrella term comprising several fields, such as film-induced, literary, and music tourism, as well as special events, and technology tourism, among others. The chapter argues that PCT can encourage youth tourism and help accommodate the needs of tourists coming from diverse households and families (multi-generational groups, singles, 'second' families). The chapter highlights the need to diversify the traditional tourism product by embracing specialty markets. The second chapter outlines the theoretical framework, rationale, and conceptual structure for the materials to follow. The third chapter introduces Study 1. The first study uses survey data to uncovers behaviour patterns and preferences of local popular culture tourists. It compares the events and locations in the context of PCT, and works with important cues (e.g., associations and preferences) and key features (e.g., consumption rates and travel intentions) by matching them with several hypotheses related to the consumption of popular culture. In Chapter 4, the scope of the investigation widens to include the international perspective. Study 2 is concerned with qualitative aspects of the cultural economy, namely the subjective experiences and expectation of past, existing and potential visitors. This study employs social listening and content analysis to observe and analyse online discussions related to popular culture events and locations in Australia. The captured experiences (impressions, feelings, thoughts, and observations) helped: (1) identify how Australia is being represented in popular culture discourse; (2) identify how the particular imagery of local popular culture commodities can influence the Australian tourism development strategy. The last study, Chapter 5, is concerned with practical applications. It offers a rigorous analysis of the marketing strategies that utilise popular culture in destination promotion. It discusses how these integrations are carried out by the DMOs in real-world practices. The chapter identifies seven advanced destination marketing tactics as efficient methods that can be used for tourism promotion in Australia. It offers recommendations and comments on the use of PCT in national tourism campaigns. Chapter 6 is devoted to the discussion of findings, implications, and limitations. The key findings contribute to the academic literature on cultural tourism. This thesis investigates the possibilities of using location-specific popular culture tools in 'narrative' marketing campaigns. The work identifies different PCT activities and their impacts on destination's image and tourists' experiences. The results and work also emerge as practical solutions for implementation of PCT tools in destination promotion for Australia

Reinterpreting Pioneer Deep Space Station

This project argues for the interpretation of NASA’s historic space probes. Robotic space exploration programs have permanently transformed humans’ understanding of Earth, but the probes at the center of NASA’s first pioneering missions to deep space in the 1960s and 1970s will never return to their origin. Today, the historical significance of these objects is embodied in a network of resources that reflect triumphs of human curiosity, not just advancements in technology. The widely unrecognized contributions that women and men made to the history of “unmanned” space exploration reveal themselves on Earth through the enduring infrastructure that humans built and the data that humans rendered in their pursuit to understand the Solar System. After a review of the existing literature on outer space preservation, I outline the historical significance of space probes and establish interpretive themes designed to articulate the perspective that probes help generate, the human ingenuity underlying the development and operation of probes, and the impact of discoveries that robotic space exploration has enabled. I argue that by relying on the data returned to Earth by space probes and by considering a significant site at which these data were historically received, space probes can be meaningfully interpreted for public audiences. The project concludes with a proposal for reinterpreting the Pioneer Deep Space Station radio antenna site at Goldstone Deep Space Communications Complex, based on analysis of current interpretations and effective case study methods

Searching and mining in enriched geo-spatial data

The emergence of new data collection mechanisms in geo-spatial applications paired with a heightened tendency of users to volunteer information provides an ever-increasing flow of data of high volume, complex nature, and often associated with inherent uncertainty. Such mechanisms include crowdsourcing, automated knowledge inference, tracking, and social media data repositories. Such data bearing additional information from multiple sources like probability distributions, text or numerical attributes, social context, or multimedia content can be called multi-enriched. Searching and mining this abundance of information holds many challenges, if all of the data's potential is to be released. This thesis addresses several major issues arising in that field, namely path queries using multi-enriched data, trend mining in social media data, and handling uncertainty in geo-spatial data. In all cases, the developed methods have made significant contributions and have appeared in or were accepted into various renowned international peer-reviewed venues. A common use of geo-spatial data is path queries in road networks where traditional methods optimise results based on absolute and ofttimes singular metrics, i.e., finding the shortest paths based on distance or the best trade-off between distance and travel time. Integrating additional aspects like qualitative or social data by enriching the data model with knowledge derived from sources as mentioned above allows for queries that can be issued to fit a broader scope of needs or preferences. This thesis presents two implementations of incorporating multi-enriched data into road networks. In one case, a range of qualitative data sources is evaluated to gain knowledge about user preferences which is subsequently matched with locations represented in a road network and integrated into its components. Several methods are presented for highly customisable path queries that incorporate a wide spectrum of data. In a second case, a framework is described for resource distribution with reappearance in road networks to serve one or more clients, resulting in paths that provide maximum gain based on a probabilistic evaluation of available resources. Applications for this include finding parking spots. Social media trends are an emerging research area giving insight in user sentiment and important topics. Such trends consist of bursts of messages concerning a certain topic within a time frame, significantly deviating from the average appearance frequency of the same topic. By investigating the dissemination of such trends in space and time, this thesis presents methods to classify trend archetypes to predict future dissemination of a trend. Processing and querying uncertain data is particularly demanding given the additional knowledge required to yield results with probabilistic guarantees. Since such knowledge is not always available and queries are not easily scaled to larger datasets due to the #P-complete nature of the problem, many existing approaches reduce the data to a deterministic representation of its underlying model to eliminate uncertainty. However, data uncertainty can also provide valuable insight into the nature of the data that cannot be represented in a deterministic manner. This thesis presents techniques for clustering uncertain data as well as query processing, that take the additional information from uncertainty models into account while preserving scalability using a sampling-based approach, while previous approaches could only provide one of the two. The given solutions enable the application of various existing clustering techniques or query types to a framework that manages the uncertainty.Das Erscheinen neuer Methoden zur Datenerhebung in räumlichen Applikationen gepaart mit einer erhöhten Bereitschaft der Nutzer, Daten über sich preiszugeben, generiert einen stetig steigenden Fluss von Daten in großer Menge, komplexer Natur, und oft gepaart mit inhärenter Unsicherheit. Beispiele für solche Mechanismen sind Crowdsourcing, automatisierte Wissensinferenz, Tracking, und Daten aus sozialen Medien. Derartige Daten, angereichert mit mit zusätzlichen Informationen aus verschiedenen Quellen wie Wahrscheinlichkeitsverteilungen, Text- oder numerische Attribute, sozialem Kontext, oder Multimediainhalten, werden als multi-enriched bezeichnet. Suche und Datamining in dieser weiten Datenmenge hält viele Herausforderungen bereit, wenn das gesamte Potenzial der Daten genutzt werden soll. Diese Arbeit geht auf mehrere große Fragestellungen in diesem Feld ein, insbesondere Pfadanfragen in multi-enriched Daten, Trend-mining in Daten aus sozialen Netzwerken, und die Beherrschung von Unsicherheit in räumlichen Daten. In all diesen Fällen haben die entwickelten Methoden signifikante Forschungsbeiträge geleistet und wurden veröffentlicht oder angenommen zu diversen renommierten internationalen, von Experten begutachteten Konferenzen und Journals. Ein gängiges Anwendungsgebiet räumlicher Daten sind Pfadanfragen in Straßennetzwerken, wo traditionelle Methoden die Resultate anhand absoluter und oft auch singulärer Maße optimieren, d.h., der kürzeste Pfad in Bezug auf die Distanz oder der beste Kompromiss zwischen Distanz und Reisezeit. Durch die Integration zusätzlicher Aspekte wie qualitativer Daten oder Daten aus sozialen Netzwerken als Anreicherung des Datenmodells mit aus diesen Quellen abgeleitetem Wissen werden Anfragen möglich, die ein breiteres Spektrum an Anforderungen oder Präferenzen erfüllen. Diese Arbeit präsentiert zwei Ansätze, solche multi-enriched Daten in Straßennetze einzufügen. Zum einen wird eine Reihe qualitativer Datenquellen ausgewertet, um Wissen über Nutzerpräferenzen zu generieren, welches darauf mit Örtlichkeiten im Straßennetz abgeglichen und in das Netz integriert wird. Diverse Methoden werden präsentiert, die stark personalisierbare Pfadanfragen ermöglichen, die ein weites Spektrum an Daten mit einbeziehen. Im zweiten Fall wird ein Framework präsentiert, das eine Ressourcenverteilung im Straßennetzwerk modelliert, bei der einmal verbrauchte Ressourcen erneut auftauchen können. Resultierende Pfade ergeben einen maximalen Ertrag basieren auf einer probabilistischen Evaluation der verfügbaren Ressourcen. Eine Anwendung ist die Suche nach Parkplätzen. Trends in sozialen Medien sind ein entstehendes Forscchungsgebiet, das Einblicke in Benutzerverhalten und wichtige Themen zulässt. Solche Trends bestehen aus großen Mengen an Nachrichten zu einem bestimmten Thema innerhalb eines Zeitfensters, so dass die Auftrittsfrequenz signifikant über den durchschnittlichen Level liegt. Durch die Untersuchung der Fortpflanzung solcher Trends in Raum und Zeit präsentiert diese Arbeit Methoden, um Trends nach Archetypen zu klassifizieren und ihren zukünftigen Weg vorherzusagen. Die Anfragebearbeitung und Datamining in unsicheren Daten ist besonders herausfordernd, insbesondere im Hinblick auf das notwendige Zusatzwissen, um Resultate mit probabilistischen Garantien zu erzielen. Solches Wissen ist nicht immer verfügbar und Anfragen lassen sich aufgrund der \P-Vollständigkeit des Problems nicht ohne Weiteres auf größere Datensätze skalieren. Dennoch kann Datenunsicherheit wertvollen Einblick in die Struktur der Daten liefern, der mit deterministischen Methoden nicht erreichbar wäre. Diese Arbeit präsentiert Techniken zum Clustering unsicherer Daten sowie zur Anfragebearbeitung, die die Zusatzinformation aus dem Unsicherheitsmodell in Betracht ziehen, jedoch gleichzeitig die Skalierbarkeit des Ansatzes auf große Datenmengen sicherstellen