Fireground location understanding by semantic linking of visual objects and building information models

This paper presents an outline for improved localization and situational awareness in fire emergency situations based on semantic technology and computer vision techniques. The novelty of our methodology lies in the semantic linking of video object recognition results from visual and thermal cameras with Building Information Models (BIM). The current limitations and possibilities of certain building information streams in the context of fire safety or fire incident management are addressed in this paper. Furthermore, our data management tools match higher-level semantic metadata descriptors of BIM and deep-learning based visual object recognition and classification networks. Based on these matches, estimations can be generated of camera, objects and event positions in the BIM model, transforming it from a static source of information into a rich, dynamic data provider. Previous work has already investigated the possibilities to link BIM and low-cost point sensors for fireground understanding, but these approaches did not take into account the benefits of video analysis and recent developments in semantics and feature learning research. Finally, the strengths of the proposed approach compared to the state-of-the-art is its (semi -)automatic workflow, generic and modular setup and multi-modal strategy, which allows to automatically create situational awareness, to improve localization and to facilitate the overall fire understanding

Modellgetriebene Entwicklung inhaltsbasierter Bildretrieval-Systeme auf der Basis von objektrelationalen Datenbank-Management-Systeme

In this thesis, the model-driven software development paradigm is employed in order to support the development of Content-based Image Retrieval Systems (CBIRS) for different application domains. Modeling techniques, based on an adaptable conceptual framework model, are proposed for deriving the components of a concrete CBIRS. Transformation techniques are defined to automatically implement the derived application specific models in an object-relational database management system. A set of criteria assuring the quality of the transformation are derived from the theory for preserving information capacity applied in database design.In dieser Dissertation wird das Paradigma des modellgetriebenen Softwareentwurfs für die Erstellung von inhaltsbasierten Bildretrieval-Systemen verwendet. Ein adaptierbares Frameworkmodell wird für die Ableitung des Modells eines konkreten Bildretrieval-Systems eingesetzt. Transformationstechniken für die automatische Generierung von Implementierungen in Objektorientierten Datenbank-Management-Systemen aus dem konzeptuellen Modell werden erarbeitet. Die aus der Theorie des Datenbankentwurfs bekannten Anforderungen zur Kapazitätserhaltung der Transformation werden verwendet, um Kriterien für die erforderliche Qualität der Transformation zu definieren

Development of a MPEG-7 based multimedia content description and retrieval tool for internet protocol television (IPTV)

Search and retrieval of multimedia content from open platforms such as the Internet and IPTV platforms has long been found to be hugely inefficient. It has been noted that a major cause of such inefficient results is the improper labeling or incomplete description of multimedia content by its creators. The lack of adequate description of video content by the proper annotation of video content with the relevant metadata leads to poor search and retrieval yields. The creation of such metadata itself is a major problem as there are various metadata description standards which users could employ. On the other hand there are tools such as FFprobe that can retrieve important features of video that can be used in searching and retrieval. The combination of such tools and metadata description standards could be the solution to the metadata problem. The Multimedia Content Description Interface (MPEG-7) is an example of a metadata description standard. It has been adopted by TISPAN for the description of IPTV multimedia content. The MPEG-7 standard is rather complex, seeing as it has over 1200 global Descriptors and Description Schemes which a user would have to know in order to implement such technology. This complexity is a nuisance when we consider the existence of multitudes of amateur video producers. These multimedia content creators have no idea how to use the MPEG-7 standard to annotate their creations with metadata. Consequently, overloading of the IPTV platform with content that has not been annotated in a standardized manner occurs, making search and retrieval of the multimedia content (videos, in this instance) inefficient. Therefore, it was imperative to try and determine whether the use of the MPEG-7 standard could be made much easier by creating a tool that is MPEG-7 enabled which will allow for the annotation of video content by any user without concerning themselves about how to use the MPEG-7 standard. In attempting to develop a tool for metadata generation, it was incumbent for us to understand the issues associated with metadata generation for users wishing to create IPTV services. An extensive literature review on IPTV standardization was carried out to determine the issues associated with metadata generation for IPTV and their proposed solutions. An experimental research approach was taken in an attempt to figure out if our proposed solution to the lack of technical expertise by users about the MPEG-7 standard could be the final solution to the metadata generation problem. We developed a Multimedia Content Description and Management System (MCDMS) prototype which enabled us to describe video content by annotating it with 16 different metadata elements and storing the descriptions in XML MPEG-7 format. Incremental development and re-use oriented development were used during the development phase of this research. The MCDMS underwent functional testing; smoke testing of the individual system components and Big Bang integration testing for the combined components. Our results indicate that the more metadata is appended to a video as description the better it is to search for and retrieve. The MCDMS hides the complexity of MPEG-7 metadata creation from the users. With the effortless creation of MPEG-7 based metadata, it becomes easier to annotate videos. Consequently, search and retrieval of video content becomes more efficient. It is important to note that the description of multimedia content remains a complex feat. Even with the metadata elements laid out for users, there still exist other issues that affect metadata creation such as polysemy and the semantic gap. However, the provision of a tool that does the MPEG-7 standardizing behind the scenes for users when they upload a video makes the description of multimedia content in a standardized manner a much easier feat to achieve

Ontologiebasierte Indexierung und Kontextualisierung multimedialer Dokumente für das persönliche Wissensmanagement

Die Verwaltung persönlicher, multimedialer Dokumente kann mit Hilfe semantischer Technologien und Ontologien intelligent und effektiv unterstützt werden. Dies setzt jedoch Verfahren voraus, die den grundlegenden Annotations- und Bearbeitungsaufwand für den Anwender minimieren und dabei eine ausreichende Datenqualität und -konsistenz sicherstellen. Im Rahmen der Dissertation wurden notwendige Mechanismen zur semi-automatischen Modellierung und Wartung semantischer Dokumentenbeschreibungen spezifiziert. Diese bildeten die Grundlage für den Entwurf einer komponentenbasierten, anwendungsunabhängigen Architektur als Basis für die Entwicklung innovativer, semantikbasierter Lösungen zur persönlichen Dokumenten- und Wissensverwaltung.Personal multimedia document management benefits from Semantic Web technologies and the application of ontologies. However, an ontology-based document management system has to meet a number of challenges regarding flexibility, soundness, and controllability of the semantic data model. The first part of the dissertation proposes necessary mechanisms for the semi-automatic modeling and maintenance of semantic document descriptions. The second part introduces a component-based, application-independent architecture which forms the basis for the development of innovative, semantic-driven solutions for personal document and information management

A component framework for personalized multimedia applications

Eine praktikable Unterstützung für eine dynamische Erstellung von personalisierten Multimedia-Präsentationen bieten bisher weder industrielle Lösungen noch Forschungsansätze. Mit dem Software-technischen Ansatz des MM4U-Frameworks („MultiMedia For You“) wird erstmals eine generische und zugleich praktikable Unterstützung für den dynamischen Erstellungsprozess bereitgestellt. Das Ziel des MM4U-Frameworks ist es den Anwendungsentwicklern eine umfangreiche und anwendungsunabhängige Unterstützung zur Erstellung von personalisierten Multimedia-Inhalten anzubieten und damit den Entwicklungsprozess solcher Anwendungen erheblich zu erleichtern. Um das Ziel eines Software-Frameworks zur generischen Unterstützung der Entwicklung von personalisierten Multimedia-Anwendungen zu erreichen, stellt sich die Frage nach einer geeigneten Software-technischen Unterstützung zur Entwicklung eines solchen Frameworks. Seit der Einführung von objektorientierten Frameworks, ist heute die Entwicklung immer noch aufwendig und schwierig. Um die Entwicklungsrisiken zu reduzieren, sind geeignete Vorgehensmodelle und Entwicklungsmethoden erstellt worden. Mit der Komponenten-Technologie sind auch so genannte Komponenten-Frameworks entstanden. Im Gegensatz zu objekt-orientierten Frameworks fehlt derzeit jedoch ein geeignetes Vorgehensmodell für Komponenten-Frameworks. Um den Entwicklungsprozess von Komponenten-Frameworks zu verbessern ist mit ProMoCF („Process Model for Component Frameworks“) ein neuartiger Ansatz entwickelt worden. Hierbei handelt es sich um ein leichtgewichtiges Vorgehensmodell und eine Entwicklungsmethodik für Komponenten-Frameworks. Das Vorgehensmodell wurde unter gegenseitigem Nutzen mit der Entwicklung des MM4U-Frameworks erstellt. Das MM4U-Framework stellt keine Neuerfindung der Adaption von Multimedia-Inhalten dar, sondern zielt auf die Vereinigung und Einbettung existierender Forschungsansätze und Lösungen im Umfeld der Multimedia-Personalisierung. Mit so einem Framework an der Hand können Anwendungsentwickler erstmals effizient und einfach eine dynamische Erstellung ihrer personalisierten Multimedia-Inhalte realisieren

Earth Resources, A Continuing Bibliography with Indexes

This bibliography lists 460 reports, articles and other documents introduced into the NASA scientific and technical information system between July 1 and September 30, 1984. Emphasis is placed on the use of remote sensing and geophysical instrumentation in spacecraft and aircraft to survey and inventory natural resources and urban areas. Subject matter is grouped according to agriculture and forestry, environmental changes and cultural resources, geodesy and cartography, geology and mineral resources, hydrology and water management, data processing and distribution systems, instrumentation and sensors, and economical analysis

A DOMAIN-CENTRIC APPROACH TO DESIGNING USER INTERFACES OF VIDEO RETRIEVAL SYSTEMS

Thesis (PhD) - Indiana University, Information Science, 2007User- and task-centric efforts in video information retrieval (IR) research are needed because current experiments are showing few significant results. It is our belief that unsatisfactory results in video IR can be partially attributed to the overemphasis on technologically-driven approaches to interface development and system evaluation. This study explored variables that have been consistently overlooked in video retrieval efforts, including those related to domain and search tasks. The underlying goal of this study is to promote alternative means for evaluating video retrieval systems, and to make progress toward developing new design principles and a video seeking model. A series of interactive search runs were conducted using a video retrieval system called ViewFinder. ViewFinder was implemented to search and browse the NASA K - 16 Science Education Programs. The system includes new design features that take into account the unique characteristics of the domain and associated tasks. Users with a background in Science Education, including teachers and academic majors, were recruited to perform a number of search tasks. Results from the search experiments were collected and analyzed using both objective and subjective measures. From these results, researchers gained further knowledge about domain-centric video search tasks, including how textual, visual, and hybrid tasks were all deemed important by science educators. Further analysis of experimental results also revealed associations between search tasks, user interaction, interface features and functions, and system effectiveness. The evaluation of individual interface features and functions exhibited that keyword searching was significant for retrieving Science Education video. However, these experiments also produced positive results for various visual search features. Unlike keyword searching, which was consistent and effective across many task types, the use and effectiveness of visual search and browse features were shown to be task dependent. Overall, the results from this study highlight the importance of user- and task-centric methods in video retrieval, as they provided researchers with additional understanding of the influences of domain-specific search tasks on user interaction with video systems. In addition, the experimental methodology employed for this study encourages future foundations for developing and evaluating video search interfaces designed for specific domains and search tasks

Adoption of Industry 4.0 technologies in airports -- A systematic literature review

Airports have been constantly evolving and adopting digital technologies to improve operational efficiency, enhance passenger experience, generate ancillary revenues and boost capacity from existing infrastructure. The COVID-19 pandemic has also challenged airports and aviation stakeholders alike to adapt and manage new operational challenges such as facilitating a contactless travel experience and ensuring business continuity. Digitalisation using Industry 4.0 technologies offers opportunities for airports to address short-term challenges associated with the COVID-19 pandemic while also preparing for future long-term challenges that ensue the crisis. Through a systematic literature review of 102 relevant articles, we discuss the current state of adoption of Industry 4.0 technologies in airports, the associated challenges as well as future research directions. The results of this review suggest that the implementation of Industry 4.0 technologies is slowly gaining traction within the airport environment, and shall continue to remain relevant in the digital transformation journeys in developing future airports