Creation of virtual worlds from 3D models retrieved from content aware networks based on sketch and image queries

The recent emergence of user generated content requires new content creation tools that will be both easy to learn and easy to use. These new tools should enable the user to construct new high-quality content with minimum effort; it is essential to allow existing multimedia content to be reused as building blocks when creating new content. In this work we present a new tool for automatically constructing virtual worlds with minimum user intervention. Users can create these worlds by drawing a simple sketch, or by using interactively segmented 2D objects from larger images. The system receives as a query the sketch or the segmented image, and uses it to find similar 3D models that are stored in a Content Centric Network. The user selects a suitable model from the retrieved models, and the system uses it to automatically construct a virtual 3D world

Automated mood boards - Ontology-based semantic image retrieval

The main goal of this research is to support concept designers’ search for inspirational and meaningful images in developing mood boards. Finding the right images has become a well-known challenge as the amount of images stored and shared on the Internet and elsewhere keeps increasing steadily and rapidly. The development of image retrieval technologies, which collect, store and pre-process image information to return relevant images instantly in response to users’ needs, have achieved great progress in the last decade. However, the keyword-based content description and query processing techniques for Image Retrieval (IR) currently used have their limitations. Most of these techniques are adapted from the Information Retrieval research, and therefore provide limited capabilities to grasp and exploit conceptualisations due to their inability to handle ambiguity, synonymy, and semantic constraints. Conceptual search (i.e. searching by meaning rather than literal strings) aims to solve the limitations of the keyword-based models. Starting from this point, this thesis investigates the existing IR models, which are oriented to the exploitation of domain knowledge in support of semantic search capabilities, with a focus on the use of lexical ontologies to improve the semantic perspective. It introduces a technique for extracting semantic DNA (SDNA) from textual image annotations and constructing semantic image signatures. The semantic signatures are called semantic chromosomes; they contain semantic information related to the images. Central to the method of constructing semantic signatures is the concept disambiguation technique developed, which identifies the most relevant SDNA by measuring the semantic importance of each word/phrase in the image annotation. In addition, a conceptual model of an ontology-based system for generating visual mood boards is proposed. The proposed model, which is adapted from the Vector Space Model, exploits the use of semantic chromosomes in semantic indexing and assessing the semantic similarity of images within a collection

Exploiting multimedia in creating and analysing multimedia Web archives

The data contained on the web and the social web are inherently multimedia and consist of a mixture of textual, visual and audio modalities. Community memories embodied on the web and social web contain a rich mixture of data from these modalities. In many ways, the web is the greatest resource ever created by human-kind. However, due to the dynamic and distributed nature of the web, its content changes, appears and disappears on a daily basis. Web archiving provides a way of capturing snapshots of (parts of) the web for preservation and future analysis. This paper provides an overview of techniques we have developed within the context of the EU funded ARCOMEM (ARchiving COmmunity MEMories) project to allow multimedia web content to be leveraged during the archival process and for post-archival analysis. Through a set of use cases, we explore several practical applications of multimedia analytics within the realm of web archiving, web archive analysis and multimedia data on the web in general

3D CBIR with sparse coding for image-guided neurosurgery

This research takes an application-specific approach to investigate, extend and implement the state of the art in the fields of both visual information retrieval and machine learning, bridging the gap between theoretical models and real world applications. During an image-guided neurosurgery, path planning remains the foremost and hence the most important step to perform an operation and ensures the maximum resection of an intended target and minimum sacrifice of health tissues. In this investigation, the technique of content-based image retrieval (CBIR) coupled with machine learning algorithms are exploited in designing a computer aided path planning system (CAP) to assist junior doctors in planning surgical paths while sustaining the highest precision. Specifically, after evaluation of approaches of sparse coding and K-means in constructing a codebook, the model of sparse codes of 3D SIFT has been furthered and thereafter employed for retrieving, The novelty of this work lies in the fact that not only the existing algorithms for 2D images have been successfully extended into 3D space, leading to promising results, but also the application of CBIR, that is mainly in a research realm, to a clinical sector can be achieved by the integration with machine learning techniques. Comparison with the other four popular existing methods is also conducted, which demonstrates that with the implementation of sparse coding, all methods give better retrieval results than without while constituting the codebook, implying the significant contribution of machine learning techniques