Measures of Similarity between Qualitative Descriptions of Shape, Colour and Size Applied to Mosaic Assembling

A computational approach for obtaining a similarity measure between qualitative descriptions of shape, colour and size of objects within digital images is presented. According to the definition of the qualitative features, the similarity values determined are based on conceptual neighbourhood diagrams or interval distances. An approximate matching algorithm between object descriptions is defined and applied to tile mosaic assembling and results of previous approaches are improved.This work has been partially supported by Universitat Jaume I (Fons del Pla Estratégic de 2011/2012), by the Zentrale Forschungsförderung der Universität Bremen under the project name “Providing human-understandable qualitative and semantic descriptions”, and by the Spanish Ministry of Science and Innovation under project ARTEMISA (TIN2009-14378-C02-01)

Detecting and indexing moving objects for Behavior Analysis by Video and Audio Interpretation

2012 - 2013In the last decades we have assisted to a growing need for security in many public environments. According to a study recently conducted by the European Security Observatory, one half of the entire population is worried about the crime and requires the law enforcement to be protected. This consideration has lead the proliferation of cameras and microphones, which represent a suitable solution for their relative low cost of maintenance, the possibility of installing them virtually everywhere and, finally, the capability of analysing more complex events. However, the main limitation of this traditional audiovideo surveillance systems lies in the so called psychological overcharge issue of the human operators responsible for security, that causes a decrease in their capabilities to analyse raw data flows from multiple sources of multimedia information; indeed, as stated by a study conducted by Security Solutions magazine, after 12 minutes of continuous video monitoring, a guard will often miss up to 45% of screen activity. After 22 minutes of video, up to 95% is overlooked. For the above mentioned reasons, it would be really useful to have available an intelligent surveillance system, able to provide images and video with a semantic interpretation, for trying to bridge the gap between their low-level representation in terms of pixels, and the high-level, natural language description that a human would give about them. On the other hand, this kind of systems, able to automatically understand the events occurring in a scene, would be really useful in other application fields, mainly oriented to marketing purposes. Especially in the last years, a lot of business intelligent applications have been installed for assisting decision makers and for giving an organization’s employees, partners and suppliers easy access to the information they need to effectively do their jobs... [edited by author]XII n.s

Perceptually-based language to simplify sketch recognition user interface development

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 473-495).Diagrammatic sketching is a natural modality of human-computer interaction that can be used for a variety of tasks, for example, conceptual design. Sketch recognition systems are currently being developed for many domains. However, they require signal-processing expertise if they are to handle the intricacies of each domain, and they are time-consuming to build. Our goal is to enable user interface designers and domain experts who may not have expertise in sketch recognition to be able to build these sketch systems. We created and implemented a new framework (FLUID - f acilitating user interface development) in which developers can specify a domain description indicating how domain shapes are to be recognized, displayed, and edited. This description is then automatically transformed into a sketch recognition user interface for that domain. LADDER, a language using a perceptual vocabulary based on Gestalt principles, was developed to describe how to recognize, display, and edit domain shapes. A translator and a customizable recognition system (GUILD - a generator of user interfaces using ladder descriptions) are combined with a domain description to automatically create a domain specific recognition system.(cont.) With this new technology, by writing a domain description, developers are able to create a new sketch interface for a domain, greatly reducing the time and expertise for the task Continuing in pursuit of our goal to facilitate UI development, we noted that 1) human generated descriptions contained syntactic and conceptual errors, and that 2) it is more natural for a user to specify a shape by drawing it than by editing text. However, computer generated descriptions from a single drawn example are also flawed, as one cannot express all allowable variations in a single example. In response, we created a modification of the traditional model of active learning in which the system selectively generates its own near-miss examples and uses the human teacher as a source of labels. System generated near-misses offer a number of advantages. Human generated examples are tedious to create and may not expose problems in the current concept. It seems most effective for the near-miss examples to be generated by whichever learning participant (teacher or student) knows better where the deficiencies lie; this will allow the concepts to be more quickly and effectively refined.(cont.) When working in a closed domain such as this one, the computer learner knows exactly which conceptual uncertainties remain, and which hypotheses need to be tested and confirmed. The system uses these labeled examples to automatically build a LADDER shape description, using a modification of the version spaces algorithm that handles interrelated constraints, and which also has the ability to learn negative and disjunctive constraints.by Tracy Anne Hammond.Ph.D

Doctor of Philosophy

dissertationVisualization and exploration of volumetric datasets has been an active area of research for over two decades. During this period, volumetric datasets used by domain users have evolved from univariate to multivariate. The volume datasets are typically explored and classified via transfer function design and visualized using direct volume rendering. To improve classification results and to enable the exploration of multivariate volume datasets, multivariate transfer functions emerge. In this dissertation, we describe our research on multivariate transfer function design. To improve the classification of univariate volumes, various one-dimensional (1D) or two-dimensional (2D) transfer function spaces have been proposed; however, these methods work on only some datasets. We propose a novel transfer function method that provides better classifications by combining different transfer function spaces. Methods have been proposed for exploring multivariate simulations; however, these approaches are not suitable for complex real-world datasets and may be unintuitive for domain users. To this end, we propose a method based on user-selected samples in the spatial domain to make complex multivariate volume data visualization more accessible for domain users. However, this method still requires users to fine-tune transfer functions in parameter space transfer function widgets, which may not be familiar to them. We therefore propose GuideME, a novel slice-guided semiautomatic multivariate volume exploration approach. GuideME provides the user, an easy-to-use, slice-based user interface that suggests the feature boundaries and allows the user to select features via click and drag, and then an optimal transfer function is automatically generated by optimizing a response function. Throughout the exploration process, the user does not need to interact with the parameter views at all. Finally, real-world multivariate volume datasets are also usually of large size, which is larger than the GPU memory and even the main memory of standard work stations. We propose a ray-guided out-of-core, interactive volume rendering and efficient query method to support large and complex multivariate volumes on standard work stations

Interaktive Suchprozesse in komplexen Arbeitssituationen - Ein Retrieval Framework

In recent years a steady increase of information produced in organizations can be noticed. In order to stay competitive, companies have a growing interest in reusing existing knowledge from past projects. Furthermore, a complete picture of the available information is necessary to be able to make informed decisions. The variety and complexity of information in modern organizations often exceeds the capabilities of the currently deployed enterprise search solutions. The reasons for that are manifold and range from non-linked information from multiple software systems to missing functionality to support users during search tasks. Existing search engines often do not support the search paradigms necessary in these environments. On many occasions, users are not aware of the results they will find during the formulation of the search queries. Additionally, the aspect of knowledge building and the identification of new insights into the available data is a priority for the users. Therefore, search paradigms are useful to provide users with tools that support exploratory navigation in a data set and help them to recognize relationships between search results. The goal of this publication is the introduction of a framework that supports exploratory searches in an organizational setting. The described LFRP-framework is built on top of four pillars. 1. The multi-layer functionality allows users to formulate complex search queries referring to more than one result type. Therewith, it enables search queries that - starting from a set of relevant projects - allow selections of documents that are linked to these projects. 2. The search paradigm of faceted searching supports users in formulating search queries incrementally by offering dynamic and valid filter criteria that avoid empty result sets. 3. By combining the concept of faceted search with the capability to influence the search result order based on filter criteria, users can define in a fine-grained way which criteria values shall be weighted stronger or weaker in the search results. The interaction with the ranking is conducted transparently by the so-called user preference functions. 4. The last pillar consists of the visualization type of parallel coordinates covering two tasks in the search user interface of the LFRP-Framework. On the one hand, users formulate their search queries solely graphically in the parallel coordinates and on the other hand they obtain a visual representation of the search results and are able to discover relationships between search results and their facets. The framework is introduced formally from a query model point of view as well as a prototypical implementation. It enables users to access large linked data sets by navigation and constitutes a contribution to a comprehensive information strategy for organizations.Seit einigen Jahren ist ein stetiges Ansteigen der Menge an Informationen, die in Unternehmen erzeugt werden, festzustellen. Um als Unternehmen wettbewerbsfähig zu bleiben, ist es notwendig, vorhandenes Wissen wiederzuverwenden, um aus vergangenen Projektergebnissen profitieren zu können. Weiterhin ist ein vollständiges Informationsbild unabdingbar, um informierte Entscheidungen treffen zu können. Die Informationsvielfalt in modernen Unternehmen übersteigt häufig die Fähigkeiten aktuell anzutreffender unternehmensweiter Suchlösungen. Die Gründe hierfür sind vielfältig und reichen von nicht verknüpften Informationen aus verschiedenen Softwaresystemen bis hin zu fehlenden Funktionen, um den Nutzer bei der Suche zu unterstützen. Vorhandene Suchfunktionen im Unternehmen unterstützen häufig nicht die Suchparadigmen, die in diesem Umfeld notwendig sind. Vielfach ist den Suchenden bei der Formulierung ihrer Suchanfrage nicht bekannt, welche Ergebnisse sie finden werden. Stattdessen steht der Aspekt des Wissensaufbaus und der Gewinnung neuer Einsichten in den vorhandenen Daten im Vordergrund. Hierzu werden Suchparadigmen benötigt, die dem Nutzer Werkzeuge zur Verfügung stellen, die ein exploratives Navigieren im Datenbestand erlauben und ihnen bei der Erkennung von Zusammenhängen in den Suchergebnissen unterstützen. Das Ziel dieser Arbeit ist die Vorstellung eines Rahmenwerks, dass explorative Suchvorhaben im Unternehmensumfeld unterstützt. Das beschriebene LFRP-Framework baut auf vier Säulen auf. 1. Die Multi-Layer Funktionalität erlaubt es Nutzern, komplexe Suchanfragen zu formulieren, die sich auf mehr als einen Ergebnistyp beziehen. Dies ermöglicht beispielsweise Suchabfragen, die - ausgehend von einer Menge von relevanten vergangenen Projekten - Selektionen auf den dazugehörigen Dokumenten erlauben. 2. Das Suchparadigma der facettierten Suche unterstützt Nutzer bei der inkrementellen Formulierung von Suchanfragen mithilfe von dynamisch angebotenen Filterkriterien und vermeidet leere Ergebnismengen durch die Bereitstellung gültiger Filterkriterien. 3. Die Erweiterung der facettierten Suche um die Möglichkeit, die Suchergebnisreihenfolge basierend auf Filterkriterien zu beeinflussen, erlaubt es Nutzern feingranular vorzugeben, welche Kriterienausprägungen im Suchergebnis stärker gewichtet werden sollen. Für den Nutzer geschieht die Beeinflussung des Rankings transparent über sogenannte Nutzerpräferenzfunktionen. 4. Die letzte Säule umfasst die Visualisierung der parallelen Koordinaten, die in der Suchoberfläche des LFRP-Frameworks zwei Aufgaben übernimmt. Zum einen formuliert der Nutzer damit die Suchanfrage ausschließlich grafisch über die Visualisierung und zum anderen erhält er eine grafische Repräsentation der Suchergebnisse und kann so leichter Beziehungen zwischen Suchergebnissen und deren Facetten erkennen. Das Framework, welches in dieser Arbeit formal aus Sicht des Anfragemodells sowie als prototypische Umsetzung betrachtet wird, ermöglicht Nutzern den navigierenden Zugriff auf große vernetze Datenbestände und stellt einen Baustein einer umfassenden Informationsstrategie für Unternehmen dar

Disparate View Matching

Matching of disparate views has gained significance in computer vision due to its role in many novel application areas. Being able to match images of the same scene captured during day and night, between a historic and contemporary picture of a scene, and between aerial and ground-level views of a building facade all enable novel applications ranging from loop-closure detection for structure-from-motion and re-photography to geo-localization of a street-level image using reference imagery captured from the air. The goal of this work is to develop novel features and methods that address matching problems where direct appearance-based correspondences are either difficult to obtain or infeasible because of the lack of appearance similarity altogether. To address these problems, we propose methods that span the appearance-geometry spectrum in terms of both the use of these cues as well as the ability of each method to handle variations in appearance and geometry. First, we consider the problem of geo-localization of a query street-level image using a reference database of building facades captured from a bird\u27s eye view. To address this wide-baseline facade matching problem, a novel scale-selective self-similarity feature that avoids direct comparison of appearance between disparate facade images is presented. Next, to address image matching problems with more extreme appearance variation, a novel representation for matchable images expressed in terms of the eigen-functions of the joint graph of the two images is presented. This representation is used to derive features that are persistent across wide variations in appearance. Next, the problem setting of matching between a street-level image and a digital elevation map (DEM) is considered. Given the limited appearance information available in this scenario, the matching approach has to rely more significantly on geometric cues. Therefore, a purely geometric method to establish correspondences between building corners in the DEM and the visible corners in the query image is presented. Finally, to generalize this problem setting we address the problem of establishing correspondences between 3D and 2D point clouds using geometric means alone. A novel framework for incorporating purely geometric constraints into a higher-order graph matching framework is presented with specific formulations for the three-point calibrated absolute camera pose problem (P3P), two-point upright camera pose problem (Up2p) and the three-plus-one relative camera pose problem

Semantic Trajectories:Computing and Understanding Mobility Data

Thanks to the rapid development of mobile sensing technologies (like GPS, GSM, RFID, accelerometer, gyroscope, sound and other sensors in smartphones), the large-scale capture of evolving positioning data (called mobility data or trajectories) generated by moving objects with embedded sensors has become easily feasible, both technically and economically. We have already entered a world full of trajectories. The state-of-the-art on trajectory, either from the moving object database area or in the statistical analysis viewpoint, has built a bunch of sophisticated techniques for trajectory data ad-hoc storage, indexing, querying and mining etc. However, most of these existing methods mainly focus on a spatio-temporal viewpoint of mobility data, which means they analyze only the geometric movement of trajectories (e.g., the raw ‹x, y, t› sequential data) without enough consideration on the high-level semantics that can better understand the underlying meaningful movement behaviors. Addressing this challenging issue for better understanding movement behaviors from the raw mobility data, this doctoral work aims at providing a high-level modeling and computing methodology for semantically abstracting the rapidly increasing mobility data. Therefore, we bring top-down semantic modeling and bottom-up data computing together and establish a new concept called "semantic trajectories" for mobility data representation and understanding. As the main novelty contribution, this thesis provides a rich, holistic, heterogeneous and application-independent methodology for computing semantic trajectories to better understand mobility data at different levels. In details, this methodology is composed of five main parts with dedicated contributions. Semantic Trajectory Modeling. By investigating trajectory modeling requirements to better understand mobility data, this thesis first designs a hybrid spatio-semantic trajectory model that represents mobility with rich data abstraction at different levels, i.e., from the low-level spatio-temporal trajectory to the intermediate-level structured trajectory, and finally to the high-level semantic trajectory. In addition, a semantic based ontological framework has also been designed and applied for querying and reasoning on trajectories. Offline Trajectory Computing. To utilize the hybrid model, the thesis complementarily designs a holistic trajectory computing platform with dedicated algorithms for reconstructing trajectories at different levels. The platform can preprocess collected mobility data (i.e., raw movement tracks like GPS feeds) in terms of data cleaning/compression etc., identify individual trajectories, and segment them into structurally meaningful trajectory episodes. Therefore, this trajectory computing platform can construct spatio-temporal trajectories and structured trajectories from the raw mobility data. Such computing platform is initially designed as an offline solution which is supposed to analyze past trajectories via a batch procedure. Trajectory Semantic Annotation. To achieve the final semantic level for better understanding mobility data, this thesis additionally designs a semantic annotation platform that can enrich trajectories with third party sources that are composed of geographic background information and application domain knowledge, to further infer more meaningful semantic trajectories. Such annotation platform is application-independent that can annotate various trajectories (e.g., mobility data of people, vehicle and animals) with heterogeneous data sources of semantic knowledge (e.g., third party sources in any kind of geometric shapes like point, line and region) that can help trajectory enrichment. Online Trajectory Computing. In addition to the offline trajectory computing for analyzing past trajectories, this thesis also contributes to dealing with ongoing trajectories in terms of real-time trajectory computing from movement data streams. The online trajectory computing platform is capable of providing real-life trajectory data cleaning, compression, and segmentation over streaming movement data. In addition, the online platform explores the functionality of online tagging to achieve fully semantic-aware trajectories and further evaluate trajectory computing in a real-time setting. Mining Trajectories from Multi-Sensors. Previously, the focus is on computing semantic trajectories using single-sensory data (i.e., GPS feeds), where most datasets are from moving objects with wearable GPS-embedded sensors (e.g., mobility data of animal, vehicle and people tracking). In addition, we explore the problem of mining people trajectories using multi-sensory feeds from smartphones (GPS, gyroscope, accelerometer etc). The research results reveal that the combination of two sensors (GPS+accelerometer) can significantly infer a complete life-cycle semantic trajectories of people's daily behaviors, both outdoor movement via GPS and indoor activities via accelerometer