Description Logic for Scene Understanding at the Example of Urban Road Intersections

Understanding a natural scene on the basis of external sensors is a task yet to be solved by computer algorithms. The present thesis investigates the suitability of a particular family of explicit, formal representation and reasoning formalisms for this task, which are subsumed under the term Description Logic

Qualitative spatial logic descriptors from 3D indoor scenes to generate explanations in natural language

Falomir Z, Kluth T. Qualitative spatial logic descriptors from 3D indoor scenes to generate explanations in natural language. Cognitive Processing. 2018;19(2):265-284.The challenge of describing 3D real scenes is tackled in this paper using qualitative spatial descriptors. A key point to study is which qualitative descriptors to use and how these qualitative descriptors must be organized to produce a suitable cognitive explanation. In order to find answers, a survey test was carried out with human participants which openly described a scene containing some pieces of furniture. The data obtained in this survey are analysed, and taking this into account, the QSn3D computational approach was developed which uses a XBox 360 Kinect to obtain 3D data from a real indoor scene. Object features are computed on these 3D data to identify objects in indoor scenes. The object orientation is computed, and qualitative spatial relations between the objects are extracted. These qualitative spatial relations are the input to a grammar which applies saliency rules obtained from the survey study and generates cognitive natural language descriptions of scenes. Moreover, these qualitative descriptors can be expressed as first-order logical facts in Prolog for further reasoning. Finally, a validation study is carried out to test whether the descriptions provided by QSn3D approach are human readable. The obtained results show that their acceptability is higher than 82%

Structured Knowledge Representation for Image Retrieval

We propose a structured approach to the problem of retrieval of images by content and present a description logic that has been devised for the semantic indexing and retrieval of images containing complex objects. As other approaches do, we start from low-level features extracted with image analysis to detect and characterize regions in an image. However, in contrast with feature-based approaches, we provide a syntax to describe segmented regions as basic objects and complex objects as compositions of basic ones. Then we introduce a companion extensional semantics for defining reasoning services, such as retrieval, classification, and subsumption. These services can be used for both exact and approximate matching, using similarity measures. Using our logical approach as a formal specification, we implemented a complete client-server image retrieval system, which allows a user to pose both queries by sketch and queries by example. A set of experiments has been carried out on a testbed of images to assess the retrieval capabilities of the system in comparison with expert users ranking. Results are presented adopting a well-established measure of quality borrowed from textual information retrieval

Semantic Remote Sensing Scenes Interpretation and Change Interpretation

A fundamental objective of remote sensing imagery is to spread out the knowledge about our environment and to facilitate the interpretation of different phenomena affecting the Earth’s surface. The main goal of this chapter is to understand and interpret possible changes in order to define subsequently strategies and adequate decision-making for a better soil management and protection. Consequently, the semantic interpretation of remote sensing data, which consists of extracting useful information from image date for attaching semantics to the observed phenomenon, allows easy understanding and interpretation of such occurring changes. However, performing change interpretation task is not only based on the perceptual information derived from data but also based on additional knowledge sources such as a prior and contextual. This knowledge needs to be encoded in an appropriate way for being used as a guide in the interpretation process. On the other hand, interpretation may take place at several levels of complexity from the simple recognition of objects on the analyzed scene to the inference of site conditions and to change interpretation. For each level, information elements such as data, information and knowledge need to be represented and characterized. This chapter highlights the importance of ontologies exploiting for encoding the domain knowledge and for using it as a guide in the semantic scene interpretation task

The semantics of similarity in geographic information retrieval

Similarity measures have a long tradition in fields such as information retrieval artificial intelligence and cognitive science. Within the last years these measures have been extended and reused to measure semantic similarity; i.e. for comparing meanings rather than syntactic differences. Various measures for spatial applications have been developed but a solid foundation for answering what they measure; how they are best applied in information retrieval; which role contextual information plays; and how similarity values or rankings should be interpreted is still missing. It is therefore difficult to decide which measure should be used for a particular application or to compare results from different similarity theories. Based on a review of existing similarity measures we introduce a framework to specify the semantics of similarity. We discuss similarity-based information retrieval paradigms as well as their implementation in web-based user interfaces for geographic information retrieval to demonstrate the applicability of the framework. Finally we formulate open challenges for similarity research

Qualitative Distances and Qualitative Description of Images for Indoor Scene Description and Recognition in Robotics

The automatic extraction of knowledge from the world by a robotic system as human beings interpret their environment through their senses is still an unsolved task in Artificial Intelligence. A robotic agent is in contact with the world through its sensors and other electronic components which obtain and process mainly numerical information. Sonar, infrared and laser sensors obtain distance information. Webcams obtain digital images that are represented internally as matrices of red, blue and green (RGB) colour coordinate values. All this numerical values obtained from the environment need a later interpretation in order to provide the knowledge required by the robotic agent in order to carry out a task. Similarly, light wavelengths with specific amplitude are captured by cone cells of human eyes obtaining also stimulus without meaning. However, the information that human beings can describe and remember from what they see is expressed using words, that is qualitatively. The research work done in this thesis tries to narrow the gap between the acquisition of low level information by robot sensors and the need of obtaining high level or qualitative information for enhancing human-machine communication and for applying logical reasoning processes based on concepts. Moreover, qualitative concepts can be added a meaning by relating them to others. They can be used for reasoning applying qualitative models that have been developed in the last twenty years for describing and interpreting metrical and mathematical concepts such as orientation, distance, velocity, acceleration, and so on. And they can be also understood by human-users both written and read aloud. The first contribution presented is the definition of a method for obtaining fuzzy distance patterns (which include qualitative distances such as near , far , very far and so on) from the data obtained by any kind of distance sensors incorporated in a mobile robot and the definition of a factor to measure the dissimilarity between those fuzzy patterns. Both have been applied to the integration of the distances obtained by the sonar and laser distance sensors incorporated in a Pioneer 2 dx mobile robot and, as a result, special obstacles have been detected as glass window , mirror , and so on. Moreover, the fuzzy distance patterns provided have been also defuzzified in order to obtain a smooth robot speed and used to classify orientation reference systems into open (it defines an open space to be explored) or closed . The second contribution presented is the definition of a model for qualitative image description (QID) based on qualitative models of shape, colour, topology and orientation. This model can qualitatively describe any kind of digital image and is independent of the image segmentation method used. The QID model have been tested in two scenarios in robotics: (i) the description of digital images captured by the camera of a Pioneer 2 dx mobile robot and (ii) the description of digital images of tile mosaics taken by an industrial camera located on a platform used by a robot arm to assemble tile mosaics. In order to provide a formal and explicit meaning to the qualitative description of the images generated, a Description Logic (DL) based ontology has been designed and presented as the third contribution. Our approach can automatically process any random image and obtain a set of DL-axioms that describe it visually and spatially. And objects included in the images are classified according to the ontology schema using a DL reasoner. Tests have been carried out using digital images captured by a webcam incorporated in a Pioneer 2 dx mobile robot. The images taken correspond to the corridors of a building at University Jaume I and objects with them have been classified into walls , floor , office doors and fire extinguishers under different illumination conditions and from different observer viewpoints. The final contribution is the definition of a similarity measure between qualitative descriptions of shape, colour, topology and orientation. And the integration of those measures into the definition of a general similarity measure between two qualitative descriptions of images. These similarity measures have been applied to: (i) extract objects with similar shapes from the MPEG7 CE Shape-1 library; (ii) assemble tile mosaics by qualitative shape and colour similarity matching; (iii) compare images of tile compositions; and (iv) compare images of natural landmarks in a mobile robot world for their recognition

Distances in the field : mapping similarity and familiarity in the production, curation and consumption of Australian art music

This thesis provides a timely intervention in the investigation of cultural fields by employing traditional and new data analytics to expand our understanding of fields as multi-dimensional sites of production, curation and consumption. Through a case study of contemporary Australian art music, the research explores the multiple ways in which the concept of ‘distance’ contributes to how we conceive of and engage with fields of artistic practice. While the concept of distance has often been an implicit or axiomatic concern for cultural sociology, this thesis foregrounds how it can be used to analyse fields from multiple perspectives, at multiple scales of enquiry and using diverse methodologies. In doing so, it distinguishes between notions of distance in the related concepts of similarity and familiarity. In the former, the relative proximities of cultural producers can be mapped to discern and contrast the organising principles which underlie different perspectives of a field. In the latter, the degree of an individual’s familiarity with an item or genre can be included in theorisations of cultural preferences and their social dimensions. This is disrupted in a field such as Australian art music, however, as its emphasis on experimentation and innovation presents barriers to developing familiarity. Distance can be considered a defining characteristic of this field, and motivates its selection as a critical case study from which to investigate how audiences form attachments to distant musical sounds. The investigation of distance from multiple perspectives, using different scales of analysis and across a series of focal points in the lifecycle of artist practice, provides an analysis of Australian art music in terms of the tensions which emerge from these intersecting representations of the field. The singular spatial representation of ‘objective relations’ in a field, and a concern with power and domination – as found in the approach of Bourdieu – is replaced by a multiplicity of sets of relations and a concern with their organising principles and juxtapositions. The thesis argues that the actor constellations which distances produce are intimately linked to our capacity to engage with fields as discrete and knowable domains of cultural practice. Beyond our capacity to know a cultural field, it also argues for the importance of reconsidering how we form attachments to distant musical tastes. As an avant-garde genre which embraces foreign and confounding sounds, audiences require the capacity to draw on a range of consumption strategies and techniques to successfully engage with and value the unfamiliar

Visual region understanding: unsupervised extraction and abstraction

The ability to gain a conceptual understanding of the world in uncontrolled environments is the ultimate goal of vision-based computer systems. Technological societies today are heavily reliant on surveillance and security infrastructure, robotics, medical image analysis, visual data categorisation and search, and smart device user interaction, to name a few. Out of all the complex problems tackled by computer vision today in context of these technologies, that which lies closest to the original goals of the field is the subarea of unsupervised scene analysis or scene modelling. However, its common use of low level features does not provide a good balance between generality and discriminative ability, both a result and a symptom of the sensory and semantic gaps existing between low level computer representations and high level human descriptions. In this research we explore a general framework that addresses the fundamental problem of universal unsupervised extraction of semantically meaningful visual regions and their behaviours. For this purpose we address issues related to (i) spatial and spatiotemporal segmentation for region extraction, (ii) region shape modelling, and (iii) the online categorisation of visual object classes and the spatiotemporal analysis of their behaviours. Under this framework we propose (a) a unified region merging method and spatiotemporal region reduction, (b) shape representation by the optimisation and novel simplication of contour-based growing neural gases, and (c) a foundation for the analysis of visual object motion properties using a shape and appearance based nearest-centroid classification algorithm and trajectory plots for the obtained region classes. 1 Specifically, we formulate a region merging spatial segmentation mechanism that combines and adapts features shown previously to be individually useful, namely parallel region growing, the best merge criterion, a time adaptive threshold, and region reduction techniques. For spatiotemporal region refinement we consider both scalar intensity differences and vector optical flow. To model the shapes of the visual regions thus obtained, we adapt the growing neural gas for rapid region contour representation and propose a contour simplication technique. A fast unsupervised nearest-centroid online learning technique next groups observed region instances into classes, for which we are then able to analyse spatial presence and spatiotemporal trajectories. The analysis results show semantic correlations to real world object behaviour. Performance evaluation of all steps across standard metrics and datasets validate their performance