Semantic Similarity of Spatial Scenes

The formalization of similarity in spatial information systems can unleash their functionality and contribute technology not only useful, but also desirable by broad groups of users. As a paradigm for information retrieval, similarity supersedes tedious querying techniques and unveils novel ways for user-system interaction by naturally supporting modalities such as speech and sketching. As a tool within the scope of a broader objective, it can facilitate such diverse tasks as data integration, landmark determination, and prediction making. This potential motivated the development of several similarity models within the geospatial and computer science communities. Despite the merit of these studies, their cognitive plausibility can be limited due to neglect of well-established psychological principles about properties and behaviors of similarity. Moreover, such approaches are typically guided by experience, intuition, and observation, thereby often relying on more narrow perspectives or restrictive assumptions that produce inflexible and incompatible measures. This thesis consolidates such fragmentary efforts and integrates them along with novel formalisms into a scalable, comprehensive, and cognitively-sensitive framework for similarity queries in spatial information systems. Three conceptually different similarity queries at the levels of attributes, objects, and scenes are distinguished. An analysis of the relationship between similarity and change provides a unifying basis for the approach and a theoretical foundation for measures satisfying important similarity properties such as asymmetry and context dependence. The classification of attributes into categories with common structural and cognitive characteristics drives the implementation of a small core of generic functions, able to perform any type of attribute value assessment. Appropriate techniques combine such atomic assessments to compute similarities at the object level and to handle more complex inquiries with multiple constraints. These techniques, along with a solid graph-theoretical methodology adapted to the particularities of the geospatial domain, provide the foundation for reasoning about scene similarity queries. Provisions are made so that all methods comply with major psychological findings about people’s perceptions of similarity. An experimental evaluation supplies the main result of this thesis, which separates psychological findings with a major impact on the results from those that can be safely incorporated into the framework through computationally simpler alternatives

Developing tools and models for evaluating geospatial data integration of official and VGI data sources

PhD ThesisIn recent years, systems have been developed which enable users to produce, share and update information on the web effectively and freely as User Generated Content (UGC) data (including Volunteered Geographic Information (VGI)). Data quality assessment is a major concern for supporting the accurate and efficient spatial data integration required if VGI is to be used alongside official, formal, usually governmental datasets. This thesis aims to develop tools and models for the purpose of assessing such integration possibilities. Initially, in order to undertake this task, geometrical similarity of formal and informal data was examined. Geometrical analyses were performed by developing specific programme interfaces to assess the positional, linear and polygon shape similarity among reference field survey data (FS); official datasets such as data from Ordnance Survey (OS), UK and General Directorate for Survey (GDS), Iraq agencies; and VGI information such as OpenStreetMap (OSM) datasets. A discussion of the design and implementation of these tools and interfaces is presented. A methodology has been developed to assess such positional and shape similarity by applying different metrics and standard indices such as the National Standard for Spatial Data Accuracy (NSSDA) for positional quality; techniques such as buffering overlays for linear similarity; and application of moments invariant for polygon shape similarity evaluations. The results suggested that difficulties exist for any geometrical integration of OSM data with both bench mark FS and formal datasets, but that formal data is very close to reference datasets. An investigation was carried out into contributing factors such as data sources, feature types and number of data collectors that may affect the geometrical quality of OSM data and consequently affect the integration process of OSM datasets with FS, OS and GDS. Factorial designs were undertaken in this study in order to develop and implement an experiment to discover the effect of these factors individually and the interaction between each of them. The analysis found that data source is the most significant factor that affects the geometrical quality of OSM datasets, and that there are interactions among all these factors at different levels of interaction. This work also investigated the possibility of integrating feature classification of official datasets such as data from OS and GDS geospatial data agencies, and informal datasets such as OSM information. In this context, two different models were developed. The first set of analysis included the evaluation of semantic integration of corresponding feature classifications of compared datasets. The second model was concerned with assessing the ability of XML schema matching of feature classifications of tested datasets. This initially involved a tokenization process in order to split up into single words classifications that were composed of multiple words. Subsequently, encoding feature classifications as XML schema trees was undertaken. The semantic similarity, data type similarity and structural similarity were measured between the nodes of compared schema trees. Once these three similarities had been computed, a weighted combination technique has been adopted in order to obtain the overall similarity. The findings of both sets of analysis were not encouraging as far as the possibility of effectively integrating feature classifications of VGI datasets, such as OSM information, and formal datasets, such as OS and GDS datasets, is concerned.Ministry of Higher Education and Scientific Research, Republic of Iraq

Modeling Visual Rhetoric and Semantics in Multimedia

Recent advances in machine learning have enabled computer vision algorithms to model complicated visual phenomena with accuracies unthinkable a mere decade ago. Their high-performance on a plethora of vision-related tasks has enabled computer vision researchers to begin to move beyond traditional visual recognition problems to tasks requiring higher-level image understanding. However, most computer vision research still focuses on describing what images, text, or other media literally portrays. In contrast, in this dissertation we focus on learning how and why such content is portrayed. Rather than viewing media for its content, we recast the problem as understanding visual communication and visual rhetoric. For example, the same content may be portrayed in different ways in order to present the story the author wishes to convey. We thus seek to model not only the content of the media, but its authorial intent and latent messaging. Understanding how and why visual content is portrayed a certain way requires understanding higher level abstract semantic concepts which are themselves latent within visual media. By latent, we mean the concept is not readily visually accessible within a single image (e.g. right vs left political bias), in contrast to explicit visual semantic concepts such as objects. Specifically, we study the problems of modeling photographic style (how professional photographers portray their subjects), understanding visual persuasion in image advertisements, modeling political bias in multimedia (image and text) news articles, and learning cross-modal semantic representations. While most past research in vision and natural language processing studies the case where visual content and paired text are highly aligned (as in the case of image captions), we target the case where each modality conveys complementary information to tell a larger story. We particularly focus on the problem of learning cross-modal representations from multimedia exhibiting weak alignment between the image and text modalities. A variety of techniques are presented which improve modeling of multimedia rhetoric in real-world data and enable more robust artificially intelligent systems

Grounding semantic cognition using computational modelling and network analysis

The overarching objective of this thesis is to further the field of grounded semantics using a range of computational and empirical studies. Over the past thirty years, there have been many algorithmic advances in the modelling of semantic cognition. A commonality across these cognitive models is a reliance on hand-engineering “toy-models”. Despite incorporating newer techniques (e.g. Long short-term memory), the model inputs remain unchanged. We argue that the inputs to these traditional semantic models have little resemblance with real human experiences. In this dissertation, we ground our neural network models by training them with real-world visual scenes using naturalistic photographs. Our approach is an alternative to both hand-coded features and embodied raw sensorimotor signals. We conceptually replicate the mutually reinforcing nature of hybrid (feature-based and grounded) representations using silhouettes of concrete concepts as model inputs. We next gradually develop a novel grounded cognitive semantic representation which we call scene2vec, starting with object co-occurrences and then adding emotions and language-based tags. Limitations of our scene-based representation are identified for more abstract concepts (e.g. freedom). We further present a large-scale human semantics study, which reveals small-world semantic network topologies are context-dependent and that scenes are the most dominant cognitive dimension. This finding leads us to conclude that there is no meaning without context. Lastly, scene2vec shows promising human-like context-sensitive stereotypes (e.g. gender role bias), and we explore how such stereotypes are reduced by targeted debiasing. In conclusion, this thesis provides support for a novel computational viewpoint on investigating meaning - scene-based grounded semantics. Future research scaling scene-based semantic models to human-levels through virtual grounding has the potential to unearth new insights into the human mind and concurrently lead to advancements in artificial general intelligence by enabling robots, embodied or otherwise, to acquire and represent meaning directly from the environment

Explanatory Models, Unit Standards, and Personalized Learning in Educational Measurement

The papers by Jack Stenner included in this book document the technical details of an art and science of measurement that creates new entrepreneurial business opportunities. Jack brought theory, instruments, and data together in ways that are applicable not only in the context of a given test of reading or mathematics ability, but which more importantly catalyzed literacy and numeracy capital in new fungible expressions. Though Jack did not reflect in writing on the inferential, constructive processes in which he engaged, much can be learned by reviewing his work with his accomplishments in mind. A Foreword by Stenner's colleague and co-author on multiple works, William P. Fisher, Jr., provides key clues concerning (a) how Jack's understanding of measurement and its values aligns with social and historical studies of science and technology, and (b) how recent developments in collaborations of psychometricians and metrologists are building on and expanding Jack's accomplishments. ​ This is an open access book

Semantic discovery and reuse of business process patterns

Patterns currently play an important role in modern information systems (IS) development and their use has mainly been restricted to the design and implementation phases of the development lifecycle. Given the increasing significance of business modelling in IS development, patterns have the potential of providing a viable solution for promoting reusability of recurrent generalized models in the very early stages of development. As a statement of research-in-progress this paper focuses on business process patterns and proposes an initial methodological framework for the discovery and reuse of business process patterns within the IS development lifecycle. The framework borrows ideas from the domain engineering literature and proposes the use of semantics to drive both the discovery of patterns as well as their reuse

Semantic Systems. The Power of AI and Knowledge Graphs

This open access book constitutes the refereed proceedings of the 15th International Conference on Semantic Systems, SEMANTiCS 2019, held in Karlsruhe, Germany, in September 2019. The 20 full papers and 8 short papers presented in this volume were carefully reviewed and selected from 88 submissions. They cover topics such as: web semantics and linked (open) data; machine learning and deep learning techniques; semantic information management and knowledge integration; terminology, thesaurus and ontology management; data mining and knowledge discovery; semantics in blockchain and distributed ledger technologies

Explanatory Models, Unit Standards, and Personalized Learning in Educational Measurement

The papers by Jack Stenner included in this book document the technical details of an art and science of measurement that creates new entrepreneurial business opportunities. Jack brought theory, instruments, and data together in ways that are applicable not only in the context of a given test of reading or mathematics ability, but which more importantly catalyzed literacy and numeracy capital in new fungible expressions. Though Jack did not reflect in writing on the inferential, constructive processes in which he engaged, much can be learned by reviewing his work with his accomplishments in mind. A Foreword by Stenner's colleague and co-author on multiple works, William P. Fisher, Jr., provides key clues concerning (a) how Jack's understanding of measurement and its values aligns with social and historical studies of science and technology, and (b) how recent developments in collaborations of psychometricians and metrologists are building on and expanding Jack's accomplishments. ​ This is an open access book

Role of language in conceptual coordination

Although concepts are located within individual minds, while word forms are shared across entire language communities, words and concepts are normally deemed to be tightly bound. But in fact, at least to the extent that concepts vary, the relationship between words and concepts may not be as uniform or stable as is often assumed. Nevertheless, language may itself mediate that relationship, through its entrenchment and use. Psychologists have already investigated language use in referential communication, but they have yet to focus in detail on the role of language in conceptual coordination. One of the obstacles has been the theoretical and methodological challenges that arise from seriously abandoning conceptual universals. To that end, an experimental framework was developed based on sorting tasks in which participants freely partition a set of stimuli into categories and an objective measure for comparing two outputs. Four experiments were then conducted to investigate whether people were conceptually coordinated before, during and after linguistic interaction. Experiment 1 consisted of a cross-linguistic study looking at default coordination between native speakers. Participants both sorted items into groups and named them individually. There was a relatively high degree of categorisation agreement among speakers of the same language, but not nearly as high as for naming agreement. Experiments 2-4 inquired into conceptual coordination during or immediately after linguistic interaction. Experimental manipulations involved the form of language use (full dialogue or only category labels), as well as the type of feedback (category groupings, labels, both, or neither). In particular, Experiment 2 investigated the effects of categorising a set of objects together, with or without dialogue, on subsequent individual categorisation. The results were inconclusive and revealed specific methodological issues, but yielded interesting data and were encouraging for the general framework. Experiment 3 modified the designwhile testing and extending the same general hypotheses. Participants carried out a sequence of categorisation tasks in which they tried to coordinate their categories, followed by individual categorisation and similarity tasks. The availability of dialogue and feedback was manipulated in the interactive tasks. During interaction, they also received both kinds of feedback, except in the control condition. Pairs that could talk coordinated much better than the others, but feedback didn’t help. Experiment 4 looked into the effects of the four possibilities for feedback during a longer sequence of interactive tasks. In general, conceptual coordination was found to depend on grouping feedback only. However, by the end of the task, pairs who received both kinds of feedback did best. All three interactive experiments also measured lexical convergence between pairs. The results generally revealed a dissociation, with lexical alignment showingmore convergence and occurring under a wider variety of conditions. Togetherwith previous research, these findings showthat language can bring about conceptual coordination. However, it appears that the richer the form of language use, the more conceptual convergence occurs, and the closer it gets coupled with lexical convergence. The long-term effects, if any, are much weaker. These studies have implications for the general role of language in cognition and other important issues