Case Adaptation with Qualitative Algebras

This paper proposes an approach for the adaptation of spatial or temporal cases in a case-based reasoning system. Qualitative algebras are used as spatial and temporal knowledge representation languages. The intuition behind this adaptation approach is to apply a substitution and then repair potential inconsistencies, thanks to belief revision on qualitative algebras. A temporal example from the cooking domain is given. (The paper on which this extended abstract is based was the recipient of the best paper award of the 2012 International Conference on Case-Based Reasoning.

On the automated interpretation and indexing of American football

This work combines natural language understanding and image processing with incremental learning to develop a system that can automatically interpret and index American Football. We have developed a model for representing spatio-temporal characteristics of multiple objects in dynamic scenes in this domain. Our representation combines expert knowledge, domain knowledge, spatial knowledge and temporal knowledge. We also present an incremental learning algorithm to improve the knowledge base as well as to keep previously developed concepts consistent with new data. The advantages of the incremental learning algorithm are that is that it does not split concepts and it generates a compact conceptual hierarchy which does not store instances

Spatial-Temporal Knowledge-Embedded Transformer for Video Scene Graph Generation

Video scene graph generation (VidSGG) aims to identify objects in visual scenes and infer their relationships for a given video. It requires not only a comprehensive understanding of each object scattered on the whole scene but also a deep dive into their temporal motions and interactions. Inherently, object pairs and their relationships enjoy spatial co-occurrence correlations within each image and temporal consistency/transition correlations across different images, which can serve as prior knowledge to facilitate VidSGG model learning and inference. In this work, we propose a spatial-temporal knowledge-embedded transformer (STKET) that incorporates the prior spatial-temporal knowledge into the multi-head cross-attention mechanism to learn more representative relationship representations. Specifically, we first learn spatial co-occurrence and temporal transition correlations in a statistical manner. Then, we design spatial and temporal knowledge-embedded layers that introduce the multi-head cross-attention mechanism to fully explore the interaction between visual representation and the knowledge to generate spatial- and temporal-embedded representations, respectively. Finally, we aggregate these representations for each subject-object pair to predict the final semantic labels and their relationships. Extensive experiments show that STKET outperforms current competing algorithms by a large margin, e.g., improving the mR@50 by 8.1%, 4.7%, and 2.1% on different settings over current algorithms.Comment: Technical Repor

Uncertainty management by relaxation of conflicting constraints in production process scheduling

Mathematical-analytical methods as used in Operations Research approaches are often insufficient for scheduling problems. This is due to three reasons: the combinatorial complexity of the search space, conflicting objectives for production optimization, and the uncertainty in the production process. Knowledge-based techniques, especially approximate reasoning and constraint relaxation, are promising ways to overcome these problems. A case study from an industrial CIM environment, namely high-grade steel production, is presented to demonstrate how knowledge-based scheduling with the desired capabilities could work. By using fuzzy set theory, the applied knowledge representation technique covers the uncertainty inherent in the problem domain. Based on this knowledge representation, a classification of jobs according to their importance is defined which is then used for the straightforward generation of a schedule. A control strategy which comprises organizational, spatial, temporal, and chemical constraints is introduced. The strategy supports the dynamic relaxation of conflicting constraints in order to improve tentative schedules

Surveying medieval archaeology: a new form for Harris paradigm linking photogrammetry and temporal relations

The paper presents some reflexions concerning an interdisciplinary project between Medieval Archaeologists from the University of Florence (Italy) and ICT researchers from CNRS LSIS of Marseille (France), aiming towards a connection between 3D spatial representation and archaeological knowledge. It is well known that Laser Scanner, Photogrammetry and Computer Vision are very attractive tools for archaeologists, although the integration of representation of space and representation of archaeological time has not yet found a methodological standard of reference. We try to develop an integrated system for archaeological 3D survey and all other types of archaeological data and knowledge through integrating observable (material) and non-graphic (interpretive) data. Survey plays a central role, since it is both a metric representation of the archaeological site and, to a wider extent, an interpretation of it (being also a common basis for communication between the 2 teams). More specifically 3D survey is crucial, allowing archaeologists to connect actual spatial assets to the stratigraphic formation processes (i.e. to the archaeological time) and to translate spatial observations into historical interpretation of the site. We propose a common formalism for describing photogrammetrical survey and archaeological knowledge stemming from ontologies: Indeed, ontologies are fully used to model and store 3D data and archaeological knowledge. Xe equip this formalism with a qualitative representation of time. Stratigraphic analyses (both of excavated deposits and of upstanding structures) are closely related to E. C. Harris theory of "Stratigraphic Unit" ("US" from now on). Every US is connected to the others by geometric, topological and, eventually, temporal links, and are recorded by the 3D photogrammetric survey. However, the limitations of the Harris Matrix approach lead to use another representation formalism for stratigraphic relationships, namely Qualitative Constraints Networks (QCN) successfully used in the domain of knowledge representation and reasoning in artificial intelligence for representing temporal relations

Wavelet Lifting over Information-Based EEG Graphs for Motor Imagery Data Classification

The imagination of limb movements offers an intuitive paradigm for the control of electronic devices via brain computer interfacing (BCI). The analysis of electroencephalographic (EEG) data related to motor imagery potentials has proved to be a difficult task. EEG readings are noisy, and the elicited patterns occur in different parts of the scalp, at different instants and at different frequencies. Wavelet transform has been widely used in the BCI field as it offers temporal and spectral capabilities, although it lacks spatial information. In this study we propose a tailored second generation wavelet to extract features from these three domains. This transform is applied over a graph representation of motor imaginary trials, which encodes temporal and spatial information. This graph is enhanced using per-subject knowledge in order to optimise the spatial relationships among the electrodes, and to improve the filter design. This method improves the performance of classifying different imaginary limb movements maintaining the low computational resources required by the lifting transform over graphs. By using an online dataset we were able to positively assess the feasibility of using the novel method in an online BCI context

Joint Video and Text Parsing for Understanding Events and Answering Queries

We propose a framework for parsing video and text jointly for understanding events and answering user queries. Our framework produces a parse graph that represents the compositional structures of spatial information (objects and scenes), temporal information (actions and events) and causal information (causalities between events and fluents) in the video and text. The knowledge representation of our framework is based on a spatial-temporal-causal And-Or graph (S/T/C-AOG), which jointly models possible hierarchical compositions of objects, scenes and events as well as their interactions and mutual contexts, and specifies the prior probabilistic distribution of the parse graphs. We present a probabilistic generative model for joint parsing that captures the relations between the input video/text, their corresponding parse graphs and the joint parse graph. Based on the probabilistic model, we propose a joint parsing system consisting of three modules: video parsing, text parsing and joint inference. Video parsing and text parsing produce two parse graphs from the input video and text respectively. The joint inference module produces a joint parse graph by performing matching, deduction and revision on the video and text parse graphs. The proposed framework has the following objectives: Firstly, we aim at deep semantic parsing of video and text that goes beyond the traditional bag-of-words approaches; Secondly, we perform parsing and reasoning across the spatial, temporal and causal dimensions based on the joint S/T/C-AOG representation; Thirdly, we show that deep joint parsing facilitates subsequent applications such as generating narrative text descriptions and answering queries in the forms of who, what, when, where and why. We empirically evaluated our system based on comparison against ground-truth as well as accuracy of query answering and obtained satisfactory results