Using spatiotemporal patterns to qualitatively represent and manage dynamic situations of interest : a cognitive and integrative approach

Les situations spatio-temporelles dynamiques sont des situations qui évoluent dans l’espace et dans le temps. L’être humain peut identifier des configurations de situations dans son environnement et les utilise pour prendre des décisions. Ces configurations de situations peuvent aussi être appelées « situations d’intérêt » ou encore « patrons spatio-temporels ». En informatique, les situations sont obtenues par des systèmes d’acquisition de données souvent présents dans diverses industries grâce aux récents développements technologiques et qui génèrent des bases de données de plus en plus volumineuses. On relève un problème important dans la littérature lié au fait que les formalismes de représentation utilisés sont souvent incapables de représenter des phénomènes spatiotemporels dynamiques et complexes qui reflètent la réalité. De plus, ils ne prennent pas en considération l’appréhension cognitive (modèle mental) que l’humain peut avoir de son environnement. Ces facteurs rendent difficile la mise en œuvre de tels modèles par des agents logiciels. Dans cette thèse, nous proposons un nouveau modèle de représentation des situations d’intérêt s’appuyant sur la notion des patrons spatiotemporels. Notre approche utilise les graphes conceptuels pour offrir un aspect qualitatif au modèle de représentation. Le modèle se base sur les notions d’événement et d’état pour représenter des phénomènes spatiotemporels dynamiques. Il intègre la notion de contexte pour permettre aux agents logiciels de raisonner avec les instances de patrons détectés. Nous proposons aussi un outil de génération automatisée des relations qualitatives de proximité spatiale en utilisant un classificateur flou. Finalement, nous proposons une plateforme de gestion des patrons spatiotemporels pour faciliter l’intégration de notre modèle dans des applications industrielles réelles. Ainsi, les contributions principales de notre travail sont : Un formalisme de représentation qualitative des situations spatiotemporelles dynamiques en utilisant des graphes conceptuels. ; Une approche cognitive pour la définition des patrons spatio-temporels basée sur l’intégration de l’information contextuelle. ; Un outil de génération automatique des relations spatiales qualitatives de proximité basé sur les classificateurs neuronaux flous. ; Une plateforme de gestion et de détection des patrons spatiotemporels basée sur l’extension d’un moteur de traitement des événements complexes (Complex Event Processing).Dynamic spatiotemporal situations are situations that evolve in space and time. They are part of humans’ daily life. One can be interested in a configuration of situations occurred in the environment and can use it to make decisions. In the literature, such configurations are referred to as “situations of interests” or “spatiotemporal patterns”. In Computer Science, dynamic situations are generated by large scale data acquisition systems which are deployed everywhere thanks to recent technological advances. Spatiotemporal pattern representation is a research subject which gained a lot of attraction from two main research areas. In spatiotemporal analysis, various works extended query languages to represent patterns and to query them from voluminous databases. In Artificial Intelligence, predicate-based models represent spatiotemporal patterns and detect their instances using rule-based mechanisms. Both approaches suffer several shortcomings. For example, they do not allow for representing dynamic and complex spatiotemporal phenomena due to their limited expressiveness. Furthermore, they do not take into account the human’s mental model of the environment in their representation formalisms. This limits the potential of building agent-based solutions to reason about these patterns. In this thesis, we propose a novel approach to represent situations of interest using the concept of spatiotemporal patterns. We use Conceptual Graphs to offer a qualitative representation model of these patterns. Our model is based on the concepts of spatiotemporal events and states to represent dynamic spatiotemporal phenomena. It also incorporates contextual information in order to facilitate building the knowledge base of software agents. Besides, we propose an intelligent proximity tool based on a neuro-fuzzy classifier to support qualitative spatial relations in the pattern model. Finally, we propose a framework to manage spatiotemporal patterns in order to facilitate the integration of our pattern representation model to existing applications in the industry. The main contributions of this thesis are as follows: A qualitative approach to model dynamic spatiotemporal situations of interest using Conceptual Graphs. ; A cognitive approach to represent spatiotemporal patterns by integrating contextual information. ; An automated tool to generate qualitative spatial proximity relations based on a neuro-fuzzy classifier. ; A platform for detection and management of spatiotemporal patterns using an extension of a Complex Event Processing engine

Towards an Efficient, Scalable Stream Query Operator Framework for Representing and Analyzing Continuous Fields

Advancements in sensor technology have made it less expensive to deploy massive numbers of sensors to observe continuous geographic phenomena at high sample rates and stream live sensor observations. This fact has raised new challenges since sensor streams have pushed the limits of traditional geo-sensor data management technology. Data Stream Engines (DSEs) provide facilities for near real-time processing of streams, however, algorithms supporting representing and analyzing Spatio-Temporal (ST) phenomena are limited. This dissertation investigates near real-time representation and analysis of continuous ST phenomena, observed by large numbers of mobile, asynchronously sampling sensors, using a DSE and proposes two novel stream query operator frameworks. First, the ST Interpolation Stream Query Operator Framework (STI-SQO framework) continuously transforms sensor streams into rasters using a novel set of stream query operators that perform ST-IDW interpolation. A key component of the STI-SQO framework is the 3D, main memory-based, ST Grid Index that enables high performance ST insertion and deletion of massive numbers of sensor observations through Isotropic Time Cell and Time Block-based partitioning. The ST Grid Index facilitates fast ST search for samples using ST shell-based neighborhood search templates, namely the Cylindrical Shell Template and Nested Shell Template. Furthermore, the framework contains the stream-based ST-IDW algorithms ST Shell and ST ak-Shell for high performance, parallel grid cell interpolation. Secondly, the proposed ST Predicate Stream Query Operator Framework (STP-SQO framework) efficiently evaluates value predicates over ST streams of ST continuous phenomena. The framework contains several stream-based predicate evaluation algorithms, including Region-Growing, Tile-based, and Phenomenon-Aware algorithms, that target predicate evaluation to regions with seed points and minimize the number of raster cells that are interpolated when evaluating value predicates. The performance of the proposed frameworks was assessed with regard to prediction accuracy of output results and runtime. The STI-SQO framework achieved a processing throughput of 250,000 observations in 2.5 s with a Normalized Root Mean Square Error under 0.19 using a 500×500 grid. The STP-SQO framework processed over 250,000 observations in under 0.25 s for predicate results covering less than 40% of the observation area, and the Scan Line Region Growing algorithm was consistently the fastest algorithm tested

A Survey on Service Route and Time Prediction in Instant Delivery: Taxonomy, Progress, and Prospects

Instant delivery services, such as food delivery and package delivery, have achieved explosive growth in recent years by providing customers with daily-life convenience. An emerging research area within these services is service Route\&Time Prediction (RTP), which aims to estimate the future service route as well as the arrival time of a given worker. As one of the most crucial tasks in those service platforms, RTP stands central to enhancing user satisfaction and trimming operational expenditures on these platforms. Despite a plethora of algorithms developed to date, there is no systematic, comprehensive survey to guide researchers in this domain. To fill this gap, our work presents the first comprehensive survey that methodically categorizes recent advances in service route and time prediction. We start by defining the RTP challenge and then delve into the metrics that are often employed. Following that, we scrutinize the existing RTP methodologies, presenting a novel taxonomy of them. We categorize these methods based on three criteria: (i) type of task, subdivided into only-route prediction, only-time prediction, and joint route\&time prediction; (ii) model architecture, which encompasses sequence-based and graph-based models; and (iii) learning paradigm, including Supervised Learning (SL) and Deep Reinforcement Learning (DRL). Conclusively, we highlight the limitations of current research and suggest prospective avenues. We believe that the taxonomy, progress, and prospects introduced in this paper can significantly promote the development of this field

Multigranularity Representations for Human Inter-Actions: Pose, Motion and Intention

Tracking people and their body pose in videos is a central problem in computer vision. Standard tracking representations reason about temporal coherence of detected people and body parts. They have difficulty tracking targets under partial occlusions or rare body poses, where detectors often fail, since the number of training examples is often too small to deal with the exponential variability of such configurations. We propose tracking representations that track and segment people and their body pose in videos by exploiting information at multiple detection and segmentation granularities when available, whole body, parts or point trajectories. Detections and motion estimates provide contradictory information in case of false alarm detections or leaking motion affinities. We consolidate contradictory information via graph steering, an algorithm for simultaneous detection and co-clustering in a two-granularity graph of motion trajectories and detections, that corrects motion leakage between correctly detected objects, while being robust to false alarms or spatially inaccurate detections. We first present a motion segmentation framework that exploits long range motion of point trajectories and large spatial support of image regions. We show resulting video segments adapt to targets under partial occlusions and deformations. Second, we augment motion-based representations with object detection for dealing with motion leakage. We demonstrate how to combine dense optical flow trajectory affinities with repulsions from confident detections to reach a global consensus of detection and tracking in crowded scenes. Third, we study human motion and pose estimation. We segment hard to detect, fast moving body limbs from their surrounding clutter and match them against pose exemplars to detect body pose under fast motion. We employ on-the-fly human body kinematics to improve tracking of body joints under wide deformations. We use motion segmentability of body parts for re-ranking a set of body joint candidate trajectories and jointly infer multi-frame body pose and video segmentation. We show empirically that such multi-granularity tracking representation is worthwhile, obtaining significantly more accurate multi-object tracking and detailed body pose estimation in popular datasets

A Review on Deep Learning Techniques for Video Prediction

The ability to predict, anticipate and reason about future outcomes is a key component of intelligent decision-making systems. In light of the success of deep learning in computer vision, deep-learning-based video prediction emerged as a promising research direction. Defined as a self-supervised learning task, video prediction represents a suitable framework for representation learning, as it demonstrated potential capabilities for extracting meaningful representations of the underlying patterns in natural videos. Motivated by the increasing interest in this task, we provide a review on the deep learning methods for prediction in video sequences. We firstly define the video prediction fundamentals, as well as mandatory background concepts and the most used datasets. Next, we carefully analyze existing video prediction models organized according to a proposed taxonomy, highlighting their contributions and their significance in the field. The summary of the datasets and methods is accompanied with experimental results that facilitate the assessment of the state of the art on a quantitative basis. The paper is summarized by drawing some general conclusions, identifying open research challenges and by pointing out future research directions.This work has been funded by the Spanish Government PID2019-104818RB-I00 grant for the MoDeaAS project, supported with Feder funds. This work has also been supported by two Spanish national grants for PhD studies, FPU17/00166, and ACIF/2018/197 respectively

Decentralized nation, solving the web identity crisis

The web of today whether you prefer to call it web 2.0, web 3.0, web 5.0 or even the metaverse is at a critical stage of evolution and challenge, largely centered around its crisis of identity. Like teenagers who cannot assess properly their reason for being and do not seem ready to take responsibility for their actions, we are constantly blaming the very system we are trying to get away from. To truly realize the benefits from innovation and technology, this crisis has to be resolved, not just through tactical solutions but through developments that enhance the sustainability of the web and its benefits. Significant strides are being made in the evolution of digital services enabled by technology, regulation, and the sheer pace of societal change. The journey to the decentralized web is mirroring the convergence of the physical and digital worlds across all economies and is increasingly embracing the digital native world. Technology has provided the foundational platform for individuals and entities to create and manage wealth, potentially without the need for big institutions. Ironically, despite all of the advancements, we are still facing an unprecedented and increasing wealth gap. Clearly, the system is broken, not just around the edges but at the very core of the democratic underpinning of our society. In this whitepaper, we propose how artificial intelligence on blockchain can be used to generate a new class of identity through direct human computer interaction. We demonstrate how this, combined with new perspectives for sustaining community and governance embedded within the use of blockchain technology, will underpin a sustainable solution to protect identity, authorship and privacy at the same time while contributing to restore trust amongst members of a future decentralized nation and hence contribute to solving the web most significant identity crisis.Comment: 11 pages, 1 figur

Road Traffic Congestion Analysis Via Connected Vehicles

La congestion routière est un état particulier de mobilité où les temps de déplacement augmentent et de plus en plus de temps est passé dans le véhicule. En plus d’être une expérience très stressante pour les conducteurs, la congestion a également un impact négatif sur l’environnement et l’économie. Dans ce contexte, des pressions sont exercées sur les autorités afin qu’elles prennent des mesures décisives pour améliorer le flot du trafic sur le réseau routier. En améliorant le flot, la congestion est réduite et la durée totale de déplacement des véhicules est réduite. D’une part, la congestion routière peut être récurrente, faisant référence à la congestion qui se produit régulièrement. La congestion non récurrente (NRC), quant à elle, dans un réseau urbain, est principalement causée par des incidents, des zones de construction, des événements spéciaux ou des conditions météorologiques défavorables. Les opérateurs d’infrastructure surveillent le trafic sur le réseau mais sont contraints à utiliser le moins de ressources possibles. Cette contrainte implique que l’état du trafic ne peut pas être mesuré partout car il n’est pas réaliste de déployer des équipements sophistiqués pour assurer la collecte précise des données de trafic et la détection en temps réel des événements partout sur le réseau routier. Alors certains emplacements où le flot de trafic doit être amélioré ne sont pas surveillés car ces emplacements varient beaucoup. D’un autre côté, de nombreuses études sur la congestion routière ont été consacrées aux autoroutes plutôt qu’aux régions urbaines, qui sont pourtant beaucoup plus susceptibles d’être surveillées par les autorités de la circulation. De plus, les systèmes actuels de collecte de données de trafic n’incluent pas la possibilité d’enregistrer des informations détaillées sur les événements qui surviennent sur la route, tels que les collisions, les conditions météorologiques défavorables, etc. Aussi, les études proposées dans la littérature ne font que détecter la congestion ; mais ce n’est pas suffisant, nous devrions être en mesure de mieux caractériser l’événement qui en est la cause. Les agences doivent comprendre quelle est la cause qui affecte la variabilité de flot sur leurs installations et dans quelle mesure elles peuvent prendre les actions appropriées pour atténuer la congestion.----------ABSTRACT: Road traffic congestion is a particular state of mobility where travel times increase and more and more time is spent in vehicles. Apart from being a quite-stressful experience for drivers, congestion also has a negative impact on the environment and the economy. In this context, there is pressure on the authorities to take decisive actions to improve the network traffic flow. By improving network flow, congestion is reduced and the total travel time of vehicles is decreased. In fact, congestion can be classified as recurrent and non-recurrent (NRC). Recurrent congestion refers to congestion that happens on a regular basis. Non-recurrent congestion in an urban network is mainly caused by incidents, workzones, special events and adverse weather. Infrastructure operators monitor traffic on the network while using the least possible resources. Thus, traffic state cannot be directly measured everywhere on the traffic road network. But the location where traffic flow needs to be improved varies highly and certainly, deploying highly sophisticated equipment to ensure the accurate estimation of traffic flows and timely detection of events everywhere on the road network is not feasible. Also, many studies have been devoted to highways rather than highly congested urban regions which are intricate, complex networks and far more likely to be monitored by the traffic authorities. Moreover, current traffic data collection systems do not incorporate the ability of registring detailed information on the altering events happening on the road, such as vehicle crashes, adverse weather, etc. Operators require external data sources to retireve this information in real time. Current methods only detect congestion but it’s not enough, we should be able to better characterize the event causing it. Agencies need to understand what is the cause affecting variability on their facilities and to what degree so that they can take the appropriate action to mitigate congestion

ICSEA 2021: the sixteenth international conference on software engineering advances

The Sixteenth International Conference on Software Engineering Advances (ICSEA 2021), held on October 3 - 7, 2021 in Barcelona, Spain, continued a series of events covering a broad spectrum of software-related topics. The conference covered fundamentals on designing, implementing, testing, validating and maintaining various kinds of software. The tracks treated the topics from theory to practice, in terms of methodologies, design, implementation, testing, use cases, tools, and lessons learnt. The conference topics covered classical and advanced methodologies, open source, agile software, as well as software deployment and software economics and education. The conference had the following tracks: Advances in fundamentals for software development Advanced mechanisms for software development Advanced design tools for developing software Software engineering for service computing (SOA and Cloud) Advanced facilities for accessing software Software performance Software security, privacy, safeness Advances in software testing Specialized software advanced applications Web Accessibility Open source software Agile and Lean approaches in software engineering Software deployment and maintenance Software engineering techniques, metrics, and formalisms Software economics, adoption, and education Business technology Improving productivity in research on software engineering Trends and achievements Similar to the previous edition, this event continued to be very competitive in its selection process and very well perceived by the international software engineering community. As such, it is attracting excellent contributions and active participation from all over the world. We were very pleased to receive a large amount of top quality contributions. We take here the opportunity to warmly thank all the members of the ICSEA 2021 technical program committee as well as the numerous reviewers. The creation of such a broad and high quality conference program would not have been possible without their involvement. We also kindly thank all the authors that dedicated much of their time and efforts to contribute to the ICSEA 2021. We truly believe that thanks to all these efforts, the final conference program consists of top quality contributions. This event could also not have been a reality without the support of many individuals, organizations and sponsors. We also gratefully thank the members of the ICSEA 2021 organizing committee for their help in handling the logistics and for their work that is making this professional meeting a success. We hope the ICSEA 2021 was a successful international forum for the exchange of ideas and results between academia and industry and to promote further progress in software engineering research