Ubiquitous Integration and Temporal Synchronisation (UbilTS) framework : a solution for building complex multimodal data capture and interactive systems

Contemporary Data Capture and Interactive Systems (DCIS) systems are tied in with various technical complexities such as multimodal data types, diverse hardware and software components, time synchronisation issues and distributed deployment configurations. Building these systems is inherently difficult and requires addressing of these complexities before the intended and purposeful functionalities can be attained. The technical issues are often common and similar among diverse applications. This thesis presents the Ubiquitous Integration and Temporal Synchronisation (UbiITS) framework, a generic solution to address the technical complexities in building DCISs. The proposed solution is an abstract software framework that can be extended and customised to any application requirements. UbiITS includes all fundamental software components, techniques, system level layer abstractions and reference architecture as a collection to enable the systematic construction of complex DCISs. This work details four case studies to showcase the versatility and extensibility of UbiITS framework’s functionalities and demonstrate how it was employed to successfully solve a range of technical requirements. In each case UbiITS operated as the core element of each application. Additionally, these case studies are novel systems by themselves in each of their domains. Longstanding technical issues such as flexibly integrating and interoperating multimodal tools, precise time synchronisation, etc., were resolved in each application by employing UbiITS. The framework enabled establishing a functional system infrastructure in these cases, essentially opening up new lines of research in each discipline where these research approaches would not have been possible without the infrastructure provided by the framework. The thesis further presents a sample implementation of the framework on a device firmware exhibiting its capability to be directly implemented on a hardware platform. Summary metrics are also produced to establish the complexity, reusability, extendibility, implementation and maintainability characteristics of the framework.Engineering and Physical Sciences Research Council (EPSRC) grants - EP/F02553X/1, 114433 and 11394

Calcul de trajectoires pour la préconisation de manoeuvres automobiles sur la base d'une perception multi-capteur (application à l'évitement de collision)

Les systèmes d aide à la conduite, en général, et plus particulièrement les systèmes d aide à l évitement de collision sont de plus en plus en présents dans les véhicules car ils ont un très fort potentiel de réduction du nombre d accidents de la circulation.En effet, ces systèmes ont pour rôle d assister le conducteur, voire de se substituer à lui lorsque la situation de conduite indique un risque de collision important. Cette thèse traite du développement de ces systèmes en abordant quelques problématiques rencontrées.Afin de réagir convenablement, le système a d abord besoin d une représentation aussi fidèle que possible de l environnement du véhicule. La perception est faite au moyen de capteurs extéroceptifs qui permettent de détecter les objets et d en mesurer divers paramètres selon leur principe de mesure. La fusion des données individuelles des capteurs permet d obtenir une information globale plus juste, plus certaine et plus variée. Ce travail traite en profondeur des méthodes de suivi d objets par fusion de données multi-capteur, multimodale au niveau piste. Les approches proposées ont été évaluées puis approuvées grâce à des données de roulage réel et sur des données de conduite simulées.Il est ensuite nécessaire de faire une analyse de la scène perçue au cours du temps afin d évaluer le risque de collision encouru par le véhicule porteur du système. Cette thèse propose des méthodes de prédiction de trajectoire et de calcul de probabilité de collision, à divers horizons temporels afin de quantifier le risque de collision et d établir ainsi divers niveaux d alerte au conducteur. Un simulateur de scénarios automobiles a été utilisé pour valider la cohérence des méthodes d analyse de scène.Enfin, lorsque le risque de collision atteint un seuil jugé critique, le système doit calculer une trajectoire d évitement de collision qui sera ensuite automatiquement exécutée. Les principales approches de planification de trajectoires ont été revues et un choix a été fait et motivé en accord avec le contexte de système d aide à la conduite.Driver assistant systems in general, and specially collision avoidance systems are more and more installed in recent vehicles because of their high potential in reducing the number road accidents. Indeed, those systems are designed to assist the driver or even to take its place when the risk of collision is very important. This thesis deals with the main challenges in the development of collision avoidance systems. In order to react in a convenient way, the system must, first, build a faithful representation of the environment of the ego-vehicle. Perception is made by means of exteroceptive sensors that detect objects and measure different parameters, depending on their measurement principle. The fusion of individual sensor data allows obtaining a global knowledge that is more accurate, more certain and more varied. This research work makes a deep exploration of high level multisensor, multimodal, multitarget tracking methods. The proposed approaches are evaluated and validated on real driving data and also on simulated scenarios. Then, the observed scene is continuously analyzed in order to evaluate the risk of collision on the ego-vehicle. The thesis proposes methods of vehicle trajectory prediction and methods to calculate the probability of collision at different prediction times. This allows defining different levels of alert to the driver. an automotive scenarion simulator is used to test and validate the proposed scene analysis approaches. Finally, when the risk of collision reaches a defined critical value, the system must compute a collision avoidance trajectory that will be automatically followed. The main approaches of trajectory planning have been revisited et one has chosen according to the context of driver assistant system.COMPIEGNE-BU (601592101) / SudocSudocFranceF

Temporal calibration in multisensor tracking setups

Abstract: Spatial tracking is one of the most challenging and important parts of mixed reality environments. Many applications, especially in the domain of Augmented Reality, rely on the fusion of several tracking systems in order to optimize the overall performance. While the topic of sensor fusion has already seen considerable interest, most results only deal with the integration of particular setups as opposed to dynamic sensor fusion setups. A crucial prerequisite for correct sensor fusion is the temporal alignment of the sensor signals, as sensors in general are not syn-chronized. We present a general method to calibrate the temporal offset between different sensors which can be used to perform on-line calibration. To show the correctness and the feasibility of this method, we evaluated various combinations of tracking sensors