Semantic multimedia document adaptation with functional annotations

laborie2009aInternational audienceThe diversity of presentation contexts (such as mobile phones, PDAs) for multimedia documents requires the adaptation of document specifications. In an earlier work, we have proposed a semantic adaptation framework for multimedia documents. This framework captures the semantics of the document composition and transforms the relations between multimedia objects according to adaptation constraints. In this paper, we show that capturing only the document composition for adaptation is unsatisfactory because it leads to a limited form of adapted solutions. Hence, we propose to guide adaptation with functional annotations, i.e., annotations related to multimedia objects which express a function in the document. In order to validate this framework, we propose to use RDF descriptions from SMIL documents and adapt such documents with our interactive adaptation prototype

Multimedia document summarization based on a semantic adaptation framework

Qualitative representation and reasoning ; SMIL documentsInternational audienceThe multiplication of presentation contexts (such as mobile phones, PDAs) for multimedia documents requires the adaptation of document specifications. In an earlier work, a semantic framework for multimedia document adaptation was proposed. This framework deals with the semantics of the document composition by transforming the relations between multimedia objects. However, it was lacking the capability of suppressing multimedia objects. In this paper, we extend the proposed adaptation with this capability. Thanks to this extension, we present a method for summarizing multimedia documents. Moreover, when multimedia objects are removed, the resulted document satisfies some properties such as presentation contiguity. To validate our framework, we adapt standard multimedia documents such as SMIL documents

Adaptive Image Classification on Mobile Phones

The advent of high-performance mobile phones has opened up the opportunity to develop new context-aware applications for everyday life. In particular, applications for context-aware information retrieval in conjunction with image-based object recognition have become a focal area of recent research. In this thesis we introduce an adaptive mobile museum guidance system that allows visitors in a museum to identify exhibits by taking a picture with their mobile phone. Besides approaches to object recognition, we present different adaptation techniques that improve classification performance. After providing a comprehensive background of context-aware mobile information systems in general, we present an on-device object recognition algorithm and show how its classification performance can be improved by capturing multiple images of a single exhibit. To accomplish this, we combine the classification results of the individual pictures and consider the perspective relations among the retrieved database images. In order to identify multiple exhibits in pictures we present an approach that uses the spatial relationships among the objects in images. They make it possible to infer and validate the locations of undetected objects relative to the detected ones and additionally improve classification performance. To cope with environmental influences, we introduce an adaptation technique that establishes ad-hoc wireless networks among the visitors’ mobile devices to exchange classification data. This ensures constant classification rates under varying illumination levels and changing object placement. Finally, in addition to localization using RF-technology, we present an adaptation technique that uses user-generated spatio-temporal pathway data for person movement prediction. Based on the history of previously visited exhibits, the algorithm determines possible future locations and incorporates these predictions into the object classification process. This increases classification performance and offers benefits comparable to traditional localization approaches but without the need for additional hardware. Through multiple field studies and laboratory experiments we demonstrate the benefits of each approach and show how they influence the overall classification rate.Die Einführung von Mobiltelefonen mit eingebauten Sensoren wie Kameras, GPS oder Beschleunigungssensoren, sowie Kommunikationstechniken wie Bluetooth oder WLAN ermöglicht die Entwicklung neuer kontextsensitiver Anwendungen für das tägliche Leben. Insbesondere Applikationen im Bereich kontextsensitiver Informationsbeschaffung in Verbindung mit bildbasierter Objekterkennung sind in den Fokus der aktuellen Forschung geraten. Der Beitrag dieser Arbeit ist die Entwicklung eines bildbasierten, mobilen Museumsführersystems, welches unterschiedliche Adaptionstechniken verwendet, um die Objekterkennung zu verbessern. Es wird gezeigt, wie Ojekterkennungsalgorithmen auf Mobiltelefonen realisiert werden können und wie die Erkennungsrate verbessert wird, indem man zum Beispiel ad-hoc Netzwerke einsetzt oder Bewegungsvorhersagen von Personen berücksichtigt

A computer vision system for detecting and analysing critical events in cities

Whether for commuting or leisure, cycling is a growing transport mode in many cities worldwide. However, it is still perceived as a dangerous activity. Although serious incidents related to cycling leading to major injuries are rare, the fear of getting hit or falling hinders the expansion of cycling as a major transport mode. Indeed, it has been shown that focusing on serious injuries only touches the tip of the iceberg. Near miss data can provide much more information about potential problems and how to avoid risky situations that may lead to serious incidents. Unfortunately, there is a gap in the knowledge in identifying and analysing near misses. This hinders drawing statistically significant conclusions to provide measures for the built-environment that ensure a safer environment for people on bikes. In this research, we develop a method to detect and analyse near misses and their risk factors using artificial intelligence. This is accomplished by analysing video streams linked to near miss incidents within a novel framework relying on deep learning and computer vision. This framework automatically detects near misses and extracts their risk factors from video streams before analysing their statistical significance. It also provides practical solutions implemented in a camera with embedded AI (URBAN-i Box) and a cloud-based service (URBAN-i Cloud) to tackle the stated issue in the real-world settings for use by researchers, policy-makers, or citizens. The research aims to provide human-centred evidence that may enable policy-makers and planners to provide a safer built environment for cycling in London, or elsewhere. More broadly, this research aims to contribute to the scientific literature with the theoretical and empirical foundations of a computer vision system that can be utilised for detecting and analysing other critical events in a complex environment. Such a system can be applied to a wide range of events, such as traffic incidents, crime or overcrowding

Automatic Behavior Analysis and Understanding of Collision Processes Using Video Sensors

RÉSUMÉ La sécurité routière est un des problèmes de société les plus importants à cause des multiples impacts et coûts des accidents de la route. Traditionnellement, le diagnostic de sécurité repose principalement sur les données historiques de collision. Cette approche réactive mène à remédier au problème de sécurité après que ses impacts sur la société soit déjà réalisés. Les analystes de la sécurité et les décideurs doivent attendre jusqu'à ce qu'un nombre suffisant de collisions (ce qui demande d’attendre habituellement au moins trois ans) soit collecté pour analyser ou mettre en place des mesures d’amélioration de la sécurité routière. Les méthodes substituts (« surrogate ») d'analyse de la sécurité constituent une approche alternative proactive qui s'appuie sur l'observation d’événements « dangereux » sans collision, souvent appelé accidents « évités de justesse » (« near misses ») ou « conflits ». Parmi ces approches, les techniques de conflits de trafic (TCT) reposent sur la collecte des données de conflit par des observateurs sur le terrain qui interprètent leur sévérité. Par conséquent, les TCT souffrent des variations de jugement des observateurs, de la difficulté de mesurer les indicateurs de sécurité en temps réel par les observateurs, et du coût de la collecte des données.----------ABSTRACTTraffic safety is one of the most important social issues due to the multiple costs of collisions. Traditionally, safety diagnosis depends mainly on historical collision data. This reactive approach leads to remedy the existing safety problem after the materialization of the induced social cost. Safety analysts and decision makers must wait till a sufficient number of collisions (typically at least 3 years of collision data) is collected to analyze and to devise countermeasures. Surrogate safety analysis is an alternative and proactive approach that relies on the observation of traffic events without a collision, in particular “unsafe” events often called “near misses” or “conflicts”. Among these approaches, traffic conflict techniques (TCT) rely mainly on field observers to identify conflicts and interpret their severity. Consequently, TCTs suffer from the variations of observer judgement, the cost of collecting conflict data, and the difficulty of measuring safety indicators in real time by the observers