ViSOR: VIdeo Surveillance On-line Repository for annotation retrieval

Aim of the Visor Project [1] is to gather and make freely available a repository of surveillance and video footages for the research community on pattern recognition and multimedia retrieval. Th

Scene Understanding For Real Time Processing Of Queries Over Big Data Streaming Video

With heightened security concerns across the globe and the increasing need to monitor, preserve and protect infrastructure and public spaces to ensure proper operation, quality assurance and safety, numerous video cameras have been deployed. Accordingly, they also need to be monitored effectively and efficiently. However, relying on human operators to constantly monitor all the video streams is not scalable or cost effective. Humans can become subjective, fatigued, even exhibit bias and it is difficult to maintain high levels of vigilance when capturing, searching and recognizing events that occur infrequently or in isolation. These limitations are addressed in the Live Video Database Management System (LVDBMS), a framework for managing and processing live motion imagery data. It enables rapid development of video surveillance software much like traditional database applications are developed today. Such developed video stream processing applications and ad hoc queries are able to reuse advanced image processing techniques that have been developed. This results in lower software development and maintenance costs. Furthermore, the LVDBMS can be intensively tested to ensure consistent quality across all associated video database applications. Its intrinsic privacy framework facilitates a formalized approach to the specification and enforcement of verifiable privacy policies. This is an important step towards enabling a general privacy certification for video surveillance systems by leveraging a standardized privacy specification language. With the potential to impact many important fields ranging from security and assembly line monitoring to wildlife studies and the environment, the broader impact of this work is clear. The privacy framework protects the general public from abusive use of surveillance technology; iii success in addressing the trust issue will enable many new surveillance-related applications. Although this research focuses on video surveillance, the proposed framework has the potential to support many video-based analytical applications

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

MusA: Using Indoor Positioning and Navigation to Enhance Cultural Experiences in a museum

In recent years there has been a growing interest into the use of multimedia mobile guides in museum environments. Mobile devices have the capabilities to detect the user context and to provide pieces of information suitable to help visitors discovering and following the logical and emotional connections that develop during the visit. In this scenario, location based services (LBS) currently represent an asset, and the choice of the technology to determine users' position, combined with the definition of methods that can effectively convey information, become key issues in the design process. In this work, we present MusA (Museum Assistant), a general framework for the development of multimedia interactive guides for mobile devices. Its main feature is a vision-based indoor positioning system that allows the provision of several LBS, from way-finding to the contextualized communication of cultural contents, aimed at providing a meaningful exploration of exhibits according to visitors' personal interest and curiosity. Starting from the thorough description of the system architecture, the article presents the implementation of two mobile guides, developed to respectively address adults and children, and discusses the evaluation of the user experience and the visitors' appreciation of these application

Multimedia

The nowadays ubiquitous and effortless digital data capture and processing capabilities offered by the majority of devices, lead to an unprecedented penetration of multimedia content in our everyday life. To make the most of this phenomenon, the rapidly increasing volume and usage of digitised content requires constant re-evaluation and adaptation of multimedia methodologies, in order to meet the relentless change of requirements from both the user and system perspectives. Advances in Multimedia provides readers with an overview of the ever-growing field of multimedia by bringing together various research studies and surveys from different subfields that point out such important aspects. Some of the main topics that this book deals with include: multimedia management in peer-to-peer structures & wireless networks, security characteristics in multimedia, semantic gap bridging for multimedia content and novel multimedia applications

Event Detection and Modelling for Security Application

PhD thesisThis thesis focuses on the design and implementation of a novel security domain surveillance system framework that incorporates multimodal information sources to assist the task of event detection from video and social media sources. The comprehensive framework consists of four modules including Data Source, Content Extraction, Parsing and Semantic Knowledge. The security domain ontology conceptual model is proposed for event representation and tailored in conformity with elementary aspects of event description. The adaptation of DOLCE foundational ontology promotes flexibility for heterogeneous ontologies to interoperate. The proposed mapping method using eXtensible Stylesheet Language Transformation (XSLT) stylesheet approach is presented to allow ontology enrichment and instance population to be executed efficiently. The dataset for visual semantic analysis utilizes video footage of 2011 London Riots obtained from Scotland Yard. The concepts person, face, police, car, fire, running, kicking and throwing are chosen to be analysed. The visual semantic analysis results demonstrate successful persons, actions and events detection in the video footage of riot events. For social semantic analysis, a collection of tweets from twitter channels that was actively reporting during the 2011 London Riots was compiled to create a Twitter corpus. The annotated data are mapped in the ontology based on six concepts: token, location, organization, sentence, verb, and noun. Several keywords related to the event that has been presented in the visual and social media sources are chosen to examine the correlation between both sources and to draw supplementary information regarding the event. The chosen keywords describe actions running, throwing, and kicking; activity attack, smash and loot; event fire; and location Hackney and Croydon. An experiment in respect to concept-noun relations are also been executed. The ontology-based visual and social media analysis yields a promising result in analysing long content surveillance videos and lengthy text corpus of social media user-generated content. Adopting ontology-based approach, the proposed novel security domain surveillance system framework enables a large amount of visual and social media data to be analysed systematically and automatically, and promotes a better method for event detection and understanding

Browse-to-search

This demonstration presents a novel interactive online shopping application based on visual search technologies. When users want to buy something on a shopping site, they usually have the requirement of looking for related information from other web sites. Therefore users need to switch between the web page being browsed and other websites that provide search results. The proposed application enables users to naturally search products of interest when they browse a web page, and make their even causal purchase intent easily satisfied. The interactive shopping experience is characterized by: 1) in session - it allows users to specify the purchase intent in the browsing session, instead of leaving the current page and navigating to other websites; 2) in context - -the browsed web page provides implicit context information which helps infer user purchase preferences; 3) in focus - users easily specify their search interest using gesture on touch devices and do not need to formulate queries in search box; 4) natural-gesture inputs and visual-based search provides users a natural shopping experience. The system is evaluated against a data set consisting of several millions commercial product images. © 2012 Authors

Video metadata extraction in a videoMail system

Currently the world swiftly adapts to visual communication. Online services like YouTube and Vine show that video is no longer the domain of broadcast television only. Video is used for different purposes like entertainment, information, education or communication. The rapid growth of today’s video archives with sparsely available editorial data creates a big problem of its retrieval. The humans see a video like a complex interplay of cognitive concepts. As a result there is a need to build a bridge between numeric values and semantic concepts. This establishes a connection that will facilitate videos’ retrieval by humans. The critical aspect of this bridge is video annotation. The process could be done manually or automatically. Manual annotation is very tedious, subjective and expensive. Therefore automatic annotation is being actively studied. In this thesis we focus on the multimedia content automatic annotation. Namely the use of analysis techniques for information retrieval allowing to automatically extract metadata from video in a videomail system. Furthermore the identification of text, people, actions, spaces, objects, including animals and plants. Hence it will be possible to align multimedia content with the text presented in the email message and the creation of applications for semantic video database indexing and retrieving

IoT Data Processing for Smart City and Semantic Web Applications

The world has been experiencing rapid urbanization over the last few decades, putting a strain on existing city infrastructure such as waste management, water supply management, public transport and electricity consumption. We are also seeing increasing pollution levels in cities threatening the environment, natural resources and health conditions. However, we must realize that the real growth lies in urbanization as it provides many opportunities to individuals for better employment, healthcare and better education. However, it is imperative to limit the ill effects of rapid urbanization through integrated action plans to enable the development of growing cities. This gave rise to the concept of a smart city in which all available information associated with a city will be utilized systematically for better city management. The proposed system architecture is divided in subsystems and is discussed in individual chapters. The first chapter introduces and gives overview to the reader of the complete system architecture. The second chapter discusses the data monitoring system and data lake system based on the oneM2M standards. DMS employs oneM2M as a middleware layer to achieve interoperability, and DLS uses a multi-tenant architecture with multiple logical databases, enabling efficient and reliable data management. The third chapter discusses energy monitoring and electric vehicle charging systems developed to illustrate the applicability of the oneM2M standards. The fourth chapter discusses the Data Exchange System based on the Indian Urban Data Exchange framework. DES uses IUDX standard data schema and open APIs to avoid data silos and enable secure data sharing. The fifth chapter discusses the 5D-IoT framework that provides uniform data quality assessment of sensor data with meaningful data descriptions