Distributed Coding/Decoding Complexity in Video Sensor Networks

Video Sensor Networks (VSNs) are recent communication infrastructures used to capture and transmit dense visual information from an application context. In such large scale environments which include video coding, transmission and display/storage, there are several open problems to overcome in practical implementations. This paper addresses the most relevant challenges posed by VSNs, namely stringent bandwidth usage and processing time/power constraints. In particular, the paper proposes a novel VSN architecture where large sets of visual sensors with embedded processors are used for compression and transmission of coded streams to gateways, which in turn transrate the incoming streams and adapt them to the variable complexity requirements of both the sensor encoders and end-user decoder terminals. Such gateways provide real-time transcoding functionalities for bandwidth adaptation and coding/decoding complexity distribution by transferring the most complex video encoding/decoding tasks to the transcoding gateway at the expense of a limited increase in bit rate. Then, a method to reduce the decoding complexity, suitable for system-on-chip implementation, is proposed to operate at the transcoding gateway whenever decoders with constrained resources are targeted. The results show that the proposed method achieves good performance and its inclusion into the VSN infrastructure provides an additional level of complexity control functionality

Selecting source image sensor nodes based on 2-hop information to improve image transmissions to mobile robot sinks in search \& rescue operations

We consider Robot-assisted Search $\&$ Rescue operations enhanced with some fixed image sensor nodes capable of capturing and sending visual information to a robot sink. In order to increase the performance of image transfer from image sensor nodes to the robot sinks we propose a 2-hop neighborhood information-based cover set selection to determine the most relevant image sensor nodes to activate. Then, in order to be consistent with our proposed approach, a multi-path extension of Greedy Perimeter Stateless Routing (called T-GPSR) wherein routing decisions are also based on 2-hop neighborhood information is proposed. Simulation results show that our proposal reduces packet losses, enabling fast packet delivery and higher visual quality of received images at the robot sink

Cross-layer Optimized Wireless Video Surveillance

A wireless video surveillance system contains three major components, the video capture and preprocessing, the video compression and transmission over wireless sensor networks (WSNs), and the video analysis at the receiving end. The coordination of different components is important for improving the end-to-end video quality, especially under the communication resource constraint. Cross-layer control proves to be an efficient measure for optimal system configuration. In this dissertation, we address the problem of implementing cross-layer optimization in the wireless video surveillance system. The thesis work is based on three research projects. In the first project, a single PTU (pan-tilt-unit) camera is used for video object tracking. The problem studied is how to improve the quality of the received video by jointly considering the coding and transmission process. The cross-layer controller determines the optimal coding and transmission parameters, according to the dynamic channel condition and the transmission delay. Multiple error concealment strategies are developed utilizing the special property of the PTU camera motion. In the second project, the binocular PTU camera is adopted for video object tracking. The presented work studied the fast disparity estimation algorithm and the 3D video transcoding over the WSN for real-time applications. The disparity/depth information is estimated in a coarse-to-fine manner using both local and global methods. The transcoding is coordinated by the cross-layer controller based on the channel condition and the data rate constraint, in order to achieve the best view synthesis quality. The third project is applied for multi-camera motion capture in remote healthcare monitoring. The challenge is the resource allocation for multiple video sequences. The presented cross-layer design incorporates the delay sensitive, content-aware video coding and transmission, and the adaptive video coding and transmission to ensure the optimal and balanced quality for the multi-view videos. In these projects, interdisciplinary study is conducted to synergize the surveillance system under the cross-layer optimization framework. Experimental results demonstrate the efficiency of the proposed schemes. The challenges of cross-layer design in existing wireless video surveillance systems are also analyzed to enlighten the future work. Adviser: Song C

Cross-layer Optimized Wireless Video Surveillance

A wireless video surveillance system contains three major components, the video capture and preprocessing, the video compression and transmission over wireless sensor networks (WSNs), and the video analysis at the receiving end. The coordination of different components is important for improving the end-to-end video quality, especially under the communication resource constraint. Cross-layer control proves to be an efficient measure for optimal system configuration. In this dissertation, we address the problem of implementing cross-layer optimization in the wireless video surveillance system. The thesis work is based on three research projects. In the first project, a single PTU (pan-tilt-unit) camera is used for video object tracking. The problem studied is how to improve the quality of the received video by jointly considering the coding and transmission process. The cross-layer controller determines the optimal coding and transmission parameters, according to the dynamic channel condition and the transmission delay. Multiple error concealment strategies are developed utilizing the special property of the PTU camera motion. In the second project, the binocular PTU camera is adopted for video object tracking. The presented work studied the fast disparity estimation algorithm and the 3D video transcoding over the WSN for real-time applications. The disparity/depth information is estimated in a coarse-to-fine manner using both local and global methods. The transcoding is coordinated by the cross-layer controller based on the channel condition and the data rate constraint, in order to achieve the best view synthesis quality. The third project is applied for multi-camera motion capture in remote healthcare monitoring. The challenge is the resource allocation for multiple video sequences. The presented cross-layer design incorporates the delay sensitive, content-aware video coding and transmission, and the adaptive video coding and transmission to ensure the optimal and balanced quality for the multi-view videos. In these projects, interdisciplinary study is conducted to synergize the surveillance system under the cross-layer optimization framework. Experimental results demonstrate the efficiency of the proposed schemes. The challenges of cross-layer design in existing wireless video surveillance systems are also analyzed to enlighten the future work. Adviser: Song C

Flexi-WVSNP-DASH: A Wireless Video Sensor Network Platform for the Internet of Things

abstract: Video capture, storage, and distribution in wireless video sensor networks (WVSNs) critically depends on the resources of the nodes forming the sensor networks. In the era of big data, Internet of Things (IoT), and distributed demand and solutions, there is a need for multi-dimensional data to be part of the Sensor Network data that is easily accessible and consumable by humanity as well as machinery. Images and video are expected to become as ubiquitous as is the scalar data in traditional sensor networks. The inception of video-streaming over the Internet, heralded a relentless research for effective ways of distributing video in a scalable and cost effective way. There has been novel implementation attempts across several network layers. Due to the inherent complications of backward compatibility and need for standardization across network layers, there has been a refocused attention to address most of the video distribution over the application layer. As a result, a few video streaming solutions over the Hypertext Transfer Protocol (HTTP) have been proposed. Most notable are Apple’s HTTP Live Streaming (HLS) and the Motion Picture Experts Groups Dynamic Adaptive Streaming over HTTP (MPEG-DASH). These frameworks, do not address the typical and future WVSN use cases. A highly flexible Wireless Video Sensor Network Platform and compatible DASH (WVSNP-DASH) are introduced. The platform's goal is to usher video as a data element that can be integrated into traditional and non-Internet networks. A low cost, scalable node is built from the ground up to be fully compatible with the Internet of Things Machine to Machine (M2M) concept, as well as the ability to be easily re-targeted to new applications in a short time. Flexi-WVSNP design includes a multi-radio node, a middle-ware for sensor operation and communication, a cross platform client facing data retriever/player framework, scalable security as well as a cohesive but decoupled hardware and software design.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201

Sweet Streams are Made of This: The System Engineer's View on Energy Efficiency in Video Communications

In recent years, the global use of online video services has increased rapidly. Today, a manifold of applications, such as video streaming, video conferencing, live broadcasting, and social networks, make use of this technology. A recent study found that the development and the success of these services had as a consequence that, nowadays, more than 1% of the global greenhouse-gas emissions are related to online video, with growth rates close to 10% per year. This article reviews the latest findings concerning energy consumption of online video from the system engineer's perspective, where the system engineer is the designer and operator of a typical online video service. We discuss all relevant energy sinks, highlight dependencies with quality-of-service variables as well as video properties, review energy consumption models for different devices from the literature, and aggregate these existing models into a global model for the overall energy consumption of a generic online video service. Analyzing this model and its implications, we find that end-user devices and video encoding have the largest potential for energy savings. Finally, we provide an overview of recent advances in energy efficiency improvement for video streaming and propose future research directions for energy-efficient video streaming services.Comment: 16 pages, 5 figures, accepted for IEEE Circuits and Systems Magazin

Interface Homem-Máquina Multi Robótica em Unity3D

More than ever the use of autonomous vehicles to accomplish objectives deemed too dangerous or even impossible by human standards is increasing. This demand puts to the test our capabilities for managing teams of multiple robots and creating intuitive interactions with these teams is a must. Creating means to abstract and condense the information that reaches the end user into a single kit of software would improve its manageability considerably. The development of a centralized graphical user interface is proposed to alleviate the workload of the human operator. This interface is thought out to be simple in delivering its information taking cues from video games, a well known industry in studying the theory behind the creation of user interfaces. Sensorial information is abstracted in a graphical perspective much like the attributes of a character inside a video game. The Unity game engine was used to implement such an interface, integrating ROS with a layer of DDS to manage the communications while providing QoS settings. The DDS solves the problem of multiple ROS masters by setting up a separate network where users can connect and disconnect seamlessly from the network, without the need to restart roscore on each machine. Interactions between these two software is made by using websockets on a local network. Visual representations of the sensors onboard the autonomous vehicles transform the huge stream of data into human understandable formats for immediate response by the operator. Dynamic generation of terrain was accomplished by the use of LiDAR and side-scan sensors, if available, to map the surroundings, while Mapbox provided prefetched terrain data from OpenStreetMaps.Mais do que nunca, o uso de veículos autónomos para cumprir objectivos considerados demasiado perigosos ou até mesmo impossíveis segundo os padrões humanos tem vindo a aumentar. Este requerimento testa as nossas capacidades de gestão de equipas de múltiplos robôs e torna a criação de interações intuitivas com estas equipas numa necessidade. Criar meios de abstrair e condensar a informação que chega ao utilizador final num só pacote de software iria melhorar a sua gestão consideravelmente. O desenvolvimento de uma interface gráfica centralizada é proposta de modo a aliviar a carga de trabalho do operador humano. Esta interface é pensada para transmitir a sua informação como um vídeo jogo, sendo que esta é uma indústria que conhecida pelo seu estudo de interfaces de utilizador. Informação sensorial é abstraída com uma perspectiva gráfica tal como os atributos de uma personagem de um vídeo jogo. O motor de jogo Unity foi o utilizado para implementar tal interface integrando funcionalidades de ROS com uma camada de DDS, responsável pela gestão das comunicações, fornecendo opções de QoS. O DDS resolve o problema de múltiplos ROS master estabelecendo uma rede separada em que os utilizadores podem conectar-se e desconectar-se simultaneamente sem haver a necessidade de reiniciar o roscore em cada máquina. Interações entre os dois software é efetuada através de websockets numa rede local. Representações visuais dos sensores a bordo dos veículos autónomos transformam os enormes fluxos de dados em formatos facilmente compreensíveis por humanos para resposta imediata por parte do operador. Geração dinâmica de ambientes virtuais foi tornado possível com recurso a sensores como LiDAR e side-scan, caso existam, enquanto que API’s como Mapbox e OpenStreetMaps forneceram dados estáticos destes ambientes