A multi-modal dance corpus for research into interaction between humans in virtual environments

We present a new, freely available, multimodal corpus for research into, amongst other areas, real-time realistic interaction between humans in online virtual environments. The specific corpus scenario focuses on an online dance class application scenario where students, with avatars driven by whatever 3D capture technology is locally available to them, can learn choreographies with teacher guidance in an online virtual dance studio. As the dance corpus is focused on this scenario, it consists of student/teacher dance choreographies concurrently captured at two different sites using a variety of media modalities, including synchronised audio rigs, multiple cameras, wearable inertial measurement devices and depth sensors. In the corpus, each of the several dancers performs a number of fixed choreographies, which are graded according to a number of specific evaluation criteria. In addition, ground-truth dance choreography annotations are provided. Furthermore, for unsynchronised sensor modalities, the corpus also includes distinctive events for data stream synchronisation. The total duration of the recorded content is 1 h and 40 min for each single sensor, amounting to 55 h of recordings across all sensors. Although the dance corpus is tailored specifically for an online dance class application scenario, the data is free to download and use for any research and development purposes

Evaluation of Stereo Video Coding Schemes for Mobile Devices

Mobile devices such as mobile phones, personal digital assistants and personal video/game players are somehow converging and getting more powerful, thus enabling 3D mobile devices a reality. In order to store or transmit stereo video in these devices, coding techniques from both monoscopic video coding and multi-view video coding can be used. In this work, we analyze the possible stereoscopic encoding schemes for mobile devices. We have used rate-distortion curves for coding efficiency and decoding speed tests for decoder complexity. Depending on the processing power and memory of the mobile device, we concluded to use two of the settings used in our experiments

A multi-view video codec based on H.264

H.264 is the current state-of-the-art monoscopic video codec providing almost twice the coding efficiency with the same quality comparing the previous codecs. With the increasing interest in 3D TV, multi-view video sequences that are provided by multiple cameras capturing the three dimensional objects and/or scene are more widely used. Compressing multi-view sequences independently with H.264 (simulcast) is not efficient since the redundancy between the closer cameras is not exploited. In order to reduce these redundancies, we propose a Multi-View Video Codec based on H.264 using disparity estimation/compensation as well as motion estimation/compensation. In order to effectively search for disparity/motion without increasing computational complexity, we modified the buffering structure of H.264 and implemented several referencing modes. Our results show that for closely located cameras, our codec outperforms simulcast H.264 coding. For sparsely located cameras, our method can still improve coding gain depending on the video characteristics

OPTIMIZATION OF ENCODING AND ERROR PROTECTION PARAMETERS FOR 3D VIDEO BROADCAST OVER DVB-H

In this study, we propose a heuristic methodology for modeling the end-to-end distortion characteristics of an error resilient broadcast system for 3D video overDigital Video Broadcasting -Handheld (DVB-H). We also use this model to optimally select the parameters of the video encoder and the error correction scheme, namely, Multi Protocol Encapsulation Forward Error Correction (MPE-FEC), minimizing the overall distortion. The proposed method models the RQ curve of video encoder and performance of channel codec to jointly derive the optimal encoder bit rates and unequal error protection (UEP) rates specific to the 3D video broadcast. Moreover, the distortion on the 3D video quality caused by packet losses and the loss rate of the channel is estimated. Finally, with the use of analytical models and estimated single packet loss distortions, end-to-end distortions are minimized and optimal encoder bit rates and UEP rates are obtained

Camera tamper detection using wavelet analysis for video surveillance

It is generally accepted that video surveillance system operators lose their concentration after a short period of time and may miss important events taking place. In addition, many surveillance systems are frequently left unattended. Because of these reasons, automated analysis of the live video feed and automatic detection of suspicious activity have recently gained importance. To prevent capture of their images, criminals resort to several techniques such as deliberately obscuring the camera view, covering the lens with a foreign object, spraying or defocusing the camera lens. In this paper, we propose some computationally efficient wavelet domain methods for rapid camera tamper detection and identify some real-life problems and propose solutions to these

Camera Sabotage Discovery for Video Surveillance Applications

Son yıllarda video gözetim uygulamaları için kullanılan kamera sayılarında kayda değer artış olmuştur. Bu kameraların amaçlandığı gibi çalışıyor olması anlamlı bilgi yakalaması önemlidir. Suç işleyecek kişiler kamera önünü kapatarak, boya püskürterek ya da kamera odağını bozarak görüntülerinin ve eylemlerinin kaydedilmesini engellemek yoluna başvurmaktadır. Kameraları gözleyen birisinin olmaması ya da dikkatinin dağınık olması sabotajın fark edilememesine neden olur ve sistem normal olarak çalışıyor ve kaydediyor olsa bile kayıtların kullanılamaz olmasına yolaçar. Bu bildiride, dalgacık alanında arkaplan çıkarımı yöntemi kullanılarak kamerada görüş azalması ve kamera önünün kapatılması durumlarının gerçek zamanlı kestirimi önerilmektedir. Ayrıca, sistemin gerçek hayat koşullarında daha güvenilir çalışması için bazı yöntemler de önerilmiştir

MULTI-THREADED ARCHITECTURES AND BENCHMARK TESTS FOR REAL-TIME MULTI-VIEW VIDEO DECODING

3D video based on multi-view representations is becoming widely popular. Real-time encoding/decoding of such video is an important concern as the number and resolution of views increase. We present systematic methods for design and optimization of real-time multi-view video encoding/decoding algorithms using multi-core processors and provide benchmark results. The proposed multi-core decoding architectures are fully compliant with the current JVT-MVC international standard, and enable multi-threaded processing with negligible loss of encoding efficiency. Benchmark results show that multi-core processors and multi-threading decoding is necessary for real-time multi-view video decoding and display

SUBJECTIVE EVALUATION OF EFFECTS OF SPECTRAL AND SPATIAL REDUNDANCY REDUCTION ON STEREO IMAGES

Human visual system is more sensitive to luminance than to chrominance. In order to reduce information that is not perceived by human visual system, color channels are downsampled while keeping luminance as original. Similarly in stereo case, human visual system uses high frequency information from the high resolution image of the mixed resolution image pair. By downsampling one of the pair, higher compression is achieved in stereo image coding. In this paper, we have examined downsampling color channels in higher ratios in color stereo image pairs. In our experiments, we have used “double-stimulus continuous-quality scale ” (DSCQS) method. We have found out that the depth perception is not changed by compression or filtering. However, in order to keep perceived image quality similar to the original stereo pair, filtering should be applied to chrominance but not to luminance channels. 1

Architectures for multi-threaded MVC-compliant multi-view video decoding and benchmark tests

3D video based on stereo/multi-view representations is becoming widely popular. Real-time encoding/decoding of such video is an important concern as the number and spatial/temporal resolution of views increase. We present a systematic method for design and optimization of multi-threaded multi-view video encoding/decoding algorithms using multi-core processors and provide benchmark results for real-time decoding. The proposed multi-core decoding architectures are compliant with the current MVC extension of H.264/AVC international standard, and enable multi-threaded processing with negligible loss of encoding efficiency and minimum processing overhead. Benchmark results show that multi-core processors and multi-threading decoding are necessary for real-time high-definition multi-view video decoding and display

Rate-distortion optimization for stereoscopic video streaming with unequal error protection

We consider an error-resilient stereoscopic streaming system that uses an H.264-based multiview video codec and a rateless Raptor code for recovery from packet losses. One aim of the present work is to suggest a heuristic methodology for modeling the end-to-end rate-distortion (RD) characteristic of such a system. Another aim is to show how to make use of such a model to optimally select the parameters of the video codec and the Raptor code to minimize the overall distortion. Specifically, the proposed system models the RD curve of video encoder and performance of channel codec to jointly derive the optimal encoder bit rates and unequal error protection (UEP) rates specific to the layered stereoscopic video streaming. We define analytical RD curve modeling for each layer that includes the interdependency of these layers. A heuristic analytical model of the performance of Raptor codes is also defined. Furthermore, the distortion on the stereoscopic video quality caused by packet losses is estimated. Finally, analytical models and estimated single-packet loss distortions are used to minimize the end-to-end distortion and to obtain optimal encoder bit rates and UEP rates. The simulation results clearly demonstrate the significant quality gain against the nonoptimized schemes