Rate-Accuracy Trade-Off In Video Classification With Deep Convolutional Neural Networks

Advanced video classification systems decode video frames to derive the necessary texture and motion representations for ingestion and analysis by spatio-temporal deep convolutional neural networks (CNNs). However, when considering visual Internet-of-Things applications, surveillance systems and semantic crawlers of large video repositories, the video capture and the CNN-based semantic analysis parts do not tend to be co-located. This necessitates the transport of compressed video over networks and incurs significant overhead in bandwidth and energy consumption, thereby significantly undermining the deployment potential of such systems. In this paper, we investigate the trade-off between the encoding bitrate and the achievable accuracy of CNN-based video classification models that directly ingest AVC/H.264 and HEVC encoded videos. Instead of retaining entire compressed video bitstreams and applying complex optical flow calculations prior to CNN processing, we only retain motion vector and select texture information at significantly-reduced bitrates and apply no additional processing prior to CNN ingestion. Based on three CNN architectures and two action recognition datasets, we achieve 11%-94% saving in bitrate with marginal effect on classification accuracy. A model-based selection between multiple CNNs increases these savings further, to the point where, if up to 7% loss of accuracy can be tolerated, video classification can take place with as little as 3 kbps for the transport of the required compressed video information to the system implementing the CNN models

Video Stream Adaptation In Computer Vision Systems

Computer Vision (CV) has been deployed recently in a wide range of applications, including surveillance and automotive industries. According to a recent report, the market for CV technologies will grow to $33.3 billion by 2019. Surveillance and automotive industries share over 20% of this market. This dissertation considers the design of real-time CV systems with live video streaming, especially those over wireless and mobile networks. Such systems include video cameras/sensors and monitoring stations. The cameras should adapt their captured videos based on the events and/or available resources and time requirement. The monitoring station receives video streams from all cameras and run CV algorithms for decisions, warnings, control, and/or other actions. Real-time CV systems have constraints in power, computational, and communicational resources. Most video adaptation techniques considered the video distortion as the primary metric. In CV systems, however, the main objective is enhancing the event/object detection/recognition/tracking accuracy. The accuracy can essentially be thought of as the quality perceived by machines, as opposed to the human perceptual quality. High-Efficiency Video Coding (HEVC) is a recent encoding standard that seeks to address the limited communication bandwidth problem as a result of the popularity of High Definition (HD) videos. Unfortunately, HEVC adopts algorithms that greatly slow down the encoding process, and thus results in complications in real-time systems. This dissertation presents a method for adapting live video streams to limited and varying network bandwidth and energy resources. It analyzes and compares the rate-accuracy and rate-energy characteristics of various video streams adaptation techniques in CV systems. We model the video capturing, encoding, and transmission aspects and then provide an overall model of the power consumed by the video cameras and/or sensors. In addition to modeling the power consumption, we model the achieved bitrate of video encoding. We validate and analyze the power consumption models of each phase as well as the aggregate power consumption model through extensive experiments. The analysis includes examining individual parameters separately and examining the impacts of changing more than one parameter at a time. For HEVC, we develop an algorithm that predicts the size of the block without iterating through the exhaustive Rate Distortion Optimization (RDO) method. We demonstrate the effectiveness of the proposed algorithm in comparison with existing algorithms. The proposed algorithm achieves approximately 5 times the encoding speed of the RDO algorithm and 1.42 times the encoding speed of the fastest analyzed algorithm

Sensor-Assisted Global Motion Estimation for Efficient UAV Video Coding

This is the author accepted manuscript. The final version is available from IEEE via the DOI in this record.In this paper, we propose a novel video coding scheme to significantly reduce the coding complexity and enhance overall coding efficiency in videos acquired by high mobility devices such as unmanned aerial vehicles (UAVs). In order to reduce the encoded data bits and encoding time to facilitate real-time data transmission, as well as minimize the image distortion caused by the jitter of onboard camera, a sensor-assisted global motion estimation (GMV) algorithm is designed to calculate perspective transformation model and global motion vectors, which are used in both the inter-frame coding to improve the coding efficiency and intra-frame coding to reduce block search complexity. We conducted comprehensive simulation experiments on official HM-16.10 codec and the performance results show the proposed method can achieve faster block search by 50% to 60% speedup and lower bitrate by 15% to 30% compared with standard HEVC coding software

Data sharing in secure multimedia wireless sensor networks

© 2016 IEEE. The use of Multimedia Wireless Sensor Networks (MWSNs) is becoming common nowadays with a rapid growth in communication facilities. Similar to any other WSNs, these networks face various challenges while providing security, trust and privacy for user data. Provisioning of the aforementioned services become an uphill task especially while dealing with real-time streaming data. These networks operates with resource-constrained sensor nodes for days, months and even years depending on the nature of an application. The resource-constrained nature of these networks makes it difficult for the nodes to tackle real-time data in mission-critical applications such as military surveillance, forest fire monitoring, health-care and industrial automation. For a secured MWSN, the transmission and processing of streaming data needs to be explored deeply. The conventional data authentication schemes are not suitable for MWSNs due to the limitations imposed on sensor nodes in terms of battery power, computation, available bandwidth and storage. In this paper, we propose a novel quality-driven clustering-based technique for authenticating streaming data in MWSNs. Nodes with maximum energy are selected as Cluster Heads (CHs). The CHs collect data from member nodes and forward it to the Base Station (BS), thus preventing member nodes with low energy from dying soon and increasing life span of the underlying network. The proposed approach not only authenticates the streaming data but also maintains the quality of transmitted data. The proposed data authentication scheme coupled with an Error Concealment technique provides an energy-efficient and distortion-free real-time data streaming. The proposed scheme is compared with an unsupervised resources scenario. The simulation results demonstrate better network lifetime along with 21.34 dB gain in Peak Signal-to-Noise Ratio (PSNR) of received video data streams

Optimal coding unit decision for early termination in high efficiency video coding using enhanced whale optimization algorithm

Video compression is an emerging research topic in the field of block based video encoders. Due to the growth of video coding technologies, high efficiency video coding (HEVC) delivers superior coding performance. With the increased encoding complexity, the HEVC enhances the rate-distortion (RD) performance. In the video compression, the out-sized coding units (CUs) have higher encoding complexity. Therefore, the computational encoding cost and complexity remain vital concerns, which need to be considered as an optimization task. In this manuscript, an enhanced whale optimization algorithm (EWOA) is implemented to reduce the computational time and complexity of the HEVC. In the EWOA, a cosine function is incorporated with the controlling parameter A and two correlation factors are included in the WOA for controlling the position of whales and regulating the movement of search mechanism during the optimization and search processes. The bit streams in the Luma-coding tree block are selected using EWOA that defines the CU neighbors and is used in the HEVC. The results indicate that the EWOA achieves best bit rate (BR), time saving, and peak signal to noise ratio (PSNR). The EWOA showed 0.006-0.012 dB higher PSNR than the existing models in the real-time videos

Heterogeneous Video Transcoder for H.264/AVC to HEVC

The new video coding standard, High Efficiency Video Coding, was developed to succeed the current standard, H.264/Advance Video Coding. However, there is a lot of legacy content encoded with H.264. So the new efficient method is proposed for transcoding the H.264 encoded video into high efficiency video coding format. In proposed method, two stages are implemented. In training stage, transcoding is done using SSD method and different coding parameters or features are extracted from incoming H.264. In transcoding stage, the best mode of outgoing coding unit partitions are decided by calculating threshold value and optimum weight using extracted features. Then it is evaluated by doing experiments on different videos. DOI: 10.17762/ijritcc2321-8169.150615