Rate-Distortion Optimized Video Summary Generation and Transmission Over Packet Lossy Networks

The goal of video summarization is to select key frames from a video sequence in order to generate an optimal summary that can accommodate constraints on viewing time, storage, or bandwidth. While video summary generation without transmission considerations has been studied extensively, the problem of rate-distortion optimized summary generation and transmission in a packet-lossy network has gained little attention. We consider the transmission of summarized video over a packet-lossy network such as the Internet. We depart from traditional rate control methods by not sacrificing the image quality of each transmitted frame but instead focusing on the frames that can be dropped without seriously affecting the quality of the video sequence. We take into account the packet loss probability, and use the end-to-end distortion to optimize the video quality given constraints on the temporal rate of the summary. Different network scenarios such as when a feedback channel is not available, and when a feedback channel is available with the possibility of retransmission, are considered. In each case, we assume a strict end-to-end delay constraint such that the summarized video can be viewed in real-time. We show simulation results for each case, and also discuss the case when the feedback delay may not be constant