Cycle-SUM: Cycle-consistent Adversarial LSTM Networks for Unsupervised Video Summarization

In this paper, we present a novel unsupervised video summarization model that requires no manual annotation. The proposed model termed Cycle-SUM adopts a new cycle-consistent adversarial LSTM architecture that can effectively maximize the information preserving and compactness of the summary video. It consists of a frame selector and a cycle-consistent learning based evaluator. The selector is a bi-direction LSTM network that learns video representations that embed the long-range relationships among video frames. The evaluator defines a learnable information preserving metric between original video and summary video and "supervises" the selector to identify the most informative frames to form the summary video. In particular, the evaluator is composed of two generative adversarial networks (GANs), in which the forward GAN is learned to reconstruct original video from summary video while the backward GAN learns to invert the processing. The consistency between the output of such cycle learning is adopted as the information preserving metric for video summarization. We demonstrate the close relation between mutual information maximization and such cycle learning procedure. Experiments on two video summarization benchmark datasets validate the state-of-the-art performance and superiority of the Cycle-SUM model over previous baselines.Comment: Accepted at AAAI 201

PKFSKC: PCA Based Key Frame Similarity Kernel Clustering Algorithm

针对基于内容的视频检索领域中,关键帧特征矩阵维度不同时的相似度计算问题,提出一种基于主成分分析的关键帧相似度核聚类检索算法。首先,针对任意具有不; 同数量关键帧的视频片段,提取特征向量并构造不同维度的特征矩阵。其次,基于PCA计算对特征矩阵进行SVD计算降维矩阵后,结合矩阵运算方法及核方法设; 计出一种视频关键帧相似度核聚类检索算法,并给出其加权改进形式。最后,通过测试视频标准库和人工视频片段的实验表明,该算法能更好地提视频高视频检索的; 效率。In the content-based video retrieval research, a PCA based key frame; similarity kernel clustering algorithm is proposed to calculate the; similarity of the feature matrix of video key frame with different; dimensions. Firstly, feature vectors and structure feature matrices with; different dimensions of any different video clip key frame are; extracted. Secondly, the dimension reduction matrix with SVD method; based on PCA algorithm is calculated, the key frame similarity kernel; clustering algorithm is proposed with the matrix calculation method and; the kernel method, and its improved weighted representation is proposed; as well. Finally, the simulation experiments on the standard test video; database and artificial video clip database show that the algorithm can; improve the efficiency of video retrieval.福建省软科学一般项目; 2016年虚拟现实技术与系统国家重点实验室立项; 厦门大学立项课题项

Video Forgery Detection: A Comprehensive Study of Inter and Intra Frame Forgery With Comparison of State-Of-Art

Availability of sophisticated and low-cost smart phones, digital cameras, camcorders, surveillance CCTV cameras are extensively used to create videos in our daily life. The prevalence of video sharing techniques presently available in the market are: YouTube, Facebook, Instagram, snapchat and many more are in utilization to share the information related to videos. Besides this, there are many software which can edit the content of video: Window Movie Maker, Video Editor, Adobe Photoshop etc., with this available software anyone can edit the video content which is called as “Forgery” if edited content is harmful. Usually, videos play a vital role in terms of proof in crime scene. The Victim is judged by the proof submitted by the lawyer to the court. Many such cases have evidenced that the video being submitted as proof is been forged. Checking the authentication of the video is most important before submitting as proof. There has been a rapid development in deep learning techniques which have created deepfake videos where faces are replaced with other faces which strongly made a belief of saying “Seeing is no longer believing”. The available software which can morph the faces are FakeApp, FaceSwap etc., the increased technology really made the Authentication of proofs very doubtful and un-trusty which are not accepted as proof without proper validation of the video. The survey gives the methods that are capable of accurately computing the videos and analyses to detect different kinds of forgeries. It has revealed that most of the existing methods are relying on number of tampered frames. The proposed techniques are with compression, double compression codec videos where research is being carried out from 2016 to present. This paper gives the comprehensive study of techniques, algorithms and applications designed and developed to detect forgery in videos