15 research outputs found
Unsupervised video summarization framework using keyframe extraction and video skimming
Video is one of the robust sources of information and the consumption of
online and offline videos has reached an unprecedented level in the last few
years. A fundamental challenge of extracting information from videos is a
viewer has to go through the complete video to understand the context, as
opposed to an image where the viewer can extract information from a single
frame. Apart from context understanding, it almost impossible to create a
universal summarized video for everyone, as everyone has their own bias of
keyframe, e.g; In a soccer game, a coach person might consider those frames
which consist of information on player placement, techniques, etc; however, a
person with less knowledge about a soccer game, will focus more on frames which
consist of goals and score-board. Therefore, if we were to tackle problem video
summarization through a supervised learning path, it will require extensive
personalized labeling of data. In this paper, we attempt to solve video
summarization through unsupervised learning by employing traditional
vision-based algorithmic methodologies for accurate feature extraction from
video frames. We have also proposed a deep learning-based feature extraction
followed by multiple clustering methods to find an effective way of summarizing
a video by interesting key-frame extraction. We have compared the performance
of these approaches on the SumMe dataset and showcased that using deep
learning-based feature extraction has been proven to perform better in case of
dynamic viewpoint videos.Comment: 5 pages, 3 figures. Technical Repor
Attentive monitoring of multiple video streams driven by a Bayesian foraging strategy
In this paper we shall consider the problem of deploying attention to subsets
of the video streams for collating the most relevant data and information of
interest related to a given task. We formalize this monitoring problem as a
foraging problem. We propose a probabilistic framework to model observer's
attentive behavior as the behavior of a forager. The forager, moment to moment,
focuses its attention on the most informative stream/camera, detects
interesting objects or activities, or switches to a more profitable stream. The
approach proposed here is suitable to be exploited for multi-stream video
summarization. Meanwhile, it can serve as a preliminary step for more
sophisticated video surveillance, e.g. activity and behavior analysis.
Experimental results achieved on the UCR Videoweb Activities Dataset, a
publicly available dataset, are presented to illustrate the utility of the
proposed technique.Comment: Accepted to IEEE Transactions on Image Processin
Recommended from our members
Divide-and-conquer based summarization framework for extracting affective video content
YesRecent advances in multimedia technology have led to tremendous increases in the available volume of video data, thereby creating a major requirement for efficient systems to manage such huge data volumes. Video summarization is one of the key techniques for accessing and managing large video libraries. Video summarization can be used to extract the affective contents of a video sequence to generate a concise representation of its content. Human attention models are an efficient means of affective content extraction. Existing visual attention driven summarization frameworks have high computational cost and memory requirements, as well as a lack of efficiency in accurately perceiving human attention. To cope with these issues, we propose a divide-and-conquer based framework for an efficient summarization of big video data. We divide the original video data into shots, where an attention model is computed from each shot in parallel. Viewer's attention is based on multiple sensory perceptions, i.e., aural and visual, as well as the viewer's neuronal signals. The aural attention model is based on the Teager energy, instant amplitude, and instant frequency, whereas the visual attention model employs multi-scale contrast and motion intensity. Moreover, the neuronal attention is computed using the beta-band frequencies of neuronal signals. Next, an aggregated attention curve is generated using an intra- and inter-modality fusion mechanism. Finally, the affective content in each video shot is extracted. The fusion of multimedia and neuronal signals provides a bridge that links the digital representation of multimedia with the viewer’s perceptions. Our experimental results indicate that the proposed shot-detection based divide-and-conquer strategy mitigates the time and computational complexity. Moreover, the proposed attention model provides an accurate reflection of the user preferences and facilitates the extraction of highly affective and personalized summaries.Supported by the ICT R&D program of MSIP/IITP. [2014(R0112-14-1014), The Development of Open Platform for Service of Convergence Contents]