3,678 research outputs found
Real-time detection and tracking of multiple objects with partial decoding in H.264/AVC bitstream domain
In this paper, we show that we can apply probabilistic spatiotemporal
macroblock filtering (PSMF) and partial decoding processes to effectively
detect and track multiple objects in real time in H.264|AVC bitstreams with
stationary background. Our contribution is that our method cannot only show
fast processing time but also handle multiple moving objects that are
articulated, changing in size or internally have monotonous color, even though
they contain a chaotic set of non-homogeneous motion vectors inside. In
addition, our partial decoding process for H.264|AVC bitstreams enables to
improve the accuracy of object trajectories and overcome long occlusion by
using extracted color information.Comment: SPIE Real-Time Image and Video Processing Conference 200
Real-time moving object segmentation in H.264 compressed domain based on approximate reasoning
AbstractThis paper presents a real-time segmentation algorithm to obtain moving objects from the H.264 compressed domain. The proposed segmentation works with very little information and is based on two features of the H.264 compressed video: motion vectors associated to the macroblocks and decision modes. The algorithm uses fuzzy logic and allows to describe position, velocity and size of the detected regions in a comprehensive way, so the proposed approach works with low level information but manages highly comprehensive linguistic concepts. The performance of the algorithm is improved using dynamic design of fuzzy sets that avoids merge and split problems. Experimental results for several traffic scenes demonstrate the real-time performance and the encouraging results in diverse situations
Video Classification With CNNs: Using The Codec As A Spatio-Temporal Activity Sensor
We investigate video classification via a two-stream convolutional neural
network (CNN) design that directly ingests information extracted from
compressed video bitstreams. Our approach begins with the observation that all
modern video codecs divide the input frames into macroblocks (MBs). We
demonstrate that selective access to MB motion vector (MV) information within
compressed video bitstreams can also provide for selective, motion-adaptive, MB
pixel decoding (a.k.a., MB texture decoding). This in turn allows for the
derivation of spatio-temporal video activity regions at extremely high speed in
comparison to conventional full-frame decoding followed by optical flow
estimation. In order to evaluate the accuracy of a video classification
framework based on such activity data, we independently train two CNN
architectures on MB texture and MV correspondences and then fuse their scores
to derive the final classification of each test video. Evaluation on two
standard datasets shows that the proposed approach is competitive to the best
two-stream video classification approaches found in the literature. At the same
time: (i) a CPU-based realization of our MV extraction is over 977 times faster
than GPU-based optical flow methods; (ii) selective decoding is up to 12 times
faster than full-frame decoding; (iii) our proposed spatial and temporal CNNs
perform inference at 5 to 49 times lower cloud computing cost than the fastest
methods from the literature.Comment: Accepted in IEEE Transactions on Circuits and Systems for Video
Technology. Extension of ICIP 2017 conference pape
Reliable camera motion estimation from compressed MPEG videos using machine learning approach
As an important feature in characterizing video content, camera motion has been widely applied in various multimedia and computer vision applications. A novel method for fast and reliable estimation of camera motion from MPEG videos is proposed, using support vector machine for estimation in a regression model trained on a synthesized sequence. Experiments conducted on real sequences show that the proposed method yields much improved results in estimating camera motions while the difficulty in selecting valid macroblocks and motion vectors is skipped
Surveillance centric coding
PhDThe research work presented in this thesis focuses on the development of techniques
specific to surveillance videos for efficient video compression with higher processing
speed. The Scalable Video Coding (SVC) techniques are explored to achieve higher
compression efficiency. The framework of SVC is modified to support Surveillance
Centric Coding (SCC). Motion estimation techniques specific to surveillance videos
are proposed in order to speed up the compression process of the SCC.
The main contributions of the research work presented in this thesis are divided into
two groups (i) Efficient Compression and (ii) Efficient Motion Estimation. The
paradigm of Surveillance Centric Coding (SCC) is introduced, in which coding aims
to achieve bit-rate optimisation and adaptation of surveillance videos for storing and
transmission purposes. In the proposed approach the SCC encoder communicates
with the Video Content Analysis (VCA) module that detects events of interest in
video captured by the CCTV. Bit-rate optimisation and adaptation are achieved by
exploiting the scalability properties of the employed codec. Time segments
containing events relevant to surveillance application are encoded using high spatiotemporal
resolution and quality while the irrelevant portions from the surveillance
standpoint are encoded at low spatio-temporal resolution and / or quality. Thanks to
the scalability of the resulting compressed bit-stream, additional bit-rate adaptation is
possible; for instance for the transmission purposes. Experimental evaluation showed
that significant reduction in bit-rate can be achieved by the proposed approach
without loss of information relevant to surveillance applications.
In addition to more optimal compression strategy, novel approaches to performing
efficient motion estimation specific to surveillance videos are proposed and
implemented with experimental results. A real-time background subtractor is used to
detect the presence of any motion activity in the sequence. Different approaches for
selective motion estimation, GOP based, Frame based and Block based, are
implemented. In the former, motion estimation is performed for the whole group of
pictures (GOP) only when a moving object is detected for any frame of the GOP.
iii
While for the Frame based approach; each frame is tested for the motion activity and
consequently for selective motion estimation. The selective motion estimation
approach is further explored at a lower level as Block based selective motion
estimation. Experimental evaluation showed that significant reduction in
computational complexity can be achieved by applying the proposed strategy. In
addition to selective motion estimation, a tracker based motion estimation and fast
full search using multiple reference frames has been proposed for the surveillance
videos.
Extensive testing on different surveillance videos shows benefits of
application of proposed approaches to achieve the goals of the SCC
SAIVT-QUT@TRECVid 2012: Interactive surveillance event detection
In this paper, we propose an approach which attempts to solve the problem of surveillance event detection, assuming that we know the definition of the events. To facilitate the discussion, we first define two concepts. The event of interest refers to the event that the user requests the system to detect; and the background activities are any other events in the video corpus. This is an unsolved problem due to many factors as listed below: 1) Occlusions and clustering: The surveillance scenes which are of significant interest at locations such as airports, railway stations, shopping centers are often crowded, where occlusions and clustering of people are frequently encountered. This significantly affects the feature extraction step, and for instance, trajectories generated by object tracking algorithms are usually not robust under such a situation. 2) The requirement for real time detection: The system should process the video fast enough in both of the feature extraction and the detection step to facilitate real time operation. 3) Massive size of the training data set: Suppose there is an event that lasts for 1 minute in a video with a frame rate of 25fps, the number of frames for this events is 60X25 = 1500. If we want to have a training data set with many positive instances of the event, the video is likely to be very large in size (i.e. hundreds of thousands of frames or more). How to handle such a large data set is a problem frequently encountered in this application. 4) Difficulty in separating the event of interest from background activities: The events of interest often co-exist with a set of background activities. Temporal groundtruth typically very ambiguous, as it does not distinguish the event of interest from a wide range of co-existing background activities. However, it is not practical to annotate the locations of the events in large amounts of video data. This problem becomes more serious in the detection of multi-agent interactions, since the location of these events can often not be constrained to within a bounding box. 5) Challenges in determining the temporal boundaries of the events: An event can occur at any arbitrary time with an arbitrary duration. The temporal segmentation of events is difficult and ambiguous, and also affected by other factors such as occlusions
- …