Real-time video stabilization without phantom movements for micro aerial vehicles

In recent times, micro aerial vehicles (MAVs) are becoming popular for several applications as rescue, surveillance, mapping, etc. Undesired motion between consecutive frames is a problem in a video recorded by MAVs. There are different approaches, applied in video post-processing, to solve this issue. However, there are only few algorithms able to be applied in real time. An additional and critical problem is the presence of false movements in the stabilized video. In this paper, we present a new approach of video stabilization which can be used in real time without generating false movements. Our proposal uses a combination of a low-pass filter and control action information to estimate the motion intention.Peer ReviewedPostprint (published version

Video Stabilization Using SIFT Features, Fuzzy Clustering, and Kalman Filtering

Video stabilization removes unwanted motion from video sequences, often caused by vibrations or other instabilities. This improves video viewability and can aid in detection and tracking in computer vision algorithms. We have developed a digital video stabilization process using scale-invariant feature transform (SIFT) features for tracking motion between frames. These features provide information about location and orientation in each frame. The orientation information is generally not available with other features, so we employ this knowledge directly in motion estimation. We use a fuzzy clustering scheme to separate the SIFT features representing camera motion from those representing the motion of moving objects in the scene. Each frame\u27s translation and rotation is accumulated over time, and a Kalman filter is applied to estimate the desired motion. We provide experimental results from several video sequences using peak signal-to-noise ratio (PSNR) and qualitative analysis to demonstrate the results of each design decision we made in the development of this video stabilization method

VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera

We present the first real-time method to capture the full global 3D skeletal pose of a human in a stable, temporally consistent manner using a single RGB camera. Our method combines a new convolutional neural network (CNN) based pose regressor with kinematic skeleton fitting. Our novel fully-convolutional pose formulation regresses 2D and 3D joint positions jointly in real time and does not require tightly cropped input frames. A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton. This makes our approach the first monocular RGB method usable in real-time applications such as 3D character control---thus far, the only monocular methods for such applications employed specialized RGB-D cameras. Our method's accuracy is quantitatively on par with the best offline 3D monocular RGB pose estimation methods. Our results are qualitatively comparable to, and sometimes better than, results from monocular RGB-D approaches, such as the Kinect. However, we show that our approach is more broadly applicable than RGB-D solutions, i.e. it works for outdoor scenes, community videos, and low quality commodity RGB cameras.Comment: Accepted to SIGGRAPH 201

高速ビジョンを用いたリアルタイムビデオモザイキングと安定化に関する研究

広島大学(Hiroshima University)博士(工学)Doctor of Engineeringdoctora

Source Camera Verification from Strongly Stabilized Videos

Image stabilization performed during imaging and/or post-processing poses one of the most significant challenges to photo-response non-uniformity based source camera attribution from videos. When performed digitally, stabilization involves cropping, warping, and inpainting of video frames to eliminate unwanted camera motion. Hence, successful attribution requires the inversion of these transformations in a blind manner. To address this challenge, we introduce a source camera verification method for videos that takes into account the spatially variant nature of stabilization transformations and assumes a larger degree of freedom in their search. Our method identifies transformations at a sub-frame level, incorporates a number of constraints to validate their correctness, and offers computational flexibility in the search for the correct transformation. The method also adopts a holistic approach in countering disruptive effects of other video generation steps, such as video coding and downsizing, for more reliable attribution. Tests performed on one public and two custom datasets show that the proposed method is able to verify the source of 23-30% of all videos that underwent stronger stabilization, depending on computation load, without a significant impact on false attribution

Novel Texture-based Probabilistic Object Recognition and Tracking Techniques for Food Intake Analysis and Traffic Monitoring

More complex image understanding algorithms are increasingly practical in a host of emerging applications. Object tracking has value in surveillance and data farming; and object recognition has applications in surveillance, data management, and industrial automation. In this work we introduce an object recognition application in automated nutritional intake analysis and a tracking application intended for surveillance in low quality videos. Automated food recognition is useful for personal health applications as well as nutritional studies used to improve public health or inform lawmakers. We introduce a complete, end-to-end system for automated food intake measurement. Images taken by a digital camera are analyzed, plates and food are located, food type is determined by neural network, distance and angle of food is determined and 3D volume estimated, the results are cross referenced with a nutritional database, and before and after meal photos are compared to determine nutritional intake. We compare against contemporary systems and provide detailed experimental results of our system\u27s performance. Our tracking systems consider the problem of car and human tracking on potentially very low quality surveillance videos, from fixed camera or high flying \acrfull{uav}. Our agile framework switches among different simple trackers to find the most applicable tracker based on the object and video properties. Our MAPTrack is an evolution of the agile tracker that uses soft switching to optimize between multiple pertinent trackers, and tracks objects based on motion, appearance, and positional data. In both cases we provide comparisons against trackers intended for similar applications i.e., trackers that stress robustness in bad conditions, with competitive results

Development Of A High Performance Mosaicing And Super-Resolution Algorithm

In this dissertation, a high-performance mosaicing and super-resolution algorithm is described. The scale invariant feature transform (SIFT)-based mosaicing algorithm builds an initial mosaic which is iteratively updated by the robust super resolution algorithm to achieve the final high-resolution mosaic. Two different types of datasets are used for testing: high altitude balloon data and unmanned aerial vehicle data. To evaluate our algorithm, five performance metrics are employed: mean square error, peak signal to noise ratio, singular value decomposition, slope of reciprocal singular value curve, and cumulative probability of blur detection. Extensive testing shows that the proposed algorithm is effective in improving the captured aerial data and the performance metrics are accurate in quantifying the evaluation of the algorithm

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

High-Precision Photogrammetry for Glaciology

Consumer-grade digital cameras have become ubiquitous tools for documenting short-term variability in the geosciences. However, these devices were not intended for precise timekeeping and surveying, and their use as such requires management of systematic and random errors that inevitably arise.This dissertation presents a suite of methods for registering the place and time of photographs in an absolute reference frame so that they may be analyzed and interpreted alongside other spatial and temporal data. The methods are tested on a 13-year record of 33,000 time-lapse photographs from Alaska's Columbia Glacier. This work provides insights into the capabilities and shortcomings of consumer-grade cameras as scientific instruments, the opportunistic approaches often needed to achieve the best results, and the potential of continuous high-frequency measurements for documenting rapid geomorphic processes.Subsecond-precision image capture times are achieved by measuring the offset to a reference clock display and accounting for the drift, precision, and reporting resolution of the camera clock. Two case studies illustrate the benefit of subsecond precision in contemporary investigations: georeferencing aerial photogrammetric surveys with camera positions time-interpolated from GPS tracklogs, and coupling videos of glacier-calving events to synchronous seismic waveforms. Retroactive dating of photographs, on the order of seconds to hours, is achieved by leveraging phenomena visible in the photographs -- namely, the positions of astronomical objects in the sky or the corresponding variations in solar radiation and sea level.Similarly, retroactive camera calibrations are achieved using surface and topographic features in the photographs -- specifically, point and line features of known absolute position, the motion of static features in images due to camera rotation, and the correspondences between real images and images synthesized from vertical imagery. Camera motion is corrected by computing globally optimal estimates of rotation over arbitrarily-long photographic sequences.Finally, a recently-developed tracking algorithm based on particle filtering theory is refined and applied to estimate Columbia Glacier velocities, their associated uncertainties, and the corresponding strain rate fields at 3-day intervals over a 13-year period, providing an unprecedented look at the seasonal and sub-seasonal variability of tidewater glacier dynamics over long time scales

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field