Online Video Deblurring via Dynamic Temporal Blending Network

State-of-the-art video deblurring methods are capable of removing non-uniform blur caused by unwanted camera shake and/or object motion in dynamic scenes. However, most existing methods are based on batch processing and thus need access to all recorded frames, rendering them computationally demanding and time consuming and thus limiting their practical use. In contrast, we propose an online (sequential) video deblurring method based on a spatio-temporal recurrent network that allows for real-time performance. In particular, we introduce a novel architecture which extends the receptive field while keeping the overall size of the network small to enable fast execution. In doing so, our network is able to remove even large blur caused by strong camera shake and/or fast moving objects. Furthermore, we propose a novel network layer that enforces temporal consistency between consecutive frames by dynamic temporal blending which compares and adaptively (at test time) shares features obtained at different time steps. We show the superiority of the proposed method in an extensive experimental evaluation.Comment: 10 page

A robust and efficient video representation for action recognition

This paper introduces a state-of-the-art video representation and applies it to efficient action recognition and detection. We first propose to improve the popular dense trajectory features by explicit camera motion estimation. More specifically, we extract feature point matches between frames using SURF descriptors and dense optical flow. The matches are used to estimate a homography with RANSAC. To improve the robustness of homography estimation, a human detector is employed to remove outlier matches from the human body as human motion is not constrained by the camera. Trajectories consistent with the homography are considered as due to camera motion, and thus removed. We also use the homography to cancel out camera motion from the optical flow. This results in significant improvement on motion-based HOF and MBH descriptors. We further explore the recent Fisher vector as an alternative feature encoding approach to the standard bag-of-words histogram, and consider different ways to include spatial layout information in these encodings. We present a large and varied set of evaluations, considering (i) classification of short basic actions on six datasets, (ii) localization of such actions in feature-length movies, and (iii) large-scale recognition of complex events. We find that our improved trajectory features significantly outperform previous dense trajectories, and that Fisher vectors are superior to bag-of-words encodings for video recognition tasks. In all three tasks, we show substantial improvements over the state-of-the-art results

A reconfigurable real-time morphological system for augmented vision

There is a significant number of visually impaired individuals who suffer sensitivity loss to high spatial frequencies, for whom current optical devices are limited in degree of visual aid and practical application. Digital image and video processing offers a variety of effective visual enhancement methods that can be utilised to obtain a practical augmented vision head-mounted display device. The high spatial frequencies of an image can be extracted by edge detection techniques and overlaid on top of the original image to improve visual perception among the visually impaired. Augmented visual aid devices require highly user-customisable algorithm designs for subjective configuration per task, where current digital image processing visual aids offer very little user-configurable options. This paper presents a highly user-reconfigurable morphological edge enhancement system on field-programmable gate array, where the morphological, internal and external edge gradients can be selected from the presented architecture with specified edge thickness and magnitude. In addition, the morphology architecture supports reconfigurable shape structuring elements and configurable morphological operations. The proposed morphology-based visual enhancement system introduces a high degree of user flexibility in addition to meeting real-time constraints capable of obtaining 93 fps for high-definition image resolution

MAVIS: Mobile Acquisition and VISualization - a professional tool for video recording on a mobile platform

Professional video recording is a complex process which often requires expensive cameras and large amounts of ancillary equipment. With the advancement of mobile technologies, cameras on mobile devices have improved to the point where the quality of their output is sometimes comparable to that obtained from a professional video camera and are often used in professional productions. However, tools that allow professional users to access the information they need to control the technical quality of their filming and make an informed decision about what they are recording are missing on mobile platforms. In this paper we present MAVIS (Mobile Acquisition and VISualization) a tool for professional filming on a mobile platform. MAVIS allows users to access information such as colour vectorscope, waveform monitor, false colouring, focus peaking and all other information that is needed to produce high quality professional videos. This is achieved by exploiting the capabilities of modern mobile GPUs though the use of a number of vertex and fragment shaders. Evaluation with professionals in the film industry shows that the app and its functionalities are well received and that the output and usability of the application align with professional standards

Block-Matching Optical Flow for Dynamic Vision Sensor- Algorithm and FPGA Implementation

Rapid and low power computation of optical flow (OF) is potentially useful in robotics. The dynamic vision sensor (DVS) event camera produces quick and sparse output, and has high dynamic range, but conventional OF algorithms are frame-based and cannot be directly used with event-based cameras. Previous DVS OF methods do not work well with dense textured input and are designed for implementation in logic circuits. This paper proposes a new block-matching based DVS OF algorithm which is inspired by motion estimation methods used for MPEG video compression. The algorithm was implemented both in software and on FPGA. For each event, it computes the motion direction as one of 9 directions. The speed of the motion is set by the sample interval. Results show that the Average Angular Error can be improved by 30\% compared with previous methods. The OF can be calculated on FPGA with 50\,MHz clock in 0.2\,us per event (11 clock cycles), 20 times faster than a Java software implementation running on a desktop PC. Sample data is shown that the method works on scenes dominated by edges, sparse features, and dense texture.Comment: Published in ISCAS 201

RRS Discovery Cruise 360, 19 Jan-02 Feb 2011. Trials of the Autosub LR AUV, HyBIS, PELAGRA, Ellsworth Camera and MYRTLE-X Lander systems

There were five main objectives for the trials cruise: The first tests of the Autosub Long Range AUV, testing of the HyBIS video guided grab system, testing of the MYRTLE-X Lander systems, testing of a deep camera system for the Lake Ellsworth probe and test deployments of the PELAGRA neutrally buoyant sediment capture drifters.The working area was about 300 miles south west of the Canary Islands, in international waters, over benthic plains of 4000 m depth, with some tests of the video systems over a isolated sea mount rising to 1200 m depth. Most of the objectives of the cruise where met, with successful diving and control of the Autosub LR, tests of the HyBIS and Ellsworth camera systems, and 3 deployments and recoveries of two PELAGRA floats. Several wire tests of MYRTLE-X systems were carried out, predominantly successful, but concerns over the release system prevented a deployment of the lander