Deep Video Generation, Prediction and Completion of Human Action Sequences

Current deep learning results on video generation are limited while there are only a few first results on video prediction and no relevant significant results on video completion. This is due to the severe ill-posedness inherent in these three problems. In this paper, we focus on human action videos, and propose a general, two-stage deep framework to generate human action videos with no constraints or arbitrary number of constraints, which uniformly address the three problems: video generation given no input frames, video prediction given the first few frames, and video completion given the first and last frames. To make the problem tractable, in the first stage we train a deep generative model that generates a human pose sequence from random noise. In the second stage, a skeleton-to-image network is trained, which is used to generate a human action video given the complete human pose sequence generated in the first stage. By introducing the two-stage strategy, we sidestep the original ill-posed problems while producing for the first time high-quality video generation/prediction/completion results of much longer duration. We present quantitative and qualitative evaluation to show that our two-stage approach outperforms state-of-the-art methods in video generation, prediction and video completion. Our video result demonstration can be viewed at https://iamacewhite.github.io/supp/index.htmlComment: Under review for CVPR 2018. Haoye and Chunyan have equal contributio

Quadrotor control for persistent surveillance of dynamic environments

Thesis (M.S.)--Boston UniversityThe last decade has witnessed many advances in the field of small scale unmanned aerial vehicles (UAVs). In particular, the quadrotor has attracted significant attention. Due to its ability to perform vertical takeoff and landing, and to operate in cluttered spaces, the quadrotor is utilized in numerous practical applications, such as reconnaissance and information gathering in unsafe or otherwise unreachable environments. This work considers the application of aerial surveillance over a city-like environment. The thesis presents a framework for automatic deployment of quadrotors to monitor and react to dynamically changing events. The framework has a hierarchical structure. At the top level, the UAVs perform complex behaviors that satisfy high- level mission specifications. At the bottom level, low-level controllers drive actuators on vehicles to perform the desired maneuvers. In parallel with the development of controllers, this work covers the implementation of the system into an experimental testbed. The testbed emulates a city using physical objects to represent static features and projectors to display dynamic events occurring on the ground as seen by an aerial vehicle. The experimental platform features a motion capture system that provides position data for UAVs and physical features of the environment, allowing for precise, closed-loop control of the vehicles. Experimental runs in the testbed are used to validate the effectiveness of the developed control strategies

Hierarchical morphological segmentation for image sequence coding

This paper deals with a hierarchical morphological segmentation algorithm for image sequence coding. Mathematical morphology is very attractive for this purpose because it efficiently deals with geometrical features such as size, shape, contrast, or connectivity that can be considered as segmentation-oriented features. The algorithm follows a top-down procedure. It first takes into account the global information and produces a coarse segmentation, that is, with a small number of regions. Then, the segmentation quality is improved by introducing regions corresponding to more local information. The algorithm, considering sequences as being functions on a 3-D space, directly segments 3-D regions. A 3-D approach is used to get a segmentation that is stable in time and to directly solve the region correspondence problem. Each segmentation stage relies on four basic steps: simplification, marker extraction, decision, and quality estimation. The simplification removes information from the sequence to make it easier to segment. Morphological filters based on partial reconstruction are proven to be very efficient for this purpose, especially in the case of sequences. The marker extraction identifies the presence of homogeneous 3-D regions. It is based on constrained flat region labeling and morphological contrast extraction. The goal of the decision is to precisely locate the contours of regions detected by the marker extraction. This decision is performed by a modified watershed algorithm. Finally, the quality estimation concentrates on the coding residue, all the information about the 3-D regions that have not been properly segmented and therefore coded. The procedure allows the introduction of the texture and contour coding schemes within the segmentation algorithm. The coding residue is transmitted to the next segmentation stage to improve the segmentation and coding quality. Finally, segmentation and coding examples are presented to show the validity and interest of the coding approach.Peer ReviewedPostprint (published version