Adaptive Temporal Encoding Network for Video Instance-level Human Parsing

Beyond the existing single-person and multiple-person human parsing tasks in static images, this paper makes the first attempt to investigate a more realistic video instance-level human parsing that simultaneously segments out each person instance and parses each instance into more fine-grained parts (e.g., head, leg, dress). We introduce a novel Adaptive Temporal Encoding Network (ATEN) that alternatively performs temporal encoding among key frames and flow-guided feature propagation from other consecutive frames between two key frames. Specifically, ATEN first incorporates a Parsing-RCNN to produce the instance-level parsing result for each key frame, which integrates both the global human parsing and instance-level human segmentation into a unified model. To balance between accuracy and efficiency, the flow-guided feature propagation is used to directly parse consecutive frames according to their identified temporal consistency with key frames. On the other hand, ATEN leverages the convolution gated recurrent units (convGRU) to exploit temporal changes over a series of key frames, which are further used to facilitate the frame-level instance-level parsing. By alternatively performing direct feature propagation between consistent frames and temporal encoding network among key frames, our ATEN achieves a good balance between frame-level accuracy and time efficiency, which is a common crucial problem in video object segmentation research. To demonstrate the superiority of our ATEN, extensive experiments are conducted on the most popular video segmentation benchmark (DAVIS) and a newly collected Video Instance-level Parsing (VIP) dataset, which is the first video instance-level human parsing dataset comprised of 404 sequences and over 20k frames with instance-level and pixel-wise annotations.Comment: To appear in ACM MM 2018. Code link: https://github.com/HCPLab-SYSU/ATEN. Dataset link: http://sysu-hcp.net/li

Towards High Performance Video Object Detection

There has been significant progresses for image object detection in recent years. Nevertheless, video object detection has received little attention, although it is more challenging and more important in practical scenarios. Built upon the recent works, this work proposes a unified approach based on the principle of multi-frame end-to-end learning of features and cross-frame motion. Our approach extends prior works with three new techniques and steadily pushes forward the performance envelope (speed-accuracy tradeoff), towards high performance video object detection

Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network

Depth estimation from a single image is a fundamental problem in computer vision. In this paper, we propose a simple yet effective convolutional spatial propagation network (CSPN) to learn the affinity matrix for depth prediction. Specifically, we adopt an efficient linear propagation model, where the propagation is performed with a manner of recurrent convolutional operation, and the affinity among neighboring pixels is learned through a deep convolutional neural network (CNN). We apply the designed CSPN to two depth estimation tasks given a single image: (1) To refine the depth output from state-of-the-art (SOTA) existing methods; and (2) to convert sparse depth samples to a dense depth map by embedding the depth samples within the propagation procedure. The second task is inspired by the availability of LIDARs that provides sparse but accurate depth measurements. We experimented the proposed CSPN over two popular benchmarks for depth estimation, i.e. NYU v2 and KITTI, where we show that our proposed approach improves in not only quality (e.g., 30% more reduction in depth error), but also speed (e.g., 2 to 5 times faster) than prior SOTA methods.Comment: 14 pages, 8 figures, ECCV 201

Heliospheric tracking of enhanced density structures of the 6 October 2010 CME

A Coronal Mass Ejection (CME) is an inhomogeneous structure consisting of different features which evolve differently with the propagation of the CME. Simultaneous heliospheric tracking of different observed features of a CME can improve our understanding about relative forces acting on them. It also helps to estimate accurately their arrival times at the Earth and identify them in in- situ data. This also enables to find association between remotely observed features and in-situ observations near the Earth. In this paper, we attempt to continuously track two density enhanced features, one at the front and another at the rear edge of the 6 October 2010 CME. This is achieved by using time-elongation maps constructed from STEREO/SECCHI observations. We derive the kinematics of the tracked features using various reconstruction methods. The estimated kinematics are used as inputs in the Drag Based Model (DBM) to estimate the arrival time of the tracked features of the CME at L1. On comparing the estimated kinematics as well as the arrival times of the remotely observed features with in-situ observations by ACE and Wind, we find that the tracked bright feature in the J-map at the rear edge of 6 October 2010 CME corresponds most probably to the enhanced density structure after the magnetic cloud detected by ACE and Wind. In-situ plasma and compositional parameters provide evidence that the rear edge density structure may correspond to a filament associated with the CME while the density enhancement at the front corresponds to the leading edge of the CME. Based on this single event study, we discuss the relevance and significance of heliospheric imager (HI) observations in identification of the three-part structure of the CME.Comment: 27 pages, 9 figures, accepted for Journal of Space Weather and Space Climate (SWSC

Omnipresent long-period intensity oscillations in open coronal structures

Quasi-periodic propagating disturbances in coronal structures have been interpreted as slow magneto-acoustic waves and/or periodic upflows. Here we aim to understand their nature from the observed properties using a three-hour imaging sequence from AIA/SDO in two different temperature channels. We also compare the characteristics with a simple wave model. We searched for propagating disturbances in open-loop structures at three different locations; a fan loop-structure off-limb, an on-disk plume-like structure and the plume/interplume regions in the north pole of the sun. In each of the subfield regions chosen to cover these structures, the time series at each pixel location was subjected to wavelet analysis to find the different periodicities. We then constructed powermaps in three different period ranges. We also constructed space-time maps for the on-disk plume structure to estimate the propagation speeds in different channels. We find propagating disturbances in all three structures. Powermaps indicate that the power in the long-period range is significant up to comparatively longer distances along the loop than that in the shorter periods. This nature is observed in all three structures. A detailed analysis on the on-disk plume structure gives consistently higher propagation speeds in the 193 \AA channel and also reveals spatial damping along the loop. The amplitude and the damping length values are lower in hotter channels, indicating their acoustic dependence. These properties can be explained very well with a propagating slow-wave model. We suggest that these disturbances are more likely to be caused by propagating slow magneto-acoustic waves than by high-speed quasi-periodic upflows. We find that intensity oscillations in longer periods are omnipresent at larger heights even in active regions.Comment: accepted for publication in A &