Broadcasting Convolutional Network for Visual Relational Reasoning

In this paper, we propose the Broadcasting Convolutional Network (BCN) that extracts key object features from the global field of an entire input image and recognizes their relationship with local features. BCN is a simple network module that collects effective spatial features, embeds location information and broadcasts them to the entire feature maps. We further introduce the Multi-Relational Network (multiRN) that improves the existing Relation Network (RN) by utilizing the BCN module. In pixel-based relation reasoning problems, with the help of BCN, multiRN extends the concept of `pairwise relations' in conventional RNs to `multiwise relations' by relating each object with multiple objects at once. This yields in O(n) complexity for n objects, which is a vast computational gain from RNs that take O(n^2). Through experiments, multiRN has achieved a state-of-the-art performance on CLEVR dataset, which proves the usability of BCN on relation reasoning problems.Comment: Accepted paper at ECCV 2018. 24 page

The last gasps of VY CMa: Aperture synthesis and adaptive optics imagery

We present new observations of the red supergiant VY CMa at 1.25 micron, 1.65 micron, 2.26 micron, 3.08 micron and 4.8 micron. Two complementary observational techniques were utilized: non-redundant aperture masking on the 10-m Keck-I telescope yielding images of the innermost regions at unprecedented resolution, and adaptive optics imaging on the ESO 3.6-m telescope at La Silla attaining extremely high (~10^5) peak-to-noise dynamic range over a wide field. For the first time the inner dust shell has been resolved in the near-infrared to reveal a one-sided extension of circumstellar emission within 0.1" (~15 R_star) of the star. The line-of-sight optical depths of the circumstellar dust shell at 1.65 micron, 2.26 micron, and 3.08 micron have been estimated to be 1.86 +/- 0.42, 0.85 +/- 0.20, and 0.44 +/- 0.11. These new results allow the bolometric luminosity of VY~CMa to be estimated independent of the dust shell geometry, yielding L_star ~ 2x10^5 L_sun. A variety of dust condensations, including a large scattering plume and a bow-shaped dust feature, were observed in the faint, extended nebula up to 4" from the central source. While the origin of the nebulous plume remains uncertain, a geometrical model is developed assuming the plume is produced by radially-driven dust grains forming at a rotating flow insertion point with a rotational period between 1200-4200 years, which is perhaps the stellar rotational period or the orbital period of an unseen companion.Comment: 25 pages total with 1 table and 5 figures. Accepted by Astrophysical Journal (to appear in February 1999

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Localization and recognition of the scoreboard in sports video based on SIFT point matching

In broadcast sports video, the scoreboard is attached at a fixed location in the video and generally the scoreboard always exists in all video frames in order to help viewers to understand the match’s progression quickly. Based on these observations, we present a new localization and recognition method for scoreboard text in sport videos in this paper. The method first matches the Scale Invariant Feature Transform (SIFT) points using a modified matching technique between two frames extracted from a video clip and then localizes the scoreboard by computing a robust estimate of the matched point cloud in a two-stage non-scoreboard filter process based on some domain rules. Next some enhancement operations are performed on the localized scoreboard, and a Multi-frame Voting Decision is used. Both aim to increasing the OCR rate. Experimental results demonstrate the effectiveness and efficiency of our proposed method

Structured Light-Based 3D Reconstruction System for Plants.

Camera-based 3D reconstruction of physical objects is one of the most popular computer vision trends in recent years. Many systems have been built to model different real-world subjects, but there is lack of a completely robust system for plants. This paper presents a full 3D reconstruction system that incorporates both hardware structures (including the proposed structured light system to enhance textures on object surfaces) and software algorithms (including the proposed 3D point cloud registration and plant feature measurement). This paper demonstrates the ability to produce 3D models of whole plants created from multiple pairs of stereo images taken at different viewing angles, without the need to destructively cut away any parts of a plant. The ability to accurately predict phenotyping features, such as the number of leaves, plant height, leaf size and internode distances, is also demonstrated. Experimental results show that, for plants having a range of leaf sizes and a distance between leaves appropriate for the hardware design, the algorithms successfully predict phenotyping features in the target crops, with a recall of 0.97 and a precision of 0.89 for leaf detection and less than a 13-mm error for plant size, leaf size and internode distance