Mixed Hierarchy Network for Image Restoration

Image restoration is a long-standing low-level vision problem, e.g., deblurring and deraining. In the process of image restoration, it is necessary to consider not only the spatial details and contextual information of restoration to ensure the quality, but also the system complexity. Although many methods have been able to guarantee the quality of image restoration, the system complexity of the state-of-the-art (SOTA) methods is increasing as well. Motivated by this, we present a mixed hierarchy network that can balance these competing goals. Our main proposal is a mixed hierarchy architecture, that progressively recovers contextual information and spatial details from degraded images while we design intra-blocks to reduce system complexity. Specifically, our model first learns the contextual information using encoder-decoder architectures, and then combines them with high-resolution branches that preserve spatial detail. In order to reduce the system complexity of this architecture for convenient analysis and comparison, we replace or remove the nonlinear activation function with multiplication and use a simple network structure. In addition, we replace spatial convolution with global self-attention for the middle block of encoder-decoder. The resulting tightly interlinked hierarchy architecture, named as MHNet, delivers strong performance gains on several image restoration tasks, including image deraining, and deblurring

Image patch analysis and clustering of sunspots: a dimensionality reduction approach

Sunspots, as seen in white light or continuum images, are associated with regions of high magnetic activity on the Sun, visible on magnetogram images. Their complexity is correlated with explosive solar activity and so classifying these active regions is useful for predicting future solar activity. Current classification of sunspot groups is visually based and suffers from bias. Supervised learning methods can reduce human bias but fail to optimally capitalize on the information present in sunspot images. This paper uses two image modalities (continuum and magnetogram) to characterize the spatial and modal interactions of sunspot and magnetic active region images and presents a new approach to cluster the images. Specifically, in the framework of image patch analysis, we estimate the number of intrinsic parameters required to describe the spatial and modal dependencies, the correlation between the two modalities and the corresponding spatial patterns, and examine the phenomena at different scales within the images. To do this, we use linear and nonlinear intrinsic dimension estimators, canonical correlation analysis, and multiresolution analysis of intrinsic dimension.Comment: 5 pages, 7 figures, accepted to ICIP 201

Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification

Despite the steady progress in video analysis led by the adoption of convolutional neural networks (CNNs), the relative improvement has been less drastic as that in 2D static image classification. Three main challenges exist including spatial (image) feature representation, temporal information representation, and model/computation complexity. It was recently shown by Carreira and Zisserman that 3D CNNs, inflated from 2D networks and pretrained on ImageNet, could be a promising way for spatial and temporal representation learning. However, as for model/computation complexity, 3D CNNs are much more expensive than 2D CNNs and prone to overfit. We seek a balance between speed and accuracy by building an effective and efficient video classification system through systematic exploration of critical network design choices. In particular, we show that it is possible to replace many of the 3D convolutions by low-cost 2D convolutions. Rather surprisingly, best result (in both speed and accuracy) is achieved when replacing the 3D convolutions at the bottom of the network, suggesting that temporal representation learning on high-level semantic features is more useful. Our conclusion generalizes to datasets with very different properties. When combined with several other cost-effective designs including separable spatial/temporal convolution and feature gating, our system results in an effective video classification system that that produces very competitive results on several action classification benchmarks (Kinetics, Something-something, UCF101 and HMDB), as well as two action detection (localization) benchmarks (JHMDB and UCF101-24).Comment: ECCV 2018 camera read

An Experimental Examination of Spatial DecisionSupport System Effectiveness: The Roles of Task Complexity and Technology

Alaboratory experiment was used to investigate the effects on decision maker performance of using geographic information system (GIS) technology as a spatial decision support system (SDSS). The research examined two independent variables: task complexity (i.e., low, medium, and high complexity, and SDSS use (i.e., no SDSS versus SDSS support). Professionals who are experienced decision makers completed a site location task that required decisions to be made based upon spatially-referenced information. The results confirm the hypotheses and show that SDSS use and task complexity both have an important impact on decision quality and solution time. The study builds upon and extends image theory as a basis for explaining efficiency differences resulting from differing graphical displays of spatial informatio

Subwavelength edge detection through trapped resonances in waveguides

Lenses that can collect the perfect image of an object must restore propagative and evanescent waves. However, for efficient information transfer, e.g., in compressed sensing, it is often desirable to detect only the fast spatial variations of the wave field (carried by evanescent waves), as the one created by edges or small details. Image processing edge detection algorithms perform such operation but they add time and complexity to the imaging process. Here, we present a new subwavelength approach that generates an image of only those components of the acoustic field that are equal to or smaller than the operating wavelength. The proposed technique converts evanescent waves into propagative waves exciting trapped resonances in a waveguide, and it uses periodicity to attenuate the propagative components. This approach achieves resolutions about an order of magnitude smaller than the operating wavelength and makes it possible to visualize independently edges aligned along different directions