Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference

Deep learning has recently demonstrated its excellent performance for multi-view stereo (MVS). However, one major limitation of current learned MVS approaches is the scalability: the memory-consuming cost volume regularization makes the learned MVS hard to be applied to high-resolution scenes. In this paper, we introduce a scalable multi-view stereo framework based on the recurrent neural network. Instead of regularizing the entire 3D cost volume in one go, the proposed Recurrent Multi-view Stereo Network (R-MVSNet) sequentially regularizes the 2D cost maps along the depth direction via the gated recurrent unit (GRU). This reduces dramatically the memory consumption and makes high-resolution reconstruction feasible. We first show the state-of-the-art performance achieved by the proposed R-MVSNet on the recent MVS benchmarks. Then, we further demonstrate the scalability of the proposed method on several large-scale scenarios, where previous learned approaches often fail due to the memory constraint. Code is available at https://github.com/YoYo000/MVSNet.Comment: Accepted by CVPR201

ContextDesc: Local Descriptor Augmentation with Cross-Modality Context

Most existing studies on learning local features focus on the patch-based descriptions of individual keypoints, whereas neglecting the spatial relations established from their keypoint locations. In this paper, we go beyond the local detail representation by introducing context awareness to augment off-the-shelf local feature descriptors. Specifically, we propose a unified learning framework that leverages and aggregates the cross-modality contextual information, including (i) visual context from high-level image representation, and (ii) geometric context from 2D keypoint distribution. Moreover, we propose an effective N-pair loss that eschews the empirical hyper-parameter search and improves the convergence. The proposed augmentation scheme is lightweight compared with the raw local feature description, meanwhile improves remarkably on several large-scale benchmarks with diversified scenes, which demonstrates both strong practicality and generalization ability in geometric matching applications.Comment: Accepted to CVPR 2019 (oral), supplementary materials included. (https://github.com/lzx551402/contextdesc

Decameter Stationary Type IV Burst in Imaging Observations on the 6th of September 2014

First-of-its-kind radio imaging of decameter solar stationary type IV radio burst has been presented in this paper. On 6 September 2014 the observations of type IV burst radio emission have been carried out with the two-dimensional heliograph based on the Ukrainian T-shaped radio telescope (UTR-2) together with other telescope arrays. Starting at 09:55 UT and throughout 3 hours, the radio emission was kept within the observational session of UTR-2. The interesting observation covered the full evolution of this burst, "from birth to death". During the event lifetime, two C-class solar X-ray flares with peak times 11:29 UT and 12:24 UT took place. The time profile of this burst in radio has a double-humped shape that can be explained by injection of energetic electrons, accelerated by the two flares, into the burst source. According to the heliographic observations we suggest the burst source was confined within a high coronal loop, which was a part of a relatively slow coronal mass ejection. The latter has been developed for several hours before the onset of the event. Through analyzing about 1.5 million of heliograms (3700 temporal frames with 4096 images in each frame that correspond to the number of frequency channels) the radio burst source imaging shows a fascinating dynamical evolution. Both space-based (GOES, SDO, SOHO, STEREO) data and various ground-based instrumentation (ORFEES, NDA, RSTO, NRH) records have been used for this study

An observational revisit of band-split solar type-II radio bursts

Band split of solar type II radio bursts, discovered several decades ago, is a fascinating phenomenon with the type-II lanes exhibiting two almost-parallel sub-bands with similar morphology. The underlying split mechanism remains elusive. One popular interpretation is that the splitting bands are emitted from the shock upstream and downstream, respectively, with their frequency ratio ({\gamma}) determined by the shock compression ratio. This interpretation has been taken as the physical basis for many published references. Here we report an observational analysis of type II events with nice split selected from the ground-based RSTN data from 2001 to 2014, in the metric-decametric wavelength. We investigate the temporal variation and distribution of {\gamma}, and conduct correlation analyses on the deduced spectral values. It is found that {\gamma} varies in a very narrow range with >80% of {\gamma} (one-minute averaged data) being between 1.15 to 1.25. For some well-observed and long-lasting events, {\gamma} does not show a systematic variation trend within observational uncertainties, from the onset to the termination of the splits. In addition, the parameters representing the propagation speed of the radio source (presumably the coronal shock) show a very weak or basically no correlation with {\gamma}. We suggest that these results do not favor the upstreamdownstream scenario of band splits

MVSNet: Depth Inference for Unstructured Multi-view Stereo

We present an end-to-end deep learning architecture for depth map inference from multi-view images. In the network, we first extract deep visual image features, and then build the 3D cost volume upon the reference camera frustum via the differentiable homography warping. Next, we apply 3D convolutions to regularize and regress the initial depth map, which is then refined with the reference image to generate the final output. Our framework flexibly adapts arbitrary N-view inputs using a variance-based cost metric that maps multiple features into one cost feature. The proposed MVSNet is demonstrated on the large-scale indoor DTU dataset. With simple post-processing, our method not only significantly outperforms previous state-of-the-arts, but also is several times faster in runtime. We also evaluate MVSNet on the complex outdoor Tanks and Temples dataset, where our method ranks first before April 18, 2018 without any fine-tuning, showing the strong generalization ability of MVSNet.Comment: Accepted to European Conference on Computer Vision (ECCV 2018

Reaching the continuum limit in finite-temperature ab initio field-theory computations in many-fermion systems

Finite-temperature, grand-canonical computations based on field theory are widely applied in areas including condensed matter physics, ultracold atomic gas systems, and lattice gauge theory. However, these calculations have computational costs scaling as $N_s^3$ with the size of the lattice or basis set, $N_s$. We report a new approach based on systematically controllable low-rank factorization which reduces the scaling of such computations to $N_s N_e^2$, where $N_e$ is the average number of fermions in the system. In any realistic calculations aiming to describe the continuum limit, $N_s/N_e$ is large and needs to be extrapolated effectively to infinity for convergence. The method thus fundamentally changes the prospect for finite-temperature many-body computations in correlated fermion systems. Its application, in combination with frameworks to control the sign or phase problem as needed, will provide a powerful tool in {\it ab initio} quantum chemistry and correlated electron materials. We demonstrate the method by computing exact properties of the two-dimensional Fermi gas with zero-range attractive interaction, as a function of temperature in both the normal and superfluid states.Comment: 6 pages, 4 figure

Visibility-aware Multi-view Stereo Network

Learning-based multi-view stereo (MVS) methods have demonstrated promising results. However, very few existing networks explicitly take the pixel-wise visibility into consideration, resulting in erroneous cost aggregation from occluded pixels. In this paper, we explicitly infer and integrate the pixel-wise occlusion information in the MVS network via the matching uncertainty estimation. The pair-wise uncertainty map is jointly inferred with the pair-wise depth map, which is further used as weighting guidance during the multi-view cost volume fusion. As such, the adverse influence of occluded pixels is suppressed in the cost fusion. The proposed framework Vis-MVSNet significantly improves depth accuracies in the scenes with severe occlusion. Extensive experiments are performed on DTU, BlendedMVS, and Tanks and Temples datasets to justify the effectiveness of the proposed framework.Comment: Accepted to BMVC 202

EUV and Magnetic Activities Associated with Type-I Solar Radio Bursts

Type-I bursts (i.e. noise storms) are the earliest-known type of solar radio emission at the metre wavelength. They are believed to be excited by non-thermal energetic electrons accelerated in the corona. The underlying dynamic process and exact emission mechanism still remain unresolved. Here, with a combined analysis of extreme ultraviolet (EUV), radio and photospheric magnetic field data of unprecedented quality recorded during a type-I storm on 30 July 2011, we identify a good correlation between the radio bursts and the co-spatial EUV and magnetic activities. The EUV activities manifest themselves as three major brightening stripes above a region adjacent to a compact sunspot, while the magnetic field there presents multiple moving magnetic features (MMFs) with persistent coalescence or cancelation and a morphologically similar three-part distribution. We find that the type-I intensities are correlated with those of the EUV emissions at various wavelengths with a correlation coefficient of 0.7-0.8. In addition, in the region between the brightening EUV stripes and the radio sources there appear consistent dynamic motions with a series of bi-directional flows, suggesting ongoing small-scale reconnection there. Mainly based on the induced connection between the magnetic motion at the photosphere and the EUV and radio activities in the corona, we suggest that the observed type-I noise storms and the EUV brightening activities are the consequence of small-scale magnetic reconnection driven by MMFs. This is in support of the original proposal made by Bentely et al. (Solar Phys. 193, 227, 2000)

Stacking-symmetry governed second harmonic generation in graphene trilayers

Crystal symmetry plays a central role in governing a wide range of fundamental physical phenomena. One example is the nonlinear optical second harmonic generation (SHG), which requires inversion symmetry breaking. Here we report a unique stacking-induced SHG in trilayer graphene, whose individual monolayer sheet is centrosymmetric. Depending on layer stacking sequence, we observe a strong optical SHG in Bernal (ABA) stacked non-centrosymmetric trilayer, while it vanishes in rhombohedral (ABC) stacked one which preserves inversion symmetry. This highly contrasting SHG due to the distinct stacking symmetry enables us to map out the ABA and ABC crystal domains in otherwise homogeneous graphene trilayer. The extracted second order nonlinear susceptibility of the ABA trilayer is surprisingly large, comparable to the best known 2D semiconductors enhanced by excitonic resonance. Our results reveal a novel stacking order induced nonlinear optical effect, as well as unleash the opportunity for studying intriguing physical phenomena predicted for stacking-dependent ABA and ABC graphene trilayers.Comment: To appear in Science Advance

Source Imaging of a Moving Type-IV Solar Radio Burst and its Role in Tracking Coronal Mass Ejection From the Inner to the Outer Corona

Source imaging of solar radio bursts can be used to track energetic electrons and associated magnetic structures. Here we present a combined analysis of data at different wavelengths for an eruption associated with a moving type-IV (t-IVm) radio burst. In the inner corona, the sources are correlated with a hot and twisted eruptive EUV structure, while in the outer corona the sources are associated with the top front of the bright core of a white light coronal mass ejection (CME). This reveals the potential of using t-IVm imaging data to continuously track the CME by lighting up the specific component containing radio-emitting electrons. It is found that the t-IVm burst presents a clear spatial dispersion with observing frequencies. The burst manifests broken power-law like spectra in brightness temperature, which is as high as $10^7$-$10^9$ K while the polarization level is in-general weak. In addition, the t-IVm burst starts during the declining phase of the flare with a duration as long as 2.5 hours. From the differential emission measure analysis of AIA data, the density of the T-IVm source is likely at the level of 10$^8$ cm$^{-3}$ at the start of the burst, and the temperature may reach up to several MK. These observations do not favor gyro-synchrotron to be the radiation mechanism, yet in line with a coherent plasma emission excited by energetic electrons trapped within the source. Further studies are demanded to elucidate the emission mechanism and explore the full diagnostic potential of t-IVm bursts.Comment: 22 pages, 8 figures, Accepted for publication in AP