Refined UNet V4: End-to-End Patch-Wise Network for Cloud and Shadow Segmentation with Bilateral Grid

Remote sensing images are usually contaminated by cloud and corresponding shadow regions, making cloud and shadow detection one of the essential prerequisites for processing and translation of remote sensing images. Edge-precise cloud and shadow segmentation remains challenging due to the inherent high-level semantic acquisition of current neural segmentation fashions. We, therefore, introduce the Refined UNet series to partially achieve edge-precise cloud and shadow detection, including two-stage Refined UNet, v2 with a potentially efficient gray-scale guided Gaussian filter-based CRF, and v3 with an efficient multi-channel guided Gaussian filter-based CRF. However, it is visually demonstrated that the locally linear kernel used in v2 and v3 is not sufficiently sensitive to potential edges in comparison with Refined UNet. Accordingly, we turn back to the investigation of an end-to-end UNet-CRF architecture with a Gaussian-form bilateral kernel and its relatively efficient approximation. In this paper, we present Refined UNet v4, an end-to-end edge-precise segmentation network for cloud and shadow detection, which is capable of retrieving regions of interest with relatively tight edges and potential shadow regions with ambiguous edges. Specifically, we inherit the UNet-CRF architecture exploited in the Refined UNet series, which concatenates a UNet backbone of coarsely locating cloud and shadow regions and an embedded CRF layer of refining edges. In particular, the bilateral grid-based approximation to the Gaussian-form bilateral kernel is applied to the bilateral message-passing step, in order to ensure the delineation of sufficiently tight edges and the retrieval of shadow regions with ambiguous edges. Our TensorFlow implementation of the bilateral approximation is relatively computationally efficient in comparison with Refined UNet, attributed to the straightforward GPU acceleration. Extensive experiments on Landsat 8 OLI dataset illustrate that our v4 can achieve edge-precise cloud and shadow segmentation and improve the retrieval of shadow regions, and also confirm its computational efficiency

Refined UNet: UNet-Based Refinement Network for Cloud and Shadow Precise Segmentation

Formulated as a pixel-level labeling task, data-driven neural segmentation models for cloud and corresponding shadow detection have achieved a promising accomplishment in remote sensing imagery processing. The limited capability of these methods to delineate the boundaries of clouds and shadows, however, is still referred to as a central issue of precise cloud and shadow detection. In this paper, we focus on the issue of rough cloud and shadow location and fine-grained boundary refinement of clouds on the dataset of Landsat8 OLI and therefore propose the Refined UNet to achieve this goal. To this end, a data-driven UNet-based coarse prediction and a fully-connected conditional random field (Dense CRF) are concatenated to achieve precise detection. Specifically, the UNet network with adaptive weights of balancing categories is trained from scratch, which can locate the clouds and cloud shadows roughly, while correspondingly the Dense CRF is employed to refine the cloud boundaries. Eventually, Refined UNet can give cloud and shadow proposals sharper and more precisely. The experiments and results illustrate that our model can propose sharper and more precise cloud and shadow segmentation proposals than the ground truths do. Additionally, evaluations on the Landsat 8 OLI imagery dataset of Blue, Green, Red, and NIR bands illustrate that our model can be applied to feasibly segment clouds and shadows on the four-band imagery data

An Empirical Model of Angle-of-Arrival Variance for a Gaussian Wave Propagation through Non-Kolmogorov Turbulence

This paper proposes an empirical model of the angle-of-arrival (AOA) variance for a Gaussian wave propagating through the weak non-Kolmogorov turbulence. The proposed model is approximately expressed as the linear weighted average between the AOA variances of the plane and spherical waves. The Monte Carlo method is applied to validate the proposed model. The numerical simulations indicate that, under the geometrical optics approximation, the AOA variance for a Gaussian wave is insensitive to the change of the diffraction parameter and can be closely approximated by a simple linear relationship in the refraction parameter. These two properties ensure the validity of the empirical model

Large-scale fabrication of nanostructure on bio-metallic substrate for surface enhanced raman and fluorescence scattering

The integration of surface-enhanced Raman scattering (SERS) and surface-enhanced fluorescence (SEF) has attracted increasing interest and is highly probable to improve the sensitivity and reproducibility of spectroscopic investigations in biomedical fields. In this work, dual-mode SERS and SEF hierarchical structures have been developed on a single bio-metallic substrate. The hierarchical structure was composed of micro-grooves, nano-particles, and nano-ripples. The crystal violet was selected as reporter molecule and both the intensity of Raman and fluorescence signals were enhanced because of the dual-mode SERS−SEF phenomena with enhancement factors (EFs) of 7.85 × 105 and 14.32, respectively. The Raman and fluorescence signals also exhibited good uniformity with the relative standard deviation value of 2.46% and 5.15%, respectively. Moreover, the substrate exhibited high sensitivity with the limits of detection (LOD) as low as 1 × 10−11 mol/L using Raman spectroscopy and 1 × 10−10 mol/L by fluorescence spectroscopy. The combined effect of surface plasmon resonance and “hot spots” induced by the hierarchical laser induced periodical surface structures (LIPSS) was mainly contributed to the enhancement of Raman and fluorescence signal. We propose that the integration of SERS and SEF in a single bio-metallic substrate is promising to improve the sensitivity and reproducibility of detection in biomedical investigations.Published versio

Towards Edge-Precise Cloud and Shadow Detection on the GaoFen-1 Dataset: A Visual, Comprehensive Investigation

Remote sensing images are usually contaminated by opaque cloud and shadow regions when acquired, and therefore cloud and shadow detection arises as one of the essential prerequisites for restoration and prediction of the objects of interest underneath, which are required by further processing and analysis. Cutting-edge, learning-based segmentation techniques, given a well-labeled, sufficient sample set, are significantly developed for such a detection issue and can already achieve region-accurate or even pixel-precise performance. However, it may possibly be problematic to attempt to apply the sophisticated segmentation techniques to label-free datasets in a straightforward way, more specifically, to the remote sensing data generated by the Chinese domestic satellite GaoFen-1. We wish to partially address such a segmentation problem from a practical perspective rather than in a conceptual way. This can be performed by considering a hypothesis that a segmentor, which is sufficiently trained on the well-labeled samples of common bands drawn from a source dataset, can be directly applicable to the custom, band-consistent test cases from a target set. Such a band-consistent hypothesis allows us to present a straightforward solution to the GaoFen-1 segmentation problem by treating the well-labeled Landsat 8 Operational Land Imager dataset as the source and by selecting the fourth, the third, and the second bands, also known as the false-color bands, to construct the band-consistent samples and cases. Furthermore, we attempt to achieve edge-refined segmentation performance on the GaoFen-1 dataset by adopting our prior Refined UNet and v4. We finally verify the effectiveness of the band-consistent hypothesis and the edge-refined approaches by performing a relatively comprehensive investigation, including visual comparisons, ablation experiments regarding bilateral manipulations, explorations of critical hyperparameters within our implementation of the conditional random field, and time consumption in practice. The experiments and corresponding results show that the hypothesis of selecting the false-color bands is effective for cloud and shadow segmentation on the GaoFen-1 data, and that edge-refined segmentation performance of our Refined UNet and v4 can be also achieved

Refined UNet V2: End-to-End Patch-Wise Network for Noise-Free Cloud and Shadow Segmentation

Cloud and shadow detection is an essential prerequisite for further remote sensing processing, whereas edge-precise segmentation remains a challenging issue. In Refined UNet, we considered the aforementioned task and proposed a two-stage pipeline to achieve the edge-precise segmentation. The isolated segmentation regions in Refined UNet, however, bring inferior visualization and should be sufficiently eliminated. Moreover, an end-to-end model is also expected to jointly predict and refine the segmentation results. In this paper, we propose the end-to-end Refined UNet v2 to achieve joint prediction and refinement of cloud and shadow segmentation, which is capable of visually neutralizing redundant segmentation pixels or regions. To this end, we inherit the pipeline of Refine UNet, revisit the bilateral message passing in the inference of conditional random field (CRF), and then develop a novel bilateral strategy derived from the Guided Gaussian filter. Derived from a local linear model of denoising, our v2 can considerably remove isolated segmentation pixels or regions, which is able to yield “cleaner” results. Compared to the high-dimensional Gaussian filter, the Guided Gaussian filter-based message-passing strategy is quite straightforward and easy to implement so that a brute-force implementation can be easily given in GPU frameworks, which is potentially efficient and facilitates embedding. Moreover, we prove that Guided Gaussian filter-based message passing is highly relevant to the Gaussian bilateral term in Dense CRF. Experiments and results demonstrate that our v2 is quantitatively comparable to Refined UNet, but can visually outperform that from the noise-free segmentation perspective. The comparison of time consumption also supports the potential efficiency of our v2

Towards Edge-Precise Cloud and Shadow Detection on the GaoFen-1 Dataset: A Visual, Comprehensive Investigation

Remote sensing images are usually contaminated by opaque cloud and shadow regions when acquired, and therefore cloud and shadow detection arises as one of the essential prerequisites for restoration and prediction of the objects of interest underneath, which are required by further processing and analysis. Cutting-edge, learning-based segmentation techniques, given a well-labeled, sufficient sample set, are significantly developed for such a detection issue and can already achieve region-accurate or even pixel-precise performance. However, it may possibly be problematic to attempt to apply the sophisticated segmentation techniques to label-free datasets in a straightforward way, more specifically, to the remote sensing data generated by the Chinese domestic satellite GaoFen-1. We wish to partially address such a segmentation problem from a practical perspective rather than in a conceptual way. This can be performed by considering a hypothesis that a segmentor, which is sufficiently trained on the well-labeled samples of common bands drawn from a source dataset, can be directly applicable to the custom, band-consistent test cases from a target set. Such a band-consistent hypothesis allows us to present a straightforward solution to the GaoFen-1 segmentation problem by treating the well-labeled Landsat 8 Operational Land Imager dataset as the source and by selecting the fourth, the third, and the second bands, also known as the false-color bands, to construct the band-consistent samples and cases. Furthermore, we attempt to achieve edge-refined segmentation performance on the GaoFen-1 dataset by adopting our prior Refined UNet and v4. We finally verify the effectiveness of the band-consistent hypothesis and the edge-refined approaches by performing a relatively comprehensive investigation, including visual comparisons, ablation experiments regarding bilateral manipulations, explorations of critical hyperparameters within our implementation of the conditional random field, and time consumption in practice. The experiments and corresponding results show that the hypothesis of selecting the false-color bands is effective for cloud and shadow segmentation on the GaoFen-1 data, and that edge-refined segmentation performance of our Refined UNet and v4 can be also achieved

A Post-Rectification Approach of Depth Images of Kinect v2 for 3D Reconstruction of Indoor Scenes

3D reconstruction of indoor scenes is a hot research topic in computer vision. Reconstructing fast, low-cost, and accurate dense 3D maps of indoor scenes have applications in indoor robot positioning, navigation, and semantic mapping. In other studies, the Microsoft Kinect for Windows v2 (Kinect v2) is utilized to complete this task, however, the accuracy and precision of depth information and the accuracy of correspondence between the RGB and depth (RGB-D) images still remain to be improved. In this paper, we propose a post-rectification approach of the depth images to improve the accuracy and precision of depth information. Firstly, we calibrate the Kinect v2 with a planar checkerboard pattern. Secondly, we propose a post-rectification approach of the depth images according to the reflectivity-related depth error. Finally, we conduct tests to evaluate this post-rectification approach from the perspectives of accuracy and precision. In order to validate the effect of our post-rectification approach, we apply it to RGB-D simultaneous localization and mapping (SLAM) in an indoor environment. Experimental results show that once our post-rectification approach is employed, the RGB-D SLAM system can perform a more accurate and better visual effect 3D reconstruction of indoor scenes than other state-of-the-art methods

Impact of electronic cigarette usage on the onset of respiratory symptoms and COPD among Chinese adults

Abstract The prevalence of dual usage and the relatively low cessation rate among e-cigarette (EC) users suggest that ECs have not demonstrated significant effectiveness as a smoking cessation tool. Furthermore, there has been a substantial increase in the prevalence of EC usage in recent years. Therefore, the objective of this study is to investigate the association between EC use and the incidence of respiratory symptoms and chronic obstructive pulmonary disease (COPD). A total of 10,326 participants aged between 20 and 55 years, without any respiratory diseases or COPD, were recruited for the study. These individuals attended employee physical examinations conducted at 16 public hospitals in Hebei province, China from 2015 to 2020. Logistic regression models were utilized to assess the association between EC use and the risk of respiratory symptoms and COPD using risk ratios along with their corresponding 95% confidence intervals. Restricted cubic spline functions were employed to investigate the dose–response non-linear relationship. The robustness of the logistic regression models was evaluated through subgroup analyses, and sensitivity analyses. During the 5-year follow-up period, a total of 1071 incident cases of respiratory symptoms and 146 incident cases of COPD were identified in this cohort study. After adjusting for relevant confounding factors, EC users demonstrated a respective increase in the risk of reporting respiratory symptoms and COPD by 28% and 8%. Furthermore, dual users who used both ECs and combustible cigarettes exhibited an elevated risk of incident respiratory symptoms and COPD by 41% and 18%, respectively, compared to those who had never used non-users of any cigarette products. The association between daily EC consumption and the development of respiratory symptoms, as well as COPD, demonstrated a significant J-shaped pattern. The potential adverse association between the consumption of ECs, particularly when used in combination with combustible cigarettes, and the development of respiratory symptoms and COPD necessitates careful consideration. Policymakers should approach ECs cautiously as a prospective smoking cessation tool

Refined UNet V4: End-to-End Patch-Wise Network for Cloud and Shadow Segmentation with Bilateral Grid

Remote sensing images are usually contaminated by cloud and corresponding shadow regions, making cloud and shadow detection one of the essential prerequisites for processing and translation of remote sensing images. Edge-precise cloud and shadow segmentation remains challenging due to the inherent high-level semantic acquisition of current neural segmentation fashions. We, therefore, introduce the Refined UNet series to partially achieve edge-precise cloud and shadow detection, including two-stage Refined UNet, v2 with a potentially efficient gray-scale guided Gaussian filter-based CRF, and v3 with an efficient multi-channel guided Gaussian filter-based CRF. However, it is visually demonstrated that the locally linear kernel used in v2 and v3 is not sufficiently sensitive to potential edges in comparison with Refined UNet. Accordingly, we turn back to the investigation of an end-to-end UNet-CRF architecture with a Gaussian-form bilateral kernel and its relatively efficient approximation. In this paper, we present Refined UNet v4, an end-to-end edge-precise segmentation network for cloud and shadow detection, which is capable of retrieving regions of interest with relatively tight edges and potential shadow regions with ambiguous edges. Specifically, we inherit the UNet-CRF architecture exploited in the Refined UNet series, which concatenates a UNet backbone of coarsely locating cloud and shadow regions and an embedded CRF layer of refining edges. In particular, the bilateral grid-based approximation to the Gaussian-form bilateral kernel is applied to the bilateral message-passing step, in order to ensure the delineation of sufficiently tight edges and the retrieval of shadow regions with ambiguous edges. Our TensorFlow implementation of the bilateral approximation is relatively computationally efficient in comparison with Refined UNet, attributed to the straightforward GPU acceleration. Extensive experiments on Landsat 8 OLI dataset illustrate that our v4 can achieve edge-precise cloud and shadow segmentation and improve the retrieval of shadow regions, and also confirm its computational efficiency