Multimodal Earth observation data fusion: Graph-based approach in shared latent space

Multiple and heterogenous Earth observation (EO) platforms are broadly used for a wide array of applications, and the integration of these diverse modalities facilitates better extraction of information than using them individually. The detection capability of the multispectral unmanned aerial vehicle (UAV) and satellite imagery can be significantly improved by fusing with ground hyperspectral data. However, variability in spatial and spectral resolution can affect the efficiency of such dataset's fusion. In this study, to address the modality bias, the input data was projected to a shared latent space using cross-modal generative approaches or guided unsupervised transformation. The proposed adversarial networks and variational encoder-based strategies used bi-directional transformations to model the cross-domain correlation without using cross-domain correspondence. It may be noted that an interpolation-based convolution was adopted instead of the normal convolution for learning the features of the point spectral data (ground spectra). The proposed generative adversarial network-based approach employed dynamic time wrapping based layers along with a cyclic consistency constraint to use the minimal number of unlabeled samples, having cross-domain correlation, to compute a cross-modal generative latent space. The proposed variational encoder-based transformation also addressed the cross-modal resolution differences and limited availability of cross-domain samples by using a mixture of expert-based strategy, cross-domain constraints, and adversarial learning. In addition, the latent space was modelled to be composed of modality independent and modality dependent spaces, thereby further reducing the requirement of training samples and addressing the cross-modality biases. An unsupervised covariance guided transformation was also proposed to transform the labelled samples without using cross-domain correlation prior. The proposed latent space transformation approaches resolved the requirement of cross-domain samples which has been a critical issue with the fusion of multi-modal Earth observation data. This study also proposed a latent graph generation and graph convolutional approach to predict the labels resolving the domain discrepancy and cross-modality biases. Based on the experiments over different standard benchmark airborne datasets and real-world UAV datasets, the developed approaches outperformed the prominent hyperspectral panchromatic sharpening, image fusion, and domain adaptation approaches. By using specific constraints and regularizations, the network developed was less sensitive to network parameters, unlike in similar implementations. The proposed approach illustrated improved generalizability in comparison with the prominent existing approaches. In addition to the fusion-based classification of the multispectral and hyperspectral datasets, the proposed approach was extended to the classification of hyperspectral airborne datasets where the latent graph generation and convolution were employed to resolve the domain bias with a small number of training samples. Overall, the developed transformations and architectures will be useful for the semantic interpretation and analysis of multimodal data and are applicable to signal processing, manifold learning, video analysis, data mining, and time series analysis, to name a few.This research was partly supported by the Hebrew University of Jerusalem Intramural Research Found Career Development, Association of Field Crop Farmers in Israel and the Chief Scientist of the Israeli Ministry of Agriculture and Rural Development (projects 20-02-0087 and 12-01-0041)

Site‐specific weed management—constraints and opportunities for the weed research community: Insights from a workshop

The adoption of site‐specific weed management (SSWM) technologies by farmers is not aligned with the scientific achievements in this field. While scientists have demonstrated significant success in real‐time weed identification, phenotyping and accurate weed mapping by using various sensors and platforms, the integration by farmers of SSWM and weed phenotyping tools into weed management protocols is limited. This gap was therefore a central topic of discussion at the most recent workshop of the SSWM Working Group arranged by the European Weed Research Society (EWRS). This insight paper aims to summarise the presentations and discussions of some of the workshop panels and to highlight different aspects of weed identification and spray application that were thought to hinder SSWM adoption. It also aims to share views and thoughts regarding steps that can be taken to facilitate future implementation of SSWM

New perspectives on interdisciplinary earth science at the Dead Sea: The DESERVE project

The Dead Sea region has faced substantial environmental challenges in recent decades, including water resource scarcity, ~ 1 m annual decreases in the water level, sinkhole development, ascending-brine freshwater pollution, and seismic disturbance risks. Natural processes are significantly affected by human interference as well as by climate change and tectonic developments over the long term. To get a deep understanding of processes and their interactions, innovative scientific approaches that integrate disciplinary research and education are required. The research project DESERVE (Helmholtz Virtual Institute Dead Sea Research Venue) addresses these challenges in an interdisciplinary approach that includes geophysics, hydrology, and meteorology. The project is implemented by a consortium of scientific institutions in neighboring countries of the Dead Sea (Israel, Jordan, Palestine Territories) and participating German Helmholtz Centres (KIT, GFZ, UFZ). A new monitoring network of meteorological, hydrological, and seismic/geodynamic stations has been established, and extensive field research and numerical simulations have been undertaken. For the first time, innovative measurement and modeling techniques have been applied to the extreme conditions of the Dead Sea and its surroundings. The preliminary results show the potential of these methods. First time ever performed eddy covariance measurements give insight into the governing factors of Dead Sea evaporation. High-resolution bathymetric investigations reveal a strong correlation between submarine springs and neo-tectonic patterns. Based on detailed studies of stratigraphy and borehole information, the extension of the subsurface drainage basin of the Dead Sea is now reliably estimated. Originality has been achieved in monitoring flash floods in an arid basin at its outlet and simultaneously in tributaries, supplemented by spatio-temporal rainfall data. Low-altitude, high resolution photogrammetry, allied to satellite image analysis and to geophysical surveys (e.g. shear-wave reflections) has enabled a more detailed characterization of sinkhole morphology and temporal development and the possible subsurface controls thereon. All the above listed efforts and scientific results take place with the interdisciplinary education of young scientists. They are invited to attend joint thematic workshops and winter schools as well as to participate in field experiments

Site‐specific weed management—constraints and opportunities for the weed research community: Insights from a workshop

The adoption of site‐specific weed management (SSWM) technologies by farmers is not aligned with the scientific achievements in this field. While scientists have demonstrated significant success in real‐time weed identification, phenotyping and accurate weed mapping by using various sensors and platforms, the integration by farmers of SSWM and weed phenotyping tools into weed management protocols is limited. This gap was therefore a central topic of discussion at the most recent workshop of the SSWM Working Group arranged by the European Weed Research Society (EWRS). This insight paper aims to summarise the presentations and discussions of some of the workshop panels and to highlight different aspects of weed identification and spray application that were thought to hinder SSWM adoption. It also aims to share views and thoughts regarding steps that can be taken to facilitate future implementation of SSWM