GFM: Building Geospatial Foundation Models via Continual Pretraining

Geospatial technologies are becoming increasingly essential in our world for a wide range of applications, including agriculture, urban planning, and disaster response. To help improve the applicability and performance of deep learning models on these geospatial tasks, various works have begun investigating foundation models for this domain. Researchers have explored two prominent approaches for introducing such models in geospatial applications, but both have drawbacks in terms of limited performance benefit or prohibitive training cost. Therefore, in this work, we propose a novel paradigm for building highly effective geospatial foundation models with minimal resource cost and carbon impact. We first construct a compact yet diverse dataset from multiple sources to promote feature diversity, which we term GeoPile. Then, we investigate the potential of continual pretraining from large-scale ImageNet-22k models and propose a multi-objective continual pretraining paradigm, which leverages the strong representations of ImageNet while simultaneously providing the freedom to learn valuable in-domain features. Our approach outperforms previous state-of-the-art geospatial pretraining methods in an extensive evaluation on seven downstream datasets covering various tasks such as change detection, classification, multi-label classification, semantic segmentation, and super-resolution

Roles of arabidopsis WRKY18, WRKY40 and WRKY60 transcription factors in plant responses to abscisic acid and abiotic stress

<p>Abstract</p> <p>Background</p> <p>WRKY transcription factors are involved in plant responses to both biotic and abiotic stresses. Arabidopsis WRKY18, WRKY40, and WRKY60 transcription factors interact both physically and functionally in plant defense responses. However, their role in plant abiotic stress response has not been directly analyzed.</p> <p>Results</p> <p>We report that the three WRKYs are involved in plant responses to abscisic acid (ABA) and abiotic stress. Through analysis of single, double, and triple mutants and overexpression lines for the WRKY genes, we have shown that <it>WRKY18 </it>and <it>WRKY60 </it>have a positive effect on plant ABA sensitivity for inhibition of seed germination and root growth. The same two WRKY genes also enhance plant sensitivity to salt and osmotic stress. <it>WRKY40</it>, on the other hand, antagonizes <it>WRKY18 </it>and <it>WRKY60 </it>in the effect on plant sensitivity to ABA and abiotic stress in germination and growth assays. Both <it>WRKY18 </it>and <it>WRKY40 </it>are rapidly induced by ABA, while induction of <it>WRKY60 </it>by ABA is delayed. ABA-inducible expression of <it>WRKY60 </it>is almost completely abolished in the <it>wrky18 </it>and <it>wrky40 </it>mutants. WRKY18 and WRKY40 recognize a cluster of W-box sequences in the <it>WRKY60 </it>promoter and activate WRKY60 expression in protoplasts. Thus, <it>WRKY60 </it>might be a direct target gene of WRKY18 and WRKY40 in ABA signaling. Using a stable transgenic reporter/effector system, we have shown that both WRKY18 and WRKY60 act as weak transcriptional activators while WRKY40 is a transcriptional repressor in plant cells.</p> <p>Conclusions</p> <p>We propose that the three related WRKY transcription factors form a highly interacting regulatory network that modulates gene expression in both plant defense and stress responses by acting as either transcription activator or repressor.</p