Chen, Kaiqiang

Hansch, Ronny

Huang, Hai

Persello, Claudio

Schmitt, Michael

Sun, Xian

Tang, Deke

Vivone, Gemine

Yan, Zhiyuan

English

University of Twente Research Information

146 TECHNICAL COMMITTEESIEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE    DECEMBER 2023RONNY HÄNSCH , CLAUDIO PERSELLO , GEMINE VIVONE , KAIQIANG CHEN, ZHIYUAN YAN, DEKE TANG, HAI HUANG, MICHAEL SCHMITT , AND XIAN SUNThe Image Analysis and Data Fusion Technical Committee (IADF TC) of the IEEE Geoscience and Remote Sensing Society (GRSS) has been organiz-ing the annual Data Fusion Contest (DFC) since 2006. The contest promotes the development of methods for extracting geospatial in-formation from large-scale, multisensor, multimodal, and multitemporal data. It aims to propose new prob-lem settings that are chal-lenging to address with existing techniques, and to establish new benchmarks for scientific challenges in remote sens-ing image analysis [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19].THE 2023 DATA FUSION CONTESTThe 2023 IEEE GRSS DFC (DFC23) aims to push current research on building extraction, classification, and 3D reconstruction toward urban reconstruction with fine-grained semantic information of roof types (see “Using the Data and Joining the Image Analysis and Data Fu-sion Technical Committee”). DFC23 establishes a large-scale, fine-grained, and multimodal benchmark for the classification of building roof types. The ultimate goal of the competition is to develop accurate building extrac-tion and classification models using multimodal data, including optical and synthetic aperture radar (SAR) im-ages. This contest was designed as a benchmarking com-petition following previous editions [12], [13], [14], [15], [16], [17], [18]. DFC23 fea-tured the following two tracks running in parallel:1)  “Track 1: Building De-tection and Roof Type Classification”2)  “Track 2: Multi-Task Learning of Joint Build-ing Extract ion and Height Estimation.”Both tracks were co-organized by the IADF TC of the GRSS, Aerospace Information Research In-stitute under the Chinese Academy of Sciences, Uni-versity of the Federal Armed Forces Munich, GEOVIS Technology Co., Ltd., and GEOVIS Earth Technology Co., Ltd. DFC23 provides a globally distributed large-scale urban building classification and reconstruction dataset. The buildings are distributed across 17 cities on six continents and labeled according to a detailed (fine-grained) categorization of roof types. The data-set consists of nearly 300,000 instances with 12 dif-ferent types of building roofs. To facilitate multimodal data fusion, it combines data from the following three different sources:1) optical images from the commercial very high- resolution Earth-observation satellite SuperView-1 (or Gaojing-1 in Chinese) and Gaofen-2 satellites2) SAR images from the civilian SAR satellite Gaofen-33) normalized digital surface models in Track 2 de-rived from stereo images captured by Gaofen-7 and  WorldView.Track 1 focused on the detection and classification of building roof types from high-resolution optical and SAR satellite imagery. The participants were required to sub-mit segmentation results represented by polygons (i.e., a Report on the 2023 IEEE GRSS Data Fusion Contest: Large-Scale Fine-Grained Building Classification for Semantic Urban ReconstructionDigital Object Identifier 10.1109/MGRS.2023.3302342Date of current version: 11 December 2023147 DECEMBER 2023    IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINEsequence of points that delineates the building’s contours) or run-length encoding, one fine-grained category with confi-dence for each instance, and one optional bounding box. For evaluation, the standard Common Objects in Context met-ric AP50  [with an Intersection over Union (IoU)  threshold of 0.5] is employed. The classes are taken into account when calculating the IoU score.Track 2 defines the joint task of building extraction and height estimation. The participants are required to sub-mit results, including two parts: a building extraction re-sult (for which estimation of the roof category will not be considered during the evaluation) and a pixelwise height estimation. For evaluation, we average the accuracy regard-ing building extraction and height estimation to obtain a single score. The metric of the building extraction is the same as that of Track 1, with the only difference being that the categories are ignored. The metric of height estimation is threshold accuracy NNt1d =  (1)where N is the total number of pixels, and Nt  is the number of pixels for which , .max yyyy1 25pppp1ttd n  (2)where y and yt  are the reference and predicted height,  respectively.DFC23 tackled the technical challenges rooted in ex-traction and 3D reconstruction of buildings, addressing the potential of leveraging information on fine-grained buildings roof types that, if ignored, limits further analy-sis. One of the most important features of DFC23 is the link between visual features in satellite imagery and fine-grained classification of building roof types. This states an extremely challenging task, which is tackled by the partici-pants, leading to results that have a major impact, e.g., on urban planning applications.OUTCOME OF THE CONTESTBoth tracks saw the overwhelming participation of a large number of competing international teams from 32 coun-tries. Interestingly, with 195 active CodaLab participants, Track 1 was favored by the participants, while in Track 2, 71 participants competed. This might indicate that clas-sification problems are ei-ther more interesting in the community than regression tasks, or that classification problems provide a lower en-trance threshold. The first-to-fourth-ranked teams in both tracks were awarded the winners of the contest and presented their solutions during the 2023 IEEE International Geoscience and Remote Sensing Symposium (IGARSS 2023). (Note that one team in Track 1 withdrew its submission and was therefore excluded from the list of winners.) The following is a list of the winning teams in Track 1: ◗ First place: the PIESAT-AI team of Guozhang Liu, Bao-chai Peng, Ting Liu, Pan Zhang, Mengke Yuan, Chanran Lu, Ningning Cao, Sen Zhang, Simin Huang, and Tao Wang, PIESAT Information Technology Co, Ltd., Bei-jing, China, for “Fine-Grained Building Roof Instance Segmentation Based on Domain Adapted Pretraining and Composite Dual-Backbone” [20] ◗ Second place: the IPIU-XDU team of Xiaoqiang Lu, Li-cheng Jiao, Qiong Liu, Lingling Li, Fang Liu, Xu Liu, and FIGURE 1. The banner image for the 2023 IEEE GRSS Data Fusion Contest. This nonnumbered figure is to be shown in the paper without a caption: logo of the IADF Technical Committee.AS IN PREVIOUS YEARS, DFC23 ATTRACTED PARTICIPANTS FROM A VARIETY OF DISCIPLINES, INCLUDING ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING AS WELL AS THE REMOTE SENSING COMMUNITY.148 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE    DECEMBER 2023Yuting Yang, Xidian University, China, for “A Strong Vi-sion Transformer Adapter With Adaptive Thresholding for Fine-Grained Building Classification” [21] ◗ Third place: the carryhjr team of Jiarui Hu (Wuhan Uni-versity), Zijun Huang (Guangdong University of Tech-nology), Fei Shen (Nanjing University of Science and Technology), Dian He (Tsinghua University), and Qin-gyu Xian (Tsinghua University), for “A Bag of Tricks for Fine-Grained Roof Extraction” [22]The following is a list of the winning teams in Track 2: ◗ First place: the PIESAT-AI team of Chaoran Lu, Ningn-ing Cao, Pan Zhang, Ting Liu, Baochai Peng, Guozhang Liu, Mengke Yuan, Sen Zhang, Simin Huang, and Tao Wang, PIESAT Information Technology Co, Ltd., Bei-jing, China, for “HGDNET: A Height-Hierarchy Guided Dual-Decoder Network for Single View Building Extrac-tion and Height Estimation” [23] ◗ Second place: the IPIU-XDU team of Xiaoqiang Lu, Li-cheng Jiao, Qiong Liu, Lingling Li, Fang Liu, Xu Liu, and Yuting Yang, Xidian University, China, for “Trident Co-operation Network for Building Extraction and Height Estimation” [24]. ◗ Third place: the ZheWang team of Yuxuan Guo and Zhe Wang, Wuhan University, China, for “Height Estimation Based on Semantic Segmentation” [25] ◗ Fourth place: the carryhjr team of Jiarui Hu (Wuhan Uni-versity), Zijun Huang (Guangdong University of Tech-nology), Fei Shen (Nanjing University of Science and Technology), Dian He (Tsinghua University), and Qin-gyu Xian (Tsinghua University), for “A Robust Method for Roof Extraction and Height Estimation” [26].At the end of the competition, all the winning teams wrote a paper on their approach, which was peer reviewed by the DFC organizing committee. These papers were in-cluded in the technical program of IGARSS 2023 and pre-sented in a Community Contributed Session (CCS) on the DFC during the symposium. During the CCS, a panel consisting of Gülşen Taşkın (Istanbul Technical University, Turkey), Begüm Demir (Technical University of Berlin, Ger-many), Claudio Persello (University of Twente, The Nether-lands), Saurabh Prasad (University of Houston, USA), and Ronny Hänsch (DLR, Germany) discussed the challenges of organizing data contests and compiling benchmark datasets, how benchmarks and challenges shaped the Earth-observation community, and what the future of such datasets might look like. All these teams were awarded an IEEE Certificate of Recognition for their winning participa-tion. The winning teams in each track received special priz-es, thanks to GEOVIS Technology Co., Ltd. and GEOVIS Earth Technology Co., Ltd. An extended article discussing the winning solutions of the first and second-ranked teams of both tracks will be submitted for peer review to the open access IEEE Journal of Selected Topics in Applied Earth Observa-tions and Remote Sensing.As in previous years, DFC23 attracted participants from a variety of disciplines, including artificial intelligence and machine learning as well as the remote sensing commu-nity. The participation of such a diverse range of disciplines promotes the development of novel and interdisciplinary approaches to solve technical problems in the remote sensing and geoscience communities and also leads to a movement to challenge global issues by bringing together knowledge from different fields. The winning teams are mostly student led and their extraordinary efforts have led to dramatic advances in technology for the new problems addressed in this competition, and to the formation of a vibrant community.ACKNOWLEDGMENTThe IADF TC chairs would like to thank the GRSS for con-tinuously supporting the annual DFC through funding and resources. We also thank the Aerospace Information Research Institute under the Chinese Academy of Sciences, University of the Bundeswehr Munich, GEOVIS Technol-ogy Co., Ltd., and GEOVIS Earth Technology Co., Ltd. for co-organizing the contest, curating the dataset, and join-ing the scientific discussions about the outcome of DFC23. We thank GEOVIS Technology Co., Ltd. and GEOVIS Earth Technology Co., Ltd. for supporting DFC23 by providing the award prize to the winning competitors.REFERENCES[1] L. Alparone, L. Wald, J. Chanussot, C. Thomas, P. Gamba, and L. M. Bruce, “Comparison of pansharpening algo-rithms: Outcome of the 2006 GRS-S data-fusion contest,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 10, pp. 3012–3021, Oct. 2007, doi: 10.1109/TGRS.2007.904923.[2] F. Pacifici, F. Del Frate, W. J. Emery, P. Gamba, and J. Cha-nussot, “Urban mapping using coarse SAR and optical data: Outcome of the 2007 GRSS data fusion contest,” Using the Data and Joining the Image Analysis and  Data Fusion Technical CommitteeThe data of the 2023 Data Fusion Contest (DFC23) and its CodaLab evaluation website (Track 1: https://codalab.lisn.upsaclay.fr/competitions/8987; Track 2: https://codalab.lisn.upsaclay.fr/competitions/8988) with the public leaderboard will remain available to the IEEE Geoscience and Remote Sensing Society (GRSS) community for benchmarking algorithms and publishing research works. The data are usable free of charge for scientific purposes, but the Contest Terms and Conditions on the contest webpage remain applicable. Please read them carefully at https://www.grss-ieee.org/community/technical -committees/2023-ieee-grss-data-fusion-contest/.You can contact the Image Analysis and Data Fusion Technical Committee (IADF TC) chairs at iadf_chairs@grss-ieee.org. If you are interested in joining the IADF TC, please fill in the form on our website at https://www.grss-ieee.org/technical-committees/image-analysis-and-data-fusion. Members receive information regarding research and applications on image analysis and data fusion topics as well as updates on the annual DFC and on all other activities of the IADF TC. Membership in the IADF TC is free! Also, you can join the LinkedIn IEEE GRSS Data Fusion Discussion Forum (https://www.linkedin.com/groups/3678437/) and Twitter channel (https://twitter.com/GrssIadf).149 DECEMBER 2023    IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINEIEEE Geosci. Remote Sens. Lett., vol. 5, no. 3, pp. 331–335, Jul. 2008, doi: 10.1109/LGRS.2008.915939.[3] G. Licciardi et al., “Decision fusion for the classifica-tion of hyperspectral data: Outcome of the 2008 GRS-S data fusion contest,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 11, pp. 3857–3865, Nov. 2009, doi: 10.1109/TGRS.2009.2029340.[4] N. Longbotham et al., “Multi-modal change detection, application to the detection of flooded areas: Outcome of the 2009–2010 data fusion contest,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5, no. 1, pp. 331–342, Feb. 2012, doi: 10.1109/JSTARS.2011.2179638.[5] F. Pacifici and Q. Du, “Foreword to the special issue on op-tical multiangular data exploitation and outcome of the 2011 GRSS data fusion contest,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5, no. 1, pp. 3–7, Feb. 2012, doi: 10.1109/JSTARS.2012.2186733.[6] C. Berger et al., “Multi-modal and multi-temporal data fu-sion: Outcome of the 2012 GRSS data fusion contest,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 6, no. 3, pp. 1324–1340, Jun. 2013, doi: 10.1109/JSTARS.2013.2245860.[7] C. Debes et al., “Hyperspectral and LiDAR data fusion: Outcome of the 2013 GRSS data fusion contest,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 6, pp. 2405–2418, Jun. 2014, doi: 10.1109/JSTARS.2014.2305441.[8] W. Liao et al., “Processing of multiresolution thermal hy-perspectral and digital color data: Outcome of the 2014 IEEE GRSS data fusion contest,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 8, no. 6, pp. 2984–2996, Jun. 2015, doi: 10.1109/JSTARS.2015.2420582.[9] M. Campos-Taberner et al., “Processing of extremely high-resolution LiDAR and RGB data: Outcome of the 2015 IEEE GRSS data fusion contest–Part A: 2-D contest,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 9, no. 12, pp. 5547–5559, Dec. 2016, doi: 10.1109/JSTARS. 2016.2569162.[10] A.-V. Vo et al., “Processing of extremely high resolution LiDAR and RGB data: Outcome of the 2015 IEEE GRSS data fusion contest—Part B: 3-D contest,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 9, no. 12, pp. 5560–5575, Dec. 2016, doi: 10.1109/JSTARS.2016.2581843.[11] L. Mou et al., “Multitemporal very high resolution from space: Outcome of the 2016 IEEE GRSS data fusion con-test,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 10, no. 8, pp. 3435–3447, Aug. 2017, doi: 10.1109/JSTARS.2017.2696823.[12] N. Yokoya et al., “Open data for global multimodal land use classification: Outcome of the 2017 IEEE GRSS data fusion contest,” IEEE J. Sel. Topics Appl. Earth Observ. Re-mote Sens., vol. 11, no. 5, pp. 1363–1377, May 2018, doi: 10.1109/JSTARS.2018.2799698.[13] Y. Xu et al., “Advanced multi-sensor optical remote sens-ing for urban land use and land cover classification: Outcome of the 2018 IEEE GRSS data fusion contest,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 12, no. 6, pp. 1709–1724, Jun. 2019, doi: 10.1109/JSTARS. 2019.2911113.[14] S. Kunwar et al., “Large-scale semantic 3-D reconstruc-tion: Outcome of the 2019 IEEE GRSS data fusion contest—Part A,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 14, pp. 922–935, 2021, doi: 10.1109/JSTARS. 2020.3032221.[15] Y. Lian et al., “Large-scale semantic 3-D reconstruction: Outcome of the 2019 IEEE GRSS data fusion contest—Part B,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 14, pp. 1158–1170, 2021, doi: 10.1109/JSTARS.2020.3035274.[16] C. Robinson et al., “Global land-cover mapping with weak supervision: Outcome of the 2020 IEEE GRSS data fusion contest,” IEEE J. Sel. Topics Appl. Earth Observ.  Remote Sens., vol. 14, pp. 3185–3199, Mar. 2021, doi: 10.1109/JSTARS .2021.3063849.[17] Y. Ma et al., “The outcome of the 2021 IEEE GRSS data fusion contest - Track DSE: Detection of settlements without electricity,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 14, pp. 12,375–12,385, Nov. 2021, doi: 10.1109/JSTARS.2021.3130446.[18] Z. Li et al., “The outcome of the 2021 IEEE GRSS data fusion contest—Track MSD: Multitemporal semantic change detection,” IEEE J. Sel. Topics Appl. Earth Observ. Re-mote Sens., vol. 15, pp. 1643–1655, Jan. 2022, doi: 10.1109/JSTARS.2022.3144318.[19] R. Hänsch et al., “The 2022 IEEE GRSS data fusion contest: Semisupervised learning [Technical Commit-tees],” IEEE Geosci. Remote Sens. Mag. (replaced Newslet-ter), vol. 10, no. 1, pp. 334–337, Mar. 2022, doi: 10.1109/MGRS.2022.3144291.[20] G. Liu et al., “Fine-grained building roof instance segmen-tation based on domain adapted pretraining and com-posite dual-backbone,” in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), 2023. [21] X. Lu et al., “A strong vision transformer adapter with adaptive thresholding for fine-grained building clas-sification,” in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), 2023. [22] J. Hu, Z. Huang, F. Shen, D. He, and Q. Xian, “A bag of tricks for fine-grained roof extraction,” in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), 2023. [23] C. Lu et al., “A height-hierarchy guided dualdecoder net-work for single view building extraction and height es-timation,” in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), 2023. [24] X. Lu et al., “Trident cooperation network for building ex-traction and height estimation,” in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), 2023. [25] Y. Guo and Z. Wang, “Height estimation based on seman-tic segmentation,” in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), 2023. [26] J. Hu, Z. Huang, F. Shen, D. He, and Q. Xian, “A robust method for roof extraction and height estimation,” in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), 2023. 

Report on the 2023 IEEE GRSS Data Fusion Contest:Large-Scale Fine-Grained Building Classification for Semantic Urban Reconstruction [Technical Committees]

Hänsch, Ronny

Institute of Transport Research:Publications

Report on the 2023 IEEE GRSS Data Fusion Contest: Large-Scale Fine-Grained Building Classification for Semantic Urban Reconstruction [Technical Committees]

https://ris.utwente.nl/ws/files/350434224/Report_on_the_2023_IEEE_GRSS_Data_Fusion_Contest_Large-Scale_Fine-Grained_Building_Classification_for_Semantic_Urban_Reconstruction_Technical_Committees.pdf

Report on the 2023 IEEE GRSS Data Fusion Contest:Large-Scale Fine-Grained Building Classification for Semantic Urban Reconstruction [Technical Committees]

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information

Institute of Transport Research:Publications