Multimodal Co-learning: A Domain Adaptation Method for Building Extraction from Optical Remote Sensing Imagery

In this paper, we aim to improve the transfer learning ability of 2D convolutional neural networks (CNNs) for building extraction from optical imagery and digital surface models (DSMs) using a 2D-3D co-learning framework. Unlabeled target domain data are incorporated as unlabeled training data pairs to optimize the training procedure. Our framework adaptively transfers unsupervised mutual information between the 2D and 3D modality (i.e., DSM-derived point clouds) during the training phase via a soft connection, utilizing a predefined loss function. Experimental results from a spaceborne-to-airborne cross-domain case demonstrate that the framework we present can quantitatively and qualitatively improve the testing results for building extraction from single-modality optical images

A Co-learning Method to Utilize Optical Images and Photogrammetric Point Clouds for Building Extraction

Although deep learning techniques have brought unprecedented accuracy to automatic building extraction, several main issues still constitute an obstacle to effective and practical applications. The industry is eager for higher accuracy and more flexible data usage. In this paper, we present a co-learning framework applicable to building extraction from optical images and photogrammetric point clouds, which can take the advantage of 2D/3D multimodality data. Instead of direct information fusion, our co-learning framework adaptively exploits knowledge from another modality during the training phase with a soft connection, via a predefined loss function. Compared to conventional data fusion, this method is more flexible, as it is not mandatory to provide multimodality data in the test phase. We propose two types of co-learning: a standard version and an enhanced version, depending on whether unlabeled training data are employed. Experimental results from two data sets show that the methods we present can enhance the performance of both image and point cloud networks in few-shot tasks, as well as image networks when applying fully labeled training data sets

Motif-aware temporal GCN for fraud detection in signed cryptocurrency trust networks

Graph convolutional networks (GCNs) is a class of artificial neural networks for processing data that can be represented as graphs. Since financial transactions can naturally be constructed as graphs, GCNs are widely applied in the financial industry, especially for financial fraud detection. In this paper, we focus on fraud detection on cryptocurrency truct networks. In the literature, most works focus on static networks. Whereas in this study, we consider the evolving nature of cryptocurrency networks, and use local structural as well as the balance theory to guide the training process. More specifically, we compute motif matrices to capture the local topological information, then use them in the GCN aggregation process. The generated embedding at each snapshot is a weighted average of embeddings within a time window, where the weights are learnable parameters. Since the trust networks is signed on each edge, balance theory is used to guide the training process. Experimental results on bitcoin-alpha and bitcoin-otc datasets show that the proposed model outperforms those in the literature

Exploring Cross-city Semantic Segmentation of ALS Point Clouds

Deep learning models achieve excellent semantic segmentation results for airborne laser scanning (ALS) point clouds, if sufficient training data are provided. Increasing amounts of annotated data are becoming publicly available thanks to contributors from all over the world. However, models trained on a specific dataset typically exhibit poor performance on other datasets. I.e., there are significant domain shifts, as data captured in different environments or by distinct sensors have different distributions. In this work, we study this domain shift and potential strategies to mitigate it, using two popular ALS datasets: the ISPRS Vaihingen benchmark from Germany and the LASDU benchmark from China. We compare different training strategies for cross-city ALS point cloud semantic segmentation. In our experiments, we analyse three factors that may lead to domain shift and affect the learning: point cloud density, LiDAR intensity, and the role of data augmentation. Moreover, we evaluate a well-known standard method of domain adaptation, deep CORAL (Sun and Saenko, 2016). In our experiments, adapting the point cloud density and appropriate data augmentation both help to reduce the domain gap and improve segmentation accuracy. On the contrary, intensity features can bring an improvement within a dataset, but deteriorate the generalisation across datasets. Deep CORAL does not further improve the accuracy over the simple adaptation of density and data augmentation, although it can mitigate the impact of improperly chosen point density, intensity features, and further dataset biases like lack of diversity

Natural Language Interfaces for Tabular Data Querying and Visualization: A Survey

The emergence of natural language processing has revolutionized the way users interact with tabular data, enabling a shift from traditional query languages and manual plotting to more intuitive, language-based interfaces. The rise of large language models (LLMs) such as ChatGPT and its successors has further advanced this field, opening new avenues for natural language processing techniques. This survey presents a comprehensive overview of natural language interfaces for tabular data querying and visualization, which allow users to interact with data using natural language queries. We introduce the fundamental concepts and techniques underlying these interfaces with a particular emphasis on semantic parsing, the key technology facilitating the translation from natural language to SQL queries or data visualization commands. We then delve into the recent advancements in Text-to-SQL and Text-to-Vis problems from the perspectives of datasets, methodologies, metrics, and system designs. This includes a deep dive into the influence of LLMs, highlighting their strengths, limitations, and potential for future improvements. Through this survey, we aim to provide a roadmap for researchers and practitioners interested in developing and applying natural language interfaces for data interaction in the era of large language models.Comment: 20 pages, 4 figures, 5 tables. Submitted to IEEE TKD

Interactions between depositional regime and climate proxies in the northern South China Sea since the Last Glacial Maximum

Sedimentary deposits from the northern South China Sea (SCS) can provide important constraints on past changes in ocean currents and the East Asian summer monsoon (EASM) in this region. However, the interpretation of such records spanning the last deglaciation is complicated because sea-level change may also have influenced the depositional processes and patterns. Here, we present new records of grain size, clay mineralogy, and magnetic mineralogy spanning the past 24 kyr from both shallow and deep-water sediment cores in the northern SCS. Our multi-proxy comparison among multiple cores helps constrain the influence of sea-level change, providing confidence in interpreting the regional climate-forced signals. After accounting for the influence of sea-level change, we find that these multi-proxy records reflect a combination of changes in (a) the strength of the North Pacific Intermediate Water inflow, (b) the EASM strength, and (c) the Kuroshio Current extent. Overall, this study provides new insights into the roles of varying terrestrial weathering and oceanographic processes in controlling the depositional record on the northern SCS margin in response to climate and sea-level fluctuations

Traffic Optimization for Signalized Corridors (TOSCo) Phase 2 Project

DTFH6114H00002This report provides a roadmap to details contained in four TOSCo Phase 2 detailed reports that focus on specific aspects of the four key technical objectives undertaken in the TOSCo Phase 2 Project. The four key technical objectives that the project team focused their efforts on in the project were: 1. Implement TOSCo vehicle algorithm in vehicles 2. Implement TOSCo infrastructure algorithm in infrastructure components 3. Verify and refine TOSCo system in a closed course setting then exporting it onto an actual corridor 4. Assess TOSCo functional safety and performance using real-world observations and actual on-road dat