Large-scale Weakly Supervised Learning for Road Extraction from Satellite Imagery

Automatic road extraction from satellite imagery using deep learning is a viable alternative to traditional manual mapping. Therefore it has received considerable attention recently. However, most of the existing methods are supervised and require pixel-level labeling, which is tedious and error-prone. To make matters worse, the earth has a diverse range of terrain, vegetation, and man-made objects. It is well known that models trained in one area generalize poorly to other areas. Various shooting conditions such as light and angel, as well as different image processing techniques further complicate the issue. It is impractical to develop training data to cover all image styles. This paper proposes to leverage OpenStreetMap road data as weak labels and large scale satellite imagery to pre-train semantic segmentation models. Our extensive experimental results show that the prediction accuracy increases with the amount of the weakly labeled data, as well as the road density in the areas chosen for training. Using as much as 100 times more data than the widely used DeepGlobe road dataset, our model with the D-LinkNet architecture and the ResNet-50 backbone exceeds the top performer of the current DeepGlobe leaderboard. Furthermore, due to large-scale pre-training, our model generalizes much better than those trained with only the curated datasets, implying great application potential

Finding the Pillars of Strength for Multi-Head Attention

Recent studies have revealed some issues of Multi-Head Attention (MHA), e.g., redundancy and over-parameterization. Specifically, the heads of MHA were originally designed to attend to information from different representation subspaces, whereas prior studies found that some attention heads likely learn similar features and can be pruned without harming performance. Inspired by the minimum-redundancy feature selection, we assume that focusing on the most representative and distinctive features with minimum resources can mitigate the above issues and lead to more effective and efficient MHAs. In particular, we propose Grouped Head Attention, trained with a self-supervised group constraint that group attention heads, where each group focuses on an essential but distinctive feature subset. We additionally propose a Voting-to-Stay procedure to remove redundant heads, thus achieving a transformer with lighter weights. Moreover, our method achieves significant performance gains on three well-established tasks while considerably compressing parameters.Comment: In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 2023

Reduced scale model test on cable membrane roof of Shangai Expo Central Axis

p. 2008-2018In this paper a reduced scale model test on cable membrane roof of Shanghai Expo Central Axis is introduced. The membrane pre-stresses, cable forces and membrane geometry at the initial state are carefully inspected. Numerical form-finding analysis is also carried out and its result is compared with the inspecton. The behaviors of the membrane roof under breaking of cables are observed. Test proves the practicability of the project in aspects of system safety, analysis and inspection.Zhang, Q.; Yang, Z.; Chen, L.; Tang, H.; Zhu, B. (2010). Reduced scale model test on cable membrane roof of Shangai Expo Central Axis. Editorial Universitat Politècnica de València. http://hdl.handle.net/10251/718

Large Language Models for Automated Open-domain Scientific Hypotheses Discovery

Hypothetical induction is recognized as the main reasoning type when scientists make observations about the world and try to propose hypotheses to explain those observations. Past research on hypothetical induction has a limited setting that (1) the observation annotations of the dataset are not raw web corpus but are manually selected sentences (resulting in a close-domain setting); and (2) the ground truth hypotheses annotations are mostly commonsense knowledge, making the task less challenging. In this work, we propose the first NLP dataset for social science academic hypotheses discovery, consisting of 50 recent papers published in top social science journals. Raw web corpora that are necessary for developing hypotheses in the published papers are also collected in the dataset, with the final goal of creating a system that automatically generates valid, novel, and helpful (to human researchers) hypotheses, given only a pile of raw web corpora. The new dataset can tackle the previous problems because it requires to (1) use raw web corpora as observations; and (2) propose hypotheses even new to humanity. A multi-module framework is developed for the task, as well as three different feedback mechanisms that empirically show performance gain over the base framework. Finally, our framework exhibits high performance in terms of both GPT-4 based evaluation and social science expert evaluation

A Survey on Semantic Processing Techniques

Semantic processing is a fundamental research domain in computational linguistics. In the era of powerful pre-trained language models and large language models, the advancement of research in this domain appears to be decelerating. However, the study of semantics is multi-dimensional in linguistics. The research depth and breadth of computational semantic processing can be largely improved with new technologies. In this survey, we analyzed five semantic processing tasks, e.g., word sense disambiguation, anaphora resolution, named entity recognition, concept extraction, and subjectivity detection. We study relevant theoretical research in these fields, advanced methods, and downstream applications. We connect the surveyed tasks with downstream applications because this may inspire future scholars to fuse these low-level semantic processing tasks with high-level natural language processing tasks. The review of theoretical research may also inspire new tasks and technologies in the semantic processing domain. Finally, we compare the different semantic processing techniques and summarize their technical trends, application trends, and future directions.Comment: Published at Information Fusion, Volume 101, 2024, 101988, ISSN 1566-2535. The equal contribution mark is missed in the published version due to the publication policies. Please contact Prof. Erik Cambria for detail

Eau

Modeling aluminum (Al) particle-air detonation is extremely difficult because the combustion is shock-induced, and there are multi-phase heat release and transfer in supersonic flows. Existing models typically use simplified combustion to reproduce the detonation velocity, which introduces many unresolved problems. The hybrid combustion model, coupling both the diffused- and kinetics-controlled combustion, is proposed recently, and then improved to include the effects of realistic heat capacities dependent on the particle temperature. In the present study, 2D cellular Al particle-air detonations are simulated with the realistic heat capacity model and its effects on the detonation featured parameters, such as the detonation velocity and cell width, are analyzed. Numerical results show that cell width increases as particle diameter increases, similarly to the trend observed with the original model, but the cell width is underestimated without using the realistic heat capacities. Further analysis is performed by averaging the 2D cellular detonations to quasi-1D, demonstrating that the length scale of quasi-1D detonation is larger than that of truly 1D model, similar to gaseous detonations

Initiation structure of oblique detonation waves behind conical shocks

The understanding of oblique detonation dynamics has both inherent basic research value for high-speed compressible reacting flow and propulsion application in hypersonic aerospace systems. In this study, the oblique detonation structures formed by semi-infinite cones are investigated numerically by solving the unsteady, two-dimensional axisymmetric Euler equations with a one-step irreversible Arrhenius reaction model. The present simulation results show that a novel wave structure, featured by two distinct points where there is close-coupling between the shock and combustion front, is depicted when either the cone angle or incident Mach number is reduced. This structure is analyzed by examining the variation of the reaction length scale and comparing the flow field with that of planar, wedge-induced oblique detonations. Further simulations are performed to study the effects of chemical length scale and activation energy, which are both found to influence the formation of this novel structure. The initiation mechanism behind the conical shock is discussed to investigate the interplay between the effect of the Taylor-Maccoll flow, front curvature, and energy releases from the chemical reaction in conical oblique detonations. The observed flow fields are interpreted by means of the energetic limit as in the critical regime for initiation of detonation

Effects of inflow Mach number on oblique detonation initiation with a two-step induction-reaction kinetic model

Oblique detonations induced by two-dimensional, semi-infinite wedges are simulated by solving numerically the reactive Euler equations with a two-step induction-reaction kinetic model. Previous results obtained with other models have demonstrated that for the low inflow Mach number M0 regime past a critical value, the wave in the shocked gas changes from an oblique reactive wave front into a secondary oblique detonation wave (ODW). The present numerical results not only confirm the existence of such critical phenomenon, but also indicate that the structural shift is induced by the variation of the main ODW front which becomes sensitive to M0 near a critical value. Below the critical M0,cr, oscillations of the initiation structure are observed and become severe with further decrease of M0. For low M0 cases, the non-decaying oscillation of the initiation structure exists after a sufficiently long-time computation, suggesting the quasi-steady balance of initiation wave systems. By varying the heat release rate controlled by kR, the pre-exponential factor of the second reaction step, the morphology of initiation structures does not vary for M0 = 10 cases but varies for M0 = 9 cases, demonstrating that the effects of heat release rate become more prominent when M0 decreases. The instability parameter χ is introduced to quantify the numerical results. Although χ cannot reveal the detailed mechanism of the structural shift, a linear relation between χ and kR exists at the critical condition, providing an empirical criterion to predict the structural variation of the initiation structure

ResMem: Learn what you can and memorize the rest

The impressive generalization performance of modern neural networks is attributed in part to their ability to implicitly memorize complex training patterns. Inspired by this, we explore a novel mechanism to improve model generalization via explicit memorization. Specifically, we propose the residual-memorization (ResMem) algorithm, a new method that augments an existing prediction model (e.g. a neural network) by fitting the model's residuals with a $k$-nearest neighbor based regressor. The final prediction is then the sum of the original model and the fitted residual regressor. By construction, ResMem can explicitly memorize the training labels. Empirically, we show that ResMem consistently improves the test set generalization of the original prediction model across various standard vision and natural language processing benchmarks. Theoretically, we formulate a stylized linear regression problem and rigorously show that ResMem results in a more favorable test risk over the base predictor