Interpretation on Multi-modal Visual Fusion

In this paper, we present an analytical framework and a novel metric to shed light on the interpretation of the multimodal vision community. Our approach involves measuring the proposed semantic variance and feature similarity across modalities and levels, and conducting semantic and quantitative analyses through comprehensive experiments. Specifically, we investigate the consistency and speciality of representations across modalities, evolution rules within each modality, and the collaboration logic used when optimizing a multi-modality model. Our studies reveal several important findings, such as the discrepancy in cross-modal features and the hybrid multi-modal cooperation rule, which highlights consistency and speciality simultaneously for complementary inference. Through our dissection and findings on multi-modal fusion, we facilitate a rethinking of the reasonability and necessity of popular multi-modal vision fusion strategies. Furthermore, our work lays the foundation for designing a trustworthy and universal multi-modal fusion model for a variety of tasks in the future.Comment: This version was under review since 2023/3/

A Scalable Neural Network for DSIC Affine Maximizer Auction Design

Automated auction design aims to find empirically high-revenue mechanisms through machine learning. Existing works on multi item auction scenarios can be roughly divided into RegretNet-like and affine maximizer auctions (AMAs) approaches. However, the former cannot strictly ensure dominant strategy incentive compatibility (DSIC), while the latter faces scalability issue due to the large number of allocation candidates. To address these limitations, we propose AMenuNet, a scalable neural network that constructs the AMA parameters (even including the allocation menu) from bidder and item representations. AMenuNet is always DSIC and individually rational (IR) due to the properties of AMAs, and it enhances scalability by generating candidate allocations through a neural network. Additionally, AMenuNet is permutation equivariant, and its number of parameters is independent of auction scale. We conduct extensive experiments to demonstrate that AMenuNet outperforms strong baselines in both contextual and non-contextual multi-item auctions, scales well to larger auctions, generalizes well to different settings, and identifies useful deterministic allocations. Overall, our proposed approach offers an effective solution to automated DSIC auction design, with improved scalability and strong revenue performance in various settings.Comment: NeurIPS 2023 (spotlight

KALMANBOT: KalmanNet-Aided Bollinger Bands for Pairs Trading

Pairs trading is a family of trading policies based on monitoring the relationships between pairs of assets. A common pairs trading approach relies on state space (SS) modeling, from which financial indicators can be obtained with low complexity and latency using a Kalman filter (KF), and processed using classic policies such as Bollinger bands (BB). However, such SS models are inherently approximated and mismatched, often degrading the revenue. In this work we propose KalmanBOT, a data-aided policy that preserves the advantages of KF-aided BB policies while leveraging data to overcome the approximated nature of the SS model. We adopt the recent KalmanNet architecture, and approximate the BB policy with a differentiable mapping, converting the policy into a trainable model. We empirically demonstrate that KalmanBOT yields improved rewards compared with model-based and data-driven benchmarks

Have media texts become more humorous?

As a research topic, humour has drawn much attention from multiple disciplines including linguistics. Based on Engelthaler & Hills’ (2018) humour scale, this study developed a measure named Humour Index (HMI) to quantify the degree of humour of texts. This measure was applied to examine the diachronic changes in the degree of humour of American newspapers and magazines across a time span of 118 years (1900-2017) with the use of texts from Corpus of Historical American English (COHA). Besides, the study also discussed the contributions of different types of words to the degree of humour in the two genres. The results show significant uptrends in the degree of humour of both newspapers and magazines in the examined period. Moreover, derogatory and offensive words are found to be less frequently used than other categories of words in both genres. This study provides both theoretical and methodological implications for humour studies and claims or hypotheses of previous research, such as infotainment and linguistic positivity bias

Relational Learning between Multiple Pulmonary Nodules via Deep Set Attention Transformers

Diagnosis and treatment of multiple pulmonary nodules are clinically important but challenging. Prior studies on nodule characterization use solitary-nodule approaches on multiple nodular patients, which ignores the relations between nodules. In this study, we propose a multiple instance learning (MIL) approach and empirically prove the benefit to learn the relations between multiple nodules. By treating the multiple nodules from a same patient as a whole, critical relational information between solitary-nodule voxels is extracted. To our knowledge, it is the first study to learn the relations between multiple pulmonary nodules. Inspired by recent advances in natural language processing (NLP) domain, we introduce a self-attention transformer equipped with 3D CNN, named {NoduleSAT}, to replace typical pooling-based aggregation in multiple instance learning. Extensive experiments on lung nodule false positive reduction on LUNA16 database, and malignancy classification on LIDC-IDRI database, validate the effectiveness of the proposed method.Comment: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI 2020

Hydrogen Supply Infrastructure Network Planning Approach towards Chicken-egg Conundrum

In the early commercialization stage of hydrogen fuel cell vehicles (HFCVs), reasonable hydrogen supply infrastructure (HSI) planning decisions is a premise for promoting the popularization of HFCVs. However, there is a strong causality between HFCVs and hydrogen refueling stations (HRSs): the planning decisions of HRSs could affect the hydrogen refueling demand of HFCVs, and the growth of demand would in turn stimulate the further investment in HRSs, which is also known as the ``chicken and egg'' conundrum. Meanwhile, the hydrogen demand is uncertain with insufficient prior knowledge, and thus there is a decision-dependent uncertainty (DDU) in the planning issue. This poses great challenges to solving the optimization problem. To this end, this work establishes a multi-network HSI planning model coordinating hydrogen, power, and transportation networks. Then, to reflect the causal relationship between HFCVs and HRSs effectively without sufficient historical data, a distributionally robust optimization framework with decision-dependent uncertainty is developed. The uncertainty of hydrogen demand is modeled as a Wasserstein ambiguity set with a decision-dependent empirical probability distribution. Subsequently, to reduce the computational complexity caused by the introduction of a large number of scenarios and high-dimensional nonlinear constraints, we developed an improved distribution shaping method and techniques of scenario and variable reduction to derive the solvable form with less computing burden. Finally, the simulation results demonstrate that this method can reduce costs by at least 10.4% compared with traditional methods and will be more effective in large-scale HSI planning issues. Further, we put forward effective suggestions for the policymakers and investors to formulate relevant policies and decisions