Counterfactual Cross-modality Reasoning for Weakly Supervised Video Moment Localization

Video moment localization aims to retrieve the target segment of an untrimmed video according to the natural language query. Weakly supervised methods gains attention recently, as the precise temporal location of the target segment is not always available. However, one of the greatest challenges encountered by the weakly supervised method is implied in the mismatch between the video and language induced by the coarse temporal annotations. To refine the vision-language alignment, recent works contrast the cross-modality similarities driven by reconstructing masked queries between positive and negative video proposals. However, the reconstruction may be influenced by the latent spurious correlation between the unmasked and the masked parts, which distorts the restoring process and further degrades the efficacy of contrastive learning since the masked words are not completely reconstructed from the cross-modality knowledge. In this paper, we discover and mitigate this spurious correlation through a novel proposed counterfactual cross-modality reasoning method. Specifically, we first formulate query reconstruction as an aggregated causal effect of cross-modality and query knowledge. Then by introducing counterfactual cross-modality knowledge into this aggregation, the spurious impact of the unmasked part contributing to the reconstruction is explicitly modeled. Finally, by suppressing the unimodal effect of masked query, we can rectify the reconstructions of video proposals to perform reasonable contrastive learning. Extensive experimental evaluations demonstrate the effectiveness of our proposed method. The code is available at \href{https://github.com/sLdZ0306/CCR}{https://github.com/sLdZ0306/CCR}.Comment: Accepted by ACM MM 202

A Subblock Partition Of Multi-Layer Pattern Based Image Classification Approach

Since traditional partition approach may construct very different image representation because of the changed locations of objects in the same image, a subblock partition of multi-layer pattern method for image representation is proposed. The saliency windows straddled by superpixels are utilized to partition the image into multi-layer pattern subblocks. Then all the subblocks are combined to a three order tensor. Comparing to the results of image classification item of Pascal Voc 2007 Challenge，it indicates that the proposed representation method is robust to the varied object locations and achieves better performance than other approaches. CCS Concepts Computing methodologies➝Computer vision • Computing methodologies➝ Machine learnin

Low-Rank Optimization Dictionary Training for Image Classification

Bag-of-words model has been extremely popular in image categorization. The method of constructing the dictionary is important. In this paper a category constrained low-rank optimization dictionary training approach is proposed for the dictionary construction. Through the low-rank optimization, the rank of the coefficient matrix constructed by same category images is minimized. Experimental results show that the proposed method can obtain better performance on two standard image databases (Caltech-101 and Caltech-256) than not employing the category constrained low-rank optimization

Human-guided safe and efficient trajectory replanning for unmanned aerial vehicles

Safe and efficient local trajectory replanning is essential for the navigation of unmanned aerial vehicles (UAVs). Take the quadrotor as an example, most research works focus on the static or fully mapped environment. Flying in a dynamic environment for autonomous quadrotors is still a tricky problem. However, with the emergence of first-person-view Drone Racing in recent years, professional human pilots have shown highly-skilled techniques for navigating quadrotors to avoid collisions at high speed. Therefore, this work uses the intelligence of human users in perception and decision-making and proposes a human-guided trajectory replanning (HTP) system for the safe and efficient flight operation of quadrotors. A non-constraint optimization problem is formulated, and human guidance is designed as one term of the cost functions. The proposed approach is validated in the AirSim simulation environment. The result shows that HTP saves optimization time by 58% compared with the non-human guidance (Non-HG) baseline. In addition, the HTP can assist quadrotors to pass the specified target at a higher speed and comply better with human preferences than the Non-HG approach.Agency for Science, Technology and Research (A*STAR)Civil Aviation Authority of Singapore (CAAS)Nanyang Technological UniversitySubmitted/Accepted versionThis work was supported in part by A*STAR project (No. W1925d0046), and the SUG-NAP project of Nanyang Technological University. This research / project* is supported by the Civil Aviation Authority of Singapore and Nanyang Technological University, Singapore under their collaboration in the Air Traffic Management Research Institute. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the Civil Aviation Authority of Singapore

China’s Public Firms’ Attitudes towards Environmental Protection Based on Sentiment Analysis and Random Forest Models

In this article, we investigated changes in public firms’ attitudes towards environmental protection in 2018–2021 in China. We crawled the firm–investor Q&A record on the website of East Money, extracted the carbon- and environment-related corpus, and then applied the sentiment analysis method of NLP (natural language processing) to calculate the sentiment weight of each firm-level record to estimate the attitude before and after towards carbon reduction. We found that there were significant changes in firms’ attitudes towards carbon reduction and environmental protection after the COVID-19 pandemic and the implementation of environment-related policies. We also found a heterogeneous effect of the attitude in different industries. In addition, we built several models to examine the relationship between a firm’s carbon reduction attitude and its financial performance. We found that: A goal with consequent specific policies can raise the positive attitudes of firms toward carbon reduction topics; firms’ attitudes toward ecological topics are different from industry to industry, which means that there are different needs and situations in the trend of carbon reduction from industry to industry. COVID-19 influenced firms’ attitudes toward carbon reduction and environmental protection, calling back the classic dilemma or trilemma of economic growth, carbon reduction, and energy consumption or, perhaps, epidemic control today. The stock situation also influenced the attitude toward environmental protection

Intercalation Synthesis of Prussian Blue Analogue Nanocone and Their Conversion into Fe-Doped Co_<i>x</i>P Nanocone for Enhanced Hydrogen Evolution

Compared with the monometallic phosphides, bimetallic phosphides can further improve the catalytic performance for hydrogen evolution reaction (HER). As such, the rational design and facile synthesis of bimetallic-based phosphides with well-controlled architectures and compositions is of scientific and technological importance. In this work, Fe–Co Prussian blue analogue (PBA) nanocones (NCs) have been successfully fabricated via an intercalation reaction strategy by utilizing layer structured α-Co­(OH)₂ NCs as self-sacrificing templates. After calcination and phosphorization process, Fe–Co PBA NCs can be converted to Fe-doped Co_<i>x</i>P NCs without obvious shrinkage. Electrochemical tests show that Fe incorporation can effectively promote the electrocatalytic activities of Co_<i>x</i>P. This simple and effective method will be of benefit for the development of other functional Co-based bimetallic compounds. Furthermore, this strategy can possibly be extended to fabricate a series of PBA materials with special structure and novel morphology, which can serve as a promising platform for diverse applications, especially in energy storage and conversion