Improving the Knowledge Gradient Algorithm

The knowledge gradient (KG) algorithm is a popular policy for the best arm identification (BAI) problem. It is built on the simple idea of always choosing the measurement that yields the greatest expected one-step improvement in the estimate of the best mean of the arms. In this research, we show that this policy has limitations, causing the algorithm not asymptotically optimal. We next provide a remedy for it, by following the manner of one-step look ahead of KG, but instead choosing the measurement that yields the greatest one-step improvement in the probability of selecting the best arm. The new policy is called improved knowledge gradient (iKG). iKG can be shown to be asymptotically optimal. In addition, we show that compared to KG, it is easier to extend iKG to variant problems of BAI, with the $\epsilon$-good arm identification and feasible arm identification as two examples. The superior performances of iKG on these problems are further demonstrated using numerical examples.Comment: 32 pages, 42 figure

A robust modulation classification method using convolutional neural networks

Automatic modulation classification (AMC) is a core technique in noncooperative communication systems. In particular, feature-based (FB) AMC algorithms have been widely studied. Current FB AMC methods are commonly designed for a limited set of modulation and lack of generalization ability; to tackle this challenge, a robust AMC method using convolutional neural networks (CNN) is proposed in this paper. In total, 15 different modulation types are considered. The proposed method can classify the received signal directly without feature extracion, and it can automatically learn features from the received signals. The features learned by the CNN are presented and analyzed. The robust features of the received signals in a specific SNR range are studied. The accuracy of classification using CNN is shown to be remarkable, particularly for low SNRs. The generalization ability of robust features is also proven to be excellent using the support vector machine (SVM). Finally, to help us better understand the process of feature learning, some outputs of intermediate layers of the CNN are visualized

Overview of Sensing Attacks on Autonomous Vehicle Technologies and Impact on Traffic Flow

While perception systems in Connected and Autonomous Vehicles (CAVs), which encompass both communication technologies and advanced sensors, promise to significantly reduce human driving errors, they also expose CAVs to various cyberattacks. These include both communication and sensing attacks, which potentially jeopardize not only individual vehicles but also overall traffic safety and efficiency. While much research has focused on communication attacks, sensing attacks, which are equally critical, have garnered less attention. To address this gap, this study offers a comprehensive review of potential sensing attacks and their impact on target vehicles, focusing on commonly deployed sensors in CAVs such as cameras, LiDAR, Radar, ultrasonic sensors, and GPS. Based on this review, we discuss the feasibility of integrating hardware-in-the-loop experiments with microscopic traffic simulations. We also design baseline scenarios to analyze the macro-level impact of sensing attacks on traffic flow. This study aims to bridge the research gap between individual vehicle sensing attacks and broader macroscopic impacts, thereby laying the foundation for future systemic understanding and mitigation

AdaTT: Adaptive Task-to-Task Fusion Network for Multitask Learning in Recommendations

Multi-task learning (MTL) aims at enhancing the performance and efficiency of machine learning models by training them on multiple tasks simultaneously. However, MTL research faces two challenges: 1) modeling the relationships between tasks to effectively share knowledge between them, and 2) jointly learning task-specific and shared knowledge. In this paper, we present a novel model Adaptive Task-to-Task Fusion Network (AdaTT) to address both challenges. AdaTT is a deep fusion network built with task specific and optional shared fusion units at multiple levels. By leveraging a residual mechanism and gating mechanism for task-to-task fusion, these units adaptively learn shared knowledge and task specific knowledge. To evaluate the performance of AdaTT, we conduct experiments on a public benchmark and an industrial recommendation dataset using various task groups. Results demonstrate AdaTT can significantly outperform existing state-of-the-art baselines

Flexible Robotic Scanning Device for Intraoperative Endomicroscopy in MIS

Optical biopsy methods such as probe-based confocal endomicroscopy can provide intraoperative real-time assessment of tumour margins, including during minimally invasive surgery with flexible endoscopes or robotic platforms. Mosaics can be produced by translating the probe across the target, but it remains difficult to scan over a large field-of-view with a flexible endomicroscope. In this paper, we have developed a novel flexible scanning device for intraoperative endomicroscopy in MIS. A Schott leached imaging bundle was integrated into the device and enables the approach, via a flexible path, to deep and narrow spaces in the human body that otherwise would not accessible. The proposed device uses a gear-based flexible concentric tube scanning mechanism to facilitate large field-of-view mosaicing. Experimental results show that the device is able to scan different surface trajectories (e.g. a spiral pattern over a hemi-spherical surface). Results from lens tissue paper and porcine liver tissue are demonstrated, illustrating a viable scanning approach for endomicroscopy in MIS

Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy

As a key component to intuitive cognition and reasoning solutions in human intelligence, causal knowledge provides great potential for reinforcement learning (RL) agents' interpretability towards decision-making by helping reduce the searching space. However, there is still a considerable gap in discovering and incorporating causality into RL, which hinders the rapid development of causal RL. In this paper, we consider explicitly modeling the generation process of states with the causal graphical model, based on which we augment the policy. We formulate the causal structure updating into the RL interaction process with active intervention learning of the environment. To optimize the derived objective, we propose a framework with theoretical performance guarantees that alternates between two steps: using interventions for causal structure learning during exploration and using the learned causal structure for policy guidance during exploitation. Due to the lack of public benchmarks that allow direct intervention in the state space, we design the root cause localization task in our simulated fault alarm environment and then empirically show the effectiveness and robustness of the proposed method against state-of-the-art baselines. Theoretical analysis shows that our performance improvement attributes to the virtuous cycle of causal-guided policy learning and causal structure learning, which aligns with our experimental results

Tunable topological phase transition in soft Rayleigh beam system with imperfect interfaces

Acoustic metamaterials, particularly the topological insulators, exhibit exceptional wave characteristics that have sparked considerable research interest. The study of imperfect interfaces affect is of significant importance for the modeling of wave propagation behavior in topological insulators. This paper models a soft Rayleigh beam system with imperfect interfaces, and investigates its topological phase transition process tuned by mechanical loadings. The model reveals that the topological phase transition process can be observed by modifying the distance between imperfect interfaces in the system. When a uniaxial stretch is applied, the topological phase transition points for longitudinal waves decrease within a limited frequency range, while they increase within a larger frequency scope for transverse waves. Enhancing the rigidity of the imperfect interfaces also enables shifting of the topological phase transition point within a broader frequency range for longitudinal waves and a confined range for transverse waves. The transition of topologically protected interface modes in the transmission performance of a twenty-cell system is verified, which include altering frequencies, switching from interface mode to edge mode. Overall, this study provides a new approach and guideline for controlling topological phase transition in composite and soft phononic crystal systems.Comment: 39 pages,8 figure