Bidirectional Contrastive Split Learning for Visual Question Answering

Visual Question Answering (VQA) based on multi-modal data facilitates real-life applications such as home robots and medical diagnoses. One significant challenge is to devise a robust decentralized learning framework for various client models where centralized data collection is refrained due to confidentiality concerns. This work aims to tackle privacy-preserving VQA by decoupling a multi-modal model into representation modules and a contrastive module and leveraging inter-module gradients sharing and inter-client weight sharing. To this end, we propose Bidirectional Contrastive Split Learning (BiCSL) to train a global multi-modal model on the entire data distribution of decentralized clients. We employ the contrastive loss that enables a more efficient self-supervised learning of decentralized modules. Comprehensive experiments are conducted on the VQA-v2 dataset based on five SOTA VQA models, demonstrating the effectiveness of the proposed method. Furthermore, we inspect BiCSL's robustness against a dual-key backdoor attack on VQA. Consequently, BiCSL shows much better robustness to the multi-modal adversarial attack compared to the centralized learning method, which provides a promising approach to decentralized multi-modal learning

Mobility entropy and message routing in community-structured delay tolerant networks

Many message routing schemes have been proposed in the context of delay tolerant networks (DTN) and intermittently connected mobile networks (ICMN). Those routing schemes are tested on specific environments that involve particular mobility complexity whether they are random-based or soci-ologically organized. We, in this paper, propose community structured environment (CSE) and mobility entropy to dis-cuss the effect of node mobility complexity on message rout-ing performance. We also propose potential-based entropy adaptive routing (PEAR) that adaptively carries messages over the change of mobility entropy. According to our simu-lation, PEAR has achieved high delivery rate on wide range of mobility entropy, while link-state routing has worked well only at small entropy scenarios and controlled replication-based routing only at large entropy environments

Federated Phish Bowl: LSTM-Based Decentralized Phishing Email Detection

With increasingly more sophisticated phishing campaigns in recent years, phishing emails lure people using more legitimate-looking personal contexts. To tackle this problem, instead of traditional heuristics-based algorithms, more adaptive detection systems such as natural language processing (NLP)-powered approaches are essential to understanding phishing text representations. Nevertheless, concerns surrounding the collection of phishing data that might cover confidential information hinder the effectiveness of model learning. We propose a decentralized phishing email detection framework called Federated Phish Bowl (FedPB) which facilitates collaborative phishing detection with privacy. In particular, we devise a knowledge-sharing mechanism with federated learning (FL). Using long short-term memory (LSTM) for phishing detection, the framework adapts by sharing a global word embedding matrix across the clients, with each client running its local model with Non-IID data. We collected the most recent phishing samples to study the effectiveness of the proposed method using different client numbers and data distributions. The results show that FedPB can attain a competitive performance with a centralized phishing detector, with generality to various cases of FL retaining a prediction accuracy of 83%.Comment: To be published in 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC

Resilience of Wireless Ad Hoc Federated Learning against Model Poisoning Attacks

Wireless ad hoc federated learning (WAFL) is a fully decentralized collaborative machine learning framework organized by opportunistically encountered mobile nodes. Compared to conventional federated learning, WAFL performs model training by weakly synchronizing the model parameters with others, and this shows great resilience to a poisoned model injected by an attacker. In this paper, we provide our theoretical analysis of the WAFL's resilience against model poisoning attacks, by formulating the force balance between the poisoned model and the legitimate model. According to our experiments, we confirmed that the nodes directly encountered the attacker has been somehow compromised to the poisoned model but other nodes have shown great resilience. More importantly, after the attacker has left the network, all the nodes have finally found stronger model parameters combined with the poisoned model. Most of the attack-experienced cases achieved higher accuracy than the no-attack-experienced cases.Comment: 10 pages, 7 figures, to be published in IEEE International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications 202

Associative Transformer Is A Sparse Representation Learner

Emerging from the monolithic pairwise attention mechanism in conventional Transformer models, there is a growing interest in leveraging sparse interactions that align more closely with biological principles. Approaches including the Set Transformer and the Perceiver employ cross-attention consolidated with a latent space that forms an attention bottleneck with limited capacity. Building upon recent neuroscience studies of Global Workspace Theory and associative memory, we propose the Associative Transformer (AiT). AiT induces low-rank explicit memory that serves as both priors to guide bottleneck attention in the shared workspace and attractors within associative memory of a Hopfield network. Through joint end-to-end training, these priors naturally develop module specialization, each contributing a distinct inductive bias to form attention bottlenecks. A bottleneck can foster competition among inputs for writing information into the memory. We show that AiT is a sparse representation learner, learning distinct priors through the bottlenecks that are complexity-invariant to input quantities and dimensions. AiT demonstrates its superiority over methods such as the Set Transformer, Vision Transformer, and Coordination in various vision tasks

Cooperative awareness using roadside unit networks in mixed traffic

International audienceVehicle-to-vehicle (V2V) messaging is an indispensable component of connected autonomous vehicle systems. Although V2V standards have been specified by the European Union, United States, and Japan, the deployment phase represents mixed traffic in which connected and legacy vehicles co-exist. To enhance cooperative awareness in this mixed traffic, we assessed the special roadside unit that we developed in our previous work that generates required V2V messages on behalf of sensed target vehicles. In this paper, we extend our earlier work to propose a system called Grid Proxy Cooperative Awareness Message to broaden the cooperative awareness message dissemination area by connecting infrastructure using high-speed roadside networks. To minimize delay in message delivery, we designed the proposed system to use edge computing. The proposed scheme delivers cooperative messages to a wider area with a low delay and a high packet delivery ratio by prioritizing packets by their respective safety contributions. Our simulation results indicate that the proposed scheme efficiently delivers messages in heavy road traffic conditions modeled on real maps of Tokyo and Paris

Wireless Ad Hoc Federated Learning: A Fully Distributed Cooperative Machine Learning

Privacy-sensitive data is stored in autonomous vehicles, smart devices, or sensor nodes that can move around with making opportunistic contact with each other. Federation among such nodes was mainly discussed in the context of federated learning with a centralized mechanism in many works. However, because of multi-vendor issues, those nodes do not want to rely on a specific server operated by a third party for this purpose. In this paper, we propose a wireless ad hoc federated learning (WAFL) -- a fully distributed cooperative machine learning organized by the nodes physically nearby. WAFL can develop generalized models from Non-IID datasets stored in distributed nodes locally by exchanging and aggregating them with each other over opportunistic node-to-node contacts. In our benchmark-based evaluation with various opportunistic networks, WAFL has achieved higher accuracy of 94.8-96.3% than the self-training case of 84.7%. All our evaluation results show that WAFL can train and converge the model parameters from highly-partitioned Non-IID datasets over opportunistic networks without any centralized mechanisms.Comment: 14 pages, 8 figures, 2 table

Unidirectional Link-Aware DTN-based Sensor Network in Building Monitoring Scenario

Abstract-The performance of wireless sensor network in building monitoring system (BMS) is often deteriorated by intermittently-connected and unidirectional links occurring in building environment. With DTN-based approach, Potentialbased Entropy Adaptive Routing (PEAR) protocol can achieve high reliability and scalability over intermittently-connected mesh network in the building scenario. However, the result of high delivery latency caused by ignoring the presence of unidirectional links may not be acceptable in BMS. In this paper, we propose Unidirectional Link-Aware Next-hop Selection (ULANS), the technique of detecting unidirectional links and the new nexthop selection scheme for PEAR. The real-world experimental result shows that ULANS can avoid choosing unidirectional links as the next-hop and improve delivery latency of PEAR

Bottleneck Based Gridlock Prediction in an Urban Road Network Using Long Short-Term Memory

The traffic bottlenecks in urban road networks are more challenging to investigate and discover than in freeways or simple arterial networks. A bottleneck indicates the congestion evolution and queue formation, which consequently disturb travel delay and degrade the urban traffic environment and safety. For urban road networks, sensors are needed to cover a wide range of areas, especially for bottleneck and gridlock analysis, requiring high installation and maintenance costs. The emerging widespread availability of GPS vehicles significantly helps to overcome the geographic coverage and spacing limitations of traditional fixed-location detector data. Therefore, this study investigated GPS vehicles that have passed through the links in the simulated gridlock-looped intersection area. The sample size estimation is fundamental to any traffic engineering analysis. Therefore, this study tried a different number of sample sizes to analyze the severe congestion state of gridlock. Traffic condition prediction is one of the primary components of intelligent transportation systems. In this study, the Long Short-Term Memory (LSTM) neural network was applied to predict gridlock based on bottleneck states of intersections in the simulated urban road network. This study chose to work on the Chula-Sathorn SUMO Simulator (Chula-SSS) dataset. It was calibrated with the past actual traffic data collection by using the Simulation of Urban MObility (SUMO) software. The experiments show that LSTM provides satisfactory results for gridlock prediction with temporal dependencies. The reported prediction error is based on long-range time dependencies on the respective sample sizes using the calibrated Chula-SSS dataset. On the other hand, the low sampling rate of GPS trajectories gives high RMSE and MAE error, but with reduced computation time. Analyzing the percentage of simulated GPS data with different random seed numbers suggests the possibility of gridlock identification and reports satisfying prediction errors. Document type: Articl

Live E! Project: Establishment of Infrastructure Sharing Environmental Information

The Live E! project is an open research consortium among industry and academia to explore the platform to share the digital information related with the earth and our living environment. We have getting a lot of low cost sensor nodes with Internet connectivity. The deployment of broad-band and ubiquitous networks will enable autonomous and global digital information sharing over the globe. In this paper, we describe the technical and operational overview of Live E! project, while discussing the objective, such as education, disaster protection/reduction/recovery or busi-ness cases, and goal of this project activity. 1