Collaborative Deep Reinforcement Learning for Joint Object Search

We examine the problem of joint top-down active search of multiple objects under interaction, e.g., person riding a bicycle, cups held by the table, etc.. Such objects under interaction often can provide contextual cues to each other to facilitate more efficient search. By treating each detector as an agent, we present the first collaborative multi-agent deep reinforcement learning algorithm to learn the optimal policy for joint active object localization, which effectively exploits such beneficial contextual information. We learn inter-agent communication through cross connections with gates between the Q-networks, which is facilitated by a novel multi-agent deep Q-learning algorithm with joint exploitation sampling. We verify our proposed method on multiple object detection benchmarks. Not only does our model help to improve the performance of state-of-the-art active localization models, it also reveals interesting co-detection patterns that are intuitively interpretable

Simulating inter-organizational collaboration network: a multi- relational and event-based approach

Abstract In this research, we study inter-organizational collaboration from the perspective of multi-relational networks. We develop an agent-based model to simulate how a collaboration network among organizations emerges from organizations' interactions through another network: the inter-organizational communication network. Our model adds links (or edges) into the collaboration network on the basis of events, which correspond to organizations' formation of collaborative teams for joint projects. The proposed approach also models the competitive yet non-exclusive dissemination of information among organizations, organizations' dynamic prioritization of candidate projects, and network-based influence. Applying the model to a case study of the humanitarian sector, we configure and validate the agent-based simulation, and use it to analyze how to promote inter-organizational humanitarian collaboration by encouraging communication. The simulation results suggest that encouraging communication between peripheral organizations can better promote collaboration than other strategies

Practical Collaborative Perception: A Framework for Asynchronous and Multi-Agent 3D Object Detection

In this paper, we improve the single-vehicle 3D object detection models using LiDAR by extending their capacity to process point cloud sequences instead of individual point clouds. In this step, we extend our previous work on rectification of the shadow effect in the concatenation of point clouds to boost the detection accuracy of multi-frame detection models. Our extension includes incorporating HD Map and distilling an Oracle model. Next, we further increase the performance of single-vehicle perception using multi-agent collaboration via Vehicle-to-everything (V2X) communication. We devise a simple yet effective collaboration method that achieves better bandwidth-performance tradeoffs than prior arts while minimizing changes made to single-vehicle detection models and assumptions on inter-agent synchronization. Experiments on the V2X-Sim dataset show that our collaboration method achieves 98% performance of the early collaboration while consuming the equivalent amount of bandwidth usage of late collaboration which is 0.03% of early collaboration. The code will be released at https://github.com/quan-dao/practical-collab-perception.Comment: Work in progres

Cognitive Network Modeling as a Basis for Characterizing Human Communication Dynamics and Belief Contagion in Technology Adoption

Societal level macro models of social behavior do not sufficiently capture nuances needed to adequately represent the dynamics of person-to-person interactions. Likewise, individual agent level micro models have limited scalability - even minute parameter changes can drastically affect a model's response characteristics. This work presents an approach that uses agent-based modeling to represent detailed intra- and inter-personal interactions, as well as a system dynamics model to integrate societal-level influences via reciprocating functions. A Cognitive Network Model (CNM) is proposed as a method of quantitatively characterizing cognitive mechanisms at the intra-individual level. To capture the rich dynamics of interpersonal communication for the propagation of beliefs and attitudes, a Socio-Cognitive Network Model (SCNM) is presented. The SCNM uses socio-cognitive tie strength to regulate how agents influence--and are influenced by--one another's beliefs during social interactions. We then present experimental results which support the use of this network analytical approach, and we discuss its applicability towards characterizing and understanding human information processing

Privacy-Engineered Value Decomposition Networks for Cooperative Multi-Agent Reinforcement Learning

In cooperative multi-agent reinforcement learning (Co-MARL), a team of agents must jointly optimize the team's long-term rewards to learn a designated task. Optimizing rewards as a team often requires inter-agent communication and data sharing, leading to potential privacy implications. We assume privacy considerations prohibit the agents from sharing their environment interaction data. Accordingly, we propose Privacy-Engineered Value Decomposition Networks (PE-VDN), a Co-MARL algorithm that models multi-agent coordination while provably safeguarding the confidentiality of the agents' environment interaction data. We integrate three privacy-engineering techniques to redesign the data flows of the VDN algorithm, an existing Co-MARL algorithm that consolidates the agents' environment interaction data to train a central controller that models multi-agent coordination, and develop PE-VDN. In the first technique, we design a distributed computation scheme that eliminates Vanilla VDN's dependency on sharing environment interaction data. Then, we utilize a privacy-preserving multi-party computation protocol to guarantee that the data flows of the distributed computation scheme do not pose new privacy risks. Finally, we enforce differential privacy to preempt inference threats against the agents' training data, past environment interactions, when they take actions based on their neural network predictions. We implement PE-VDN in StarCraft Multi-Agent Competition (SMAC) and show that it achieves 80% of Vanilla VDN's win rate while maintaining differential privacy levels that provide meaningful privacy guarantees. The results demonstrate that PE-VDN can safeguard the confidentiality of agents' environment interaction data without sacrificing multi-agent coordination.Comment: Paper accepted at 62nd IEEE Conference on Decision and Contro

On the role of dialogue models in the age of large language models.

We argue that Machine learning, in particular the currently prevalent generation of Large Language Models (LLMs), can work constructively with existing normative models of dialogue as exemplified by dialogue games, specifically their computational applications within, for example, inter-agent communication and automated dialogue management. Furthermore we argue that this relationship is bi-directional, that some uses of dialogue games benefit from increased functionality due to the specific capabilities of LLMs, whilst LLMs benefit from externalised models of, variously, problematic, normative, or idealised behaviour. Machine Learning (ML) approaches, especially LLMs , appear to be making great advances against long-standing Artificial Intelligence challenges. In particular, LLMs are increasingly achieving successes in areas both adjacent to, and overlapping with, those of interest to the Computational Models of Natural Argument community. A prevalent opinion, not without some basis, within the ML research community is that many, if not all, AI challenges, will eventually be solved by ML models of increasing power and utility, negating the need for alternative or traditional approaches. An exemplar of this position, is the study of distinct models of dialogue for inter-agent communication when LLM based chatbots are increasingly able to surpass their performance in specific contexts. The trajectory of increased LLM capabilities suggests no reason that this trend will not continue, at least for some time. However, it is not the case that only the one, or the other approach, is necessary. Despite a tendency for LLMs to feature creep, and to appear to subsume additional areas of study, there are very good reasons to consider three modes of study of dialogue. Firstly, LLMs as their own individual field within ML, secondly, dialogue both in terms of actual human behaviour, which can exhibit wide quality standards, but also in terms of normative and idealised models, and thirdly, the fertile area in which the two overlap and can operate collaboratively. It is this third aspect with which this paper is concerned, for the first will occur anyway as researchers seek to map out the boundaries of what LLMs, as AI models, can actually achieve, and the second will continue, because the study of how people interact naturally through argument and dialogue will remain both fascinating and of objective value regardless of advances made in LLMs. However, where LLMs, Dialogue Models, and, for completion, people, come together, there is fertile ground for the development of principled models of interaction that are well-founded, well-regulated, and supportive of mixed-initiative interactions between humans and intelligent software agents