Coalitional Bargaining via Reinforcement Learning: An Application to Collaborative Vehicle Routing

Collaborative Vehicle Routing is where delivery companies cooperate by sharing their delivery information and performing delivery requests on behalf of each other. This achieves economies of scale and thus reduces cost, greenhouse gas emissions, and road congestion. But which company should partner with whom, and how much should each company be compensated? Traditional game theoretic solution concepts, such as the Shapley value or nucleolus, are difficult to calculate for the real-world problem of Collaborative Vehicle Routing due to the characteristic function scaling exponentially with the number of agents. This would require solving the Vehicle Routing Problem (an NP-Hard problem) an exponential number of times. We therefore propose to model this problem as a coalitional bargaining game where - crucially - agents are not given access to the characteristic function. Instead, we implicitly reason about the characteristic function, and thus eliminate the need to evaluate the VRP an exponential number of times - we only need to evaluate it once. Our contribution is that our decentralised approach is both scalable and considers the self-interested nature of companies. The agents learn using a modified Independent Proximal Policy Optimisation. Our RL agents outperform a strong heuristic bot. The agents correctly identify the optimal coalitions 79% of the time with an average optimality gap of 4.2% and reduction in run-time of 62%.Comment: Accepted to NeurIPS 2021 Workshop on Cooperative A

Fair collaborative vehicle routing: A deep multi-agent reinforcement learning approach

Collaborative vehicle routing occurs when carriers collaborate through sharing their transportation requests and performing transportation requests on behalf of each other. This achieves economies of scale, thus reducing cost, greenhouse gas emissions and road congestion. But which carrier should partner with whom, and how much should each carrier be compensated? Traditional game theoretic solution concepts are expensive to calculate as the characteristic function scales exponentially with the number of agents. This would require solving the vehicle routing problem (NP-hard) an exponential number of times. We therefore propose to model this problem as a coalitional bargaining game solved using deep multi-agent reinforcement learning, where - crucially - agents are not given access to the characteristic function. Instead, we implicitly reason about the characteristic function; thus, when deployed in production, we only need to evaluate the expensive post-collaboration vehicle routing problem once. Our contribution is that we are the first to consider both the route allocation problem and gain sharing problem simultaneously - without access to the expensive characteristic function. Through decentralised machine learning, our agents bargain with each other and agree to outcomes that correlate well with the Shapley value - a fair profit allocation mechanism. Importantly, we are able to achieve a reduction in run-time of 88%.Comment: Final, published version can be found here: https://www.sciencedirect.com/science/article/pii/S0968090X2300366

AgentChat: Multi-Agent Collaborative Logistics for Carbon Reduction

Heavy Good Vehicles (HGVs) are the second largest source of greenhouse gas emissions in transportation, after cars and taxis. However, HGVs are inefficiently utilised, with more than one-third of their weight capacity not being used during travel. We, thus, in this paper address collaborative logistics, an effective pathway to enhance HGVs' utilisation and reduce carbon emissions. We investigate a multi-agent system approach to facilitate collaborative logistics, particularly carrier collaboration. We propose a simple yet effective multi-agent collaborative logistics (MACL) framework, representing key stakeholders as intelligent agents. Furthermore, we utilise the MACL framework in conjunction with a proposed system architecture to create an integrated collaborative logistics testbed. This testbed, consisting of a physical system and its digital replica, is a tailored cyber-physical system or digital twin for collaborative logistics. Through a demonstration, we show the utility of the testbed for studying collaborative logistics.Comment: This paper includes 12 pages, 14 figures, and has been submitted to IEEE for possible publicatio

Machine Learning in Nuclear Physics

Advances in machine learning methods provide tools that have broad applicability in scientific research. These techniques are being applied across the diversity of nuclear physics research topics, leading to advances that will facilitate scientific discoveries and societal applications. This Review gives a snapshot of nuclear physics research which has been transformed by machine learning techniques.Comment: Comments are welcom

Characterization of fluorescent chlorophyll charge-transfer states as intermediates in the excited state quenching of light-harvesting complex II

Light-harvesting complex II (LHCII) is the major antenna complex in higher plants and green algae. It has been suggested that a major part of the excited state energy dissipation in the so-called "non-photochemical quenching" (NPQ) is located in this antenna complex. We have performed an ultrafast kinetics study of the low-energy fluorescent states related to quenching in LHCII in both aggregated and the crystalline form. In both sample types the chlorophyll (Chl) excited states of LHCII are strongly quenched in a similar fashion. Quenching is accompanied by the appearance of new far-red (FR) fluorescence bands from energetically low-lying Chl excited states. The kinetics of quenching, its temperature dependence down to 4 K, and the properties of the FR-emitting states are very similar both in LHCII aggregates and in the crystal. No such FR-emitting states are found in unquenched trimeric LHCII. We conclude that these states represent weakly emitting Chl-Chl charge-transfer (CT) states, whose formation is part of the quenching process. Quantum chemical calculations of the lowest energy exciton and CT states, explicitly including the coupling to the specific protein environment, provide detailed insight into the chemical nature of the CT states and the mechanism of CT quenching. The experimental data combined with the results of the calculations strongly suggest that the quenching mechanism consists of a sequence of two proton-coupled electron transfer steps involving the three quenching center Chls 610/611/612. The FR-emitting CT states are reaction intermediates in this sequence. The polarity-controlled internal reprotonation of the E175/K179 aa pair is suggested as the switch controlling quenching. A unified model is proposed that is able to explain all known conditions of quenching or non-quenching of LHCII, depending on the environment without invoking any major conformational changes of the protein

Recombinant LCMV Vectors Induce Protective Immunity following Homologous and Heterologous Vaccinations

Successful vaccination against cancer and infectious diseases relies on the induction of adaptive immune responses that induce high-titer antibodies or potent cytoxic T cell responses. In contrast to humoral vaccines, the amplification of cellular immune responses is often hampered by anti-vector immunity that either pre-exists or develops after repeated homologous vaccination. Replication-defective lymphocytic choriomeningitis virus (LCMV) vectors represent a novel generation of vaccination vectors that induce potent immune responses while escaping recognition by neutralizing antibodies. Here, we characterize the CD8 T cell immune response induced by replication-defective recombinant LCMV (rLCMV) vectors with regard to expansion kinetics, trafficking, phenotype, and function and we perform head-to-head comparisons of the novel rLCMV vectors with established vectors derived from adenovirus, vaccinia virus, or Listeria monocytogenes. Our results demonstrate that replication-deficient rLCMV vectors are safe and ideally suited for both homologous and heterologous vaccination regimens to achieve optimal amplification of CD8 T cell immune responses in vivo

The ATLAS Intensity upgrade: Project overview and online operating experience

ATLAS, the world's first accelerator to use RF superconductivity for ion acceleration, has undergone a major facility upgrade with the goals of significantly increased stable-beam current for experiments and improved transmission for all beams. The dominant components of the upgrade are a) new CW-RFQ to replace the first three low β resonators, b) a new cryostat of seven β=0.077 quarter-wave resonators demonstrating world-record accelerating fields, c) an improved cryogenics system, and d) the retirement of the original tandem injector. This latest upgrade followed closely on the earlier development of a cryostat of β=0.144 quarter-wave resonators. This reconfigured ATLAS system has been in operation for over one year. This paper will discuss the on-line performance achieved for the redesigned system, plans for further improvement, and long term facility plans for new performance capabilities. This work was supported by the U.S. Department of Energy, Office of Nuclear Physics, under Contract No. DE-AC02-06CH11357. This research used resources of ANL's ATLAS facility, which is a DOE Office of Science User Facility

Colloquium: Machine learning in nuclear physics

Advances in machine learning methods provide tools that have broad applicability in scientific research. These techniques are being applied across the diversity of nuclear physics research topics, leading to advances that will facilitate scientific discoveries and societal applications. This Colloquium provides a snapshot of nuclear physics research, which has been transformed by machine learning techniques