Fuzzy Q-Learning Based Multi-Agent System for Intelligent Traffic Control by a Game Theory Approach

This paper introduces a multi-agent approach to adjust traffic lights based on traffic situation in order to reduce average delay time. In the traffic model, lights of each intersection are controlled by an autonomous agent. Since decision of each agent affects neighbor agents, this approach creates a classical non-stationary environment. Thus, each agent not only needs to learn from the past experience but also has to consider decision of neighbors to overcome dynamic changes of the traffic network. Fuzzy Q-learning and Game theory are employed to make policy based on previous experiences and decision of neighbor agents. Simulation results illustrate the advantage of the proposed method over fixed time, fuzzy, Q-learning and fuzzy Q-learning control methods.Comment: 10 pages, 10 figure

Economics 2.0: The Natural Step towards A Self-Regulating, Participatory Market Society

Despite all our great advances in science, technology and financial innovations, many societies today are struggling with a financial, economic and public spending crisis, over-regulation, and mass unemployment, as well as lack of sustainability and innovation. Can we still rely on conventional economic thinking or do we need a new approach? I argue that, as the complexity of socio-economic systems increases, networked decision-making and bottom-up self-regulation will be more and more important features. It will be explained why, besides the "homo economicus" with strictly self-regarding preferences, natural selection has also created a "homo socialis" with other-regarding preferences. While the "homo economicus" optimizes the own prospects in separation, the decisions of the "homo socialis" are self-determined, but interconnected, a fact that may be characterized by the term "networked minds". Notably, the "homo socialis" manages to earn higher payoffs than the "homo economicus". I show that the "homo economicus" and the "homo socialis" imply a different kind of dynamics and distinct aggregate outcomes. Therefore, next to the traditional economics for the "homo economicus" ("economics 1.0"), a complementary theory must be developed for the "homo socialis". This economic theory might be called "economics 2.0" or "socionomics". The names are justified, because the Web 2.0 is currently promoting a transition to a new market organization, which benefits from social media platforms and could be characterized as "participatory market society". To thrive, the "homo socialis" requires suitable institutional settings such a particular kinds of reputation systems, which will be sketched in this paper. I also propose a new kind of money, so-called "qualified money", which may overcome some of the problems of our current financial system.Comment: For related work see http://www.soms.ethz.ch and http://www.futurict.e

Self-Organization in Traffic Lights: Evolution of Signal Control with Advances in Sensors and Communications

Traffic signals are ubiquitous devices that first appeared in 1868. Recent advances in information and communications technology (ICT) have led to unprecedented improvements in such areas as mobile handheld devices (i.e., smartphones), the electric power industry (i.e., smart grids), transportation infrastructure, and vehicle area networks. Given the trend towards interconnectivity, it is only a matter of time before vehicles communicate with one another and with infrastructure. In fact, several pilots of such vehicle-to-vehicle and vehicle-to-infrastructure (e.g. traffic lights and parking spaces) communication systems are already operational. This survey of autonomous and self-organized traffic signaling control has been undertaken with these potential developments in mind. Our research results indicate that, while many sophisticated techniques have attempted to improve the scheduling of traffic signal control, either real-time sensing of traffic patterns or a priori knowledge of traffic flow is required to optimize traffic. Once this is achieved, communication between traffic signals will serve to vastly improve overall traffic efficiency

When slower is faster

The slower is faster (SIF) effect occurs when a system performs worse as its components try to do better. Thus, a moderate individual efficiency actually leads to a better systemic performance. The SIF effect takes place in a variety of phenomena. We review studies and examples of the SIF effect in pedestrian dynamics, vehicle traffic, traffic light control, logistics, public transport, social dynamics, ecological systems, and adaptation. Drawing on these examples, we generalize common features of the SIF effect and suggest possible future lines of research

Unconventional Arterial Intersection Designs under Connected and Automated Vehicle Environment: A Survey

Signalized intersections are major sources of traffic delay and collision within the modern transportation system. Conventional signal optimization has revealed its limitation in improving the mobility and safety of an intersection. Unconventional arterial intersection designs (UAIDs) are able to improve the performance of an intersection by reducing phases of a signal cycle. Furthermore, they can fundamentally alter the number and the nature of the conflicting points. However, the driver's confusion, as a result of the unconventional geometric designs, remains one of the major barriers for the widespread adoption of UAIDs. Connected and Automated Vehicle (CAV) technology has the potential to overcome this barrier by eliminating the driver's confusion of a UAID. Therefore, UAIDs can play a significant role in transportation networks in the near future. In this paper, we surveyed UAID studies and implementations. In addition, we present an overview of intersection control schemes with the emergence of CAV and highlight the opportunity rises for UAID with the CAV technology. It is believed that the benefits gained from deploying UAIDs in conjunction with CAV are significant during the initial rollout of CAV under low market penetration

Distributed Constraint Optimization Problems and Applications: A Survey

The field of Multi-Agent System (MAS) is an active area of research within Artificial Intelligence, with an increasingly important impact in industrial and other real-world applications. Within a MAS, autonomous agents interact to pursue personal interests and/or to achieve common objectives. Distributed Constraint Optimization Problems (DCOPs) have emerged as one of the prominent agent architectures to govern the agents' autonomous behavior, where both algorithms and communication models are driven by the structure of the specific problem. During the last decade, several extensions to the DCOP model have enabled them to support MAS in complex, real-time, and uncertain environments. This survey aims at providing an overview of the DCOP model, giving a classification of its multiple extensions and addressing both resolution methods and applications that find a natural mapping within each class of DCOPs. The proposed classification suggests several future perspectives for DCOP extensions, and identifies challenges in the design of efficient resolution algorithms, possibly through the adaptation of strategies from different areas

A Survey on Traffic Signal Control Methods

Traffic signal control is an important and challenging real-world problem, which aims to minimize the travel time of vehicles by coordinating their movements at the road intersections. Current traffic signal control systems in use still rely heavily on oversimplified information and rule-based methods, although we now have richer data, more computing power and advanced methods to drive the development of intelligent transportation. With the growing interest in intelligent transportation using machine learning methods like reinforcement learning, this survey covers the widely acknowledged transportation approaches and a comprehensive list of recent literature on reinforcement for traffic signal control. We hope this survey can foster interdisciplinary research on this important topic.Comment: 32 page

A General Methodology for Designing Self-Organizing Systems

Our technologies complexify our environments. Thus, new technologies need to deal with more and more complexity. Several efforts have been made to deal with this complexity using the concept of self-organization. However, in order to promote its use and understanding, we must first have a pragmatic understanding of complexity and self-organization. This paper presents a conceptual framework for speaking about self-organizing systems. The aim is to provide a methodology useful for designing and controlling systems developed to solve complex problems. First, practical notions of complexity and self-organization are given. Then, starting from the agent metaphor, a conceptual framework is presented. This provides formal ways of speaking about "satisfaction" of elements and systems. The main premise of the methodology claims that reducing the "friction" or "interference" of interactions between elements of a system will result in a higher "satisfaction" of the system, i.e. better performance. The methodology discusses different ways in which this can be achieved. A case study on self-organizing traffic lights illustrates the ideas presented in the paper.Comment: Related work at http://homepages.vub.ac.be/~cgershen/sos

An Introduction to Collective Intelligence

This paper surveys the emerging science of how to design a ``COllective INtelligence'' (COIN). A COIN is a large multi-agent system where: (i) There is little to no centralized communication or control; and (ii) There is a provided world utility function that rates the possible histories of the full system. In particular, we are interested in COINs in which each agent runs a reinforcement learning (RL) algorithm. Rather than use a conventional modeling approach (e.g., model the system dynamics, and hand-tune agents to cooperate), we aim to solve the COIN design problem implicitly, via the ``adaptive'' character of the RL algorithms of each of the agents. This approach introduces an entirely new, profound design problem: Assuming the RL algorithms are able to achieve high rewards, what reward functions for the individual agents will, when pursued by those agents, result in high world utility? In other words, what reward functions will best ensure that we do not have phenomena like the tragedy of the commons, Braess's paradox, or the liquidity trap? Although still very young, research specifically concentrating on the COIN design problem has already resulted in successes in artificial domains, in particular in packet-routing, the leader-follower problem, and in variants of Arthur's El Farol bar problem. It is expected that as it matures and draws upon other disciplines related to COINs, this research will greatly expand the range of tasks addressable by human engineers. Moreover, in addition to drawing on them, such a fully developed scie nce of COIN design may provide much insight into other already established scientific fields, such as economics, game theory, and population biology.Comment: 88 pages, 10 figs, 297 ref

Control of Connected and Automated Vehicles: State of the Art and Future Challenges

Autonomous driving technology pledges safety, convenience, and energy efficiency. Challenges include the unknown intentions of other road users: communication between vehicles and with the road infrastructure is a possible approach to enhance awareness and enable cooperation. Connected and automated vehicles (CAVs) have the potential to disrupt mobility, extending what is possible with driving automation and connectivity alone. Applications include real-time control and planning with increased awareness, routing with micro-scale traffic information, coordinated platooning using traffic signals information, eco-mobility on demand with guaranteed parking. This paper introduces a control and planning architecture for CAVs, and surveys the state of the art on each functional block therein; the main focus is on techniques to improve energy efficiency. We provide an overview of existing algorithms and their mutual interactions, we present promising optimization-based approaches to CAVs control and identify future challenges