Hierarchical self-organization of non-cooperating individuals

Hierarchy is one of the most conspicuous features of numerous natural, technological and social systems. The underlying structures are typically complex and their most relevant organizational principle is the ordering of the ties among the units they are made of according to a network displaying hierarchical features. In spite of the abundant presence of hierarchy no quantitative theoretical interpretation of the origins of a multi-level, knowledge-based social network exists. Here we introduce an approach which is capable of reproducing the emergence of a multi-levelled network structure based on the plausible assumption that the individuals (representing the nodes of the network) can make the right estimate about the state of their changing environment to a varying degree. Our model accounts for a fundamental feature of knowledge-based organizations: the less capable individuals tend to follow those who are better at solving the problems they all face. We find that relatively simple rules lead to hierarchical self-organization and the specific structures we obtain possess the two, perhaps most important features of complex systems: a simultaneous presence of adaptability and stability. In addition, the performance (success score) of the emerging networks is significantly higher than the average expected score of the individuals without letting them copy the decisions of the others. The results of our calculations are in agreement with a related experiment and can be useful from the point of designing the optimal conditions for constructing a given complex social structure as well as understanding the hierarchical organization of such biological structures of major importance as the regulatory pathways or the dynamics of neural networks.Comment: Supplementary videos are to be found at http://hal.elte.hu/~nepusz/research/supplementary/hierarchy

Towards Informed Exploration for Deep Reinforcement Learning

In this thesis, we discuss various techniques for improving exploration for deep reinforcement learning. We begin with a brief review of reinforcement learning (RL) and the fundamental v.s. exploitation trade-off. Then we review how deep RL has improved upon classical and summarize six categories of the latest exploration methods for deep RL, in the order increasing usage of prior information. We then explore representative works in three categories discuss their strengths and weaknesses. The first category, represented by Soft Q-learning, uses regularization to encourage exploration. The second category, represented by count-based via hashing, maps states to hash codes for counting and assigns higher exploration to less-encountered states. The third category utilizes hierarchy and is represented by modular architecture for RL agents to play StarCraft II. Finally, we conclude that exploration by prior knowledge is a promising research direction and suggest topics of potentially impact

Smart Technologies for Environmental Safety and Knowledge Enhancement in Intermodal Transport

International concerns about security in transport systems are leading to a new international regulation in this field. This introduces new requirements for operators and authorities as well as it opens new challenges, in particular when referred to seaports and maritime transport in the Mediterranean area, where many seaport terminals and infrastructures are affected by a noteworthy technological divide from North European contexts. In such contexts, the adoption of the new regulations can represent the right chance for upgrading the local operative standards, increasing latu sensu the quality of maritime transport performances, while conferring a greater level to security and safety checks. This paper explores the chances for increasing the level of Mediterranean seaport competitiveness allowed by technological innovations in transport systems, both in operations and organization of these infrastructures. The aim of the work is to study the effects of the adoption of technological solutions such as wireless communications and radiofrequency identification on the competitiveness of Mediterranean seaport infrastructures. Technological solutions designed to identify good items help operators in organizing activities in terminals and make maritime transport faster in delivering goods, by cutting the handling time and costs in seaport terminals. Seaports that adopt this kind of technologies, and the surrounding economic areas connected to seaports, have a greater attractiveness on shipping companies and operators, since they allow faster handling activities and easier checks on goods. Besides, the analysis of direct and indirect effects of the use of such technologies specifically focuses on the contribution that the use of these solutions gives in ensuring higher security levels, by increasing the level of information and knowledge associated to goods. The different types of security provided (e.g. for people, environment and goods) and the extreme flexibility of the technologies involved give the overall worth of the challenge. It seems to be a great chance of growth for the Mediterranean area, more than a mere compliance to the international security regulations.

CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms

How to optimally dispatch orders to vehicles and how to tradeoff between immediate and future returns are fundamental questions for a typical ride-hailing platform. We model ride-hailing as a large-scale parallel ranking problem and study the joint decision-making task of order dispatching and fleet management in online ride-hailing platforms. This task brings unique challenges in the following four aspects. First, to facilitate a huge number of vehicles to act and learn efficiently and robustly, we treat each region cell as an agent and build a multi-agent reinforcement learning framework. Second, to coordinate the agents from different regions to achieve long-term benefits, we leverage the geographical hierarchy of the region grids to perform hierarchical reinforcement learning. Third, to deal with the heterogeneous and variant action space for joint order dispatching and fleet management, we design the action as the ranking weight vector to rank and select the specific order or the fleet management destination in a unified formulation. Fourth, to achieve the multi-scale ride-hailing platform, we conduct the decision-making process in a hierarchical way where a multi-head attention mechanism is utilized to incorporate the impacts of neighbor agents and capture the key agent in each scale. The whole novel framework is named as CoRide. Extensive experiments based on multiple cities real-world data as well as analytic synthetic data demonstrate that CoRide provides superior performance in terms of platform revenue and user experience in the task of city-wide hybrid order dispatching and fleet management over strong baselines.Comment: CIKM 201

Extending TRANSIMS Technology to an Integrated Multilevel Representation

The TRANSIMS system developed at Los Alamos in the USA over the past decade is a world leader in providing an integrated land-use transportation dynamical model for large areas with a million or more inhabitants. TRANSIMS uses standard survey data to create synthetic micropopulations, including family structure, to simulate trip making and emergent traffic dynamics. We propose to extend TRANSIMS by adapting it to a new multi-level representation, allowing dynamics to be algebraically integrated at the micro-, meso- and macro-levels. The new representation builds a lattice hierarchy in a way that integrates non-partitional hierarchies of links and routes based on the usual hierarchy of geographical zones, e.g. neighbourhoods, districts, cities, counties and countries. Applying the representation to a big city starts by defining sets of zones at different levels. At the first level, N, is the street. This can be subdivided to building plots at level N-1, buildings at level N-2, and even rooms at level N-3. At level N+1 are the neighbourhoods, at level N+2 is the set of district zones (each of them containing the different neighbourhoods in the previous level), and at the top level N+3 (in this case), is just one zone, the city itself. If a larger study area is to be considered, we would have a whole set of N+3 zones defining N+4-level areas, and so on, extending to the level of counties, countries or even continents. This paper will explain the fundamentals of TRANSIMS technology and compare it to other systems. We will show how TRANSIMS and the new multi-level representation can be brought together to give new insights into the macro-dynamics of very large road systems such as London, England and even the whole of Europe