An Agent Approach to Spatial Information Grid Architecture Design

Spatial information grid (SIG) is a spatial information infrastructure that has the capability of providing services on-demand. SIG is a distributed network environment, which links spatial data resources, computing resources, storage resources, software, tools and users. SIG can integrate massive distributed heterogeneous spatial information resources, provides uniform management and process, and, furthermore, coordinate different resources to complete large-scale and complex spatial tasks and applications. In this paper, agent technology is adopted to construct a SIG framework, which contains three layers: users/applications layer, agent services layer and information layer. Different applications can get their spatial information via agent services, and agent services make the procedure of navigating and accessing spatial information transparent to users. Also, the implementation issues of the framework are discussed in detail, including Geo-Agents, an agent-based distributed GIS system, spatial information management, collaboration and parallel mechanism, load control strategy, and a sample

Unsupervised Emergence of Egocentric Spatial Structure from Sensorimotor Prediction

Despite its omnipresence in robotics application, the nature of spatial knowledgeand the mechanisms that underlie its emergence in autonomous agents are stillpoorly understood. Recent theoretical works suggest that the Euclidean structure ofspace induces invariants in an agent’s raw sensorimotor experience. We hypothesizethat capturing these invariants is beneficial for sensorimotor prediction and that,under certain exploratory conditions, a motor representation capturing the structureof the external space should emerge as a byproduct of learning to predict futuresensory experiences. We propose a simple sensorimotor predictive scheme, applyit to different agents and types of exploration, and evaluate the pertinence of thesehypotheses. We show that a naive agent can capture the topology and metricregularity of its sensor’s position in an egocentric spatial frame without any a prioriknowledge, nor extraneous supervision

Microeconomic Motives of Land Use Change in Coastal Zone Area: Agent Based Modelling Approach

Economic growth causes growing urbanization, extension of tourist sector, infrastructure and change of natural landscape. These processes of land use change attract even more attention if they take place in coastal zone area. In that case not only the efficient allocation and preservation of natural area, but also reduction of potential damage from flooding is important. Driven forces of land use at macro and micro levels should be taken into account. This paper presents an agent based model (ABM), which is designed to simulate land use change in coastal zone area based of human behaviour. The aim is to understand motives, types of connections and interactions between different actors and natural environment in order to get a feeling how different policy options and natural conditions might affect land use configuration. Microeconomic motives of land use decisions are in the focus of the research. Individual land use decisions are guided by economic and geomorphologic conditions, spatial planning and coastal protection policy. Each location choice is done according to a set of defined rules and land attributes. Space is represented as a grid of cells. Self-interested economic agents interact with each other trying to benefit from a certain type of land-use. We introduce the perception of risk of flooding in the model of land use as an innovative aspect of ABM simulations for water management problems. Based on decisions of spatially distributed individual economic agents operating in a policy framework, the model produces aggregated land-use patterns as an outcome. Understanding the factors that affect land use decisions will help policy makers design incentives to achieve policy objectives in coastal zone area. The proposed ABM will be applied to a study area in the province of North Holland in the Netherlands

A Mathematical Framework for Agent Based Models of Complex Biological Networks

Agent-based modeling and simulation is a useful method to study biological phenomena in a wide range of fields, from molecular biology to ecology. Since there is currently no agreed-upon standard way to specify such models it is not always easy to use published models. Also, since model descriptions are not usually given in mathematical terms, it is difficult to bring mathematical analysis tools to bear, so that models are typically studied through simulation. In order to address this issue, Grimm et al. proposed a protocol for model specification, the so-called ODD protocol, which provides a standard way to describe models. This paper proposes an addition to the ODD protocol which allows the description of an agent-based model as a dynamical system, which provides access to computational and theoretical tools for its analysis. The mathematical framework is that of algebraic models, that is, time-discrete dynamical systems with algebraic structure. It is shown by way of several examples how this mathematical specification can help with model analysis.Comment: To appear in Bulletin of Mathematical Biolog

CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms

How to optimally dispatch orders to vehicles and how to tradeoff between immediate and future returns are fundamental questions for a typical ride-hailing platform. We model ride-hailing as a large-scale parallel ranking problem and study the joint decision-making task of order dispatching and fleet management in online ride-hailing platforms. This task brings unique challenges in the following four aspects. First, to facilitate a huge number of vehicles to act and learn efficiently and robustly, we treat each region cell as an agent and build a multi-agent reinforcement learning framework. Second, to coordinate the agents from different regions to achieve long-term benefits, we leverage the geographical hierarchy of the region grids to perform hierarchical reinforcement learning. Third, to deal with the heterogeneous and variant action space for joint order dispatching and fleet management, we design the action as the ranking weight vector to rank and select the specific order or the fleet management destination in a unified formulation. Fourth, to achieve the multi-scale ride-hailing platform, we conduct the decision-making process in a hierarchical way where a multi-head attention mechanism is utilized to incorporate the impacts of neighbor agents and capture the key agent in each scale. The whole novel framework is named as CoRide. Extensive experiments based on multiple cities real-world data as well as analytic synthetic data demonstrate that CoRide provides superior performance in terms of platform revenue and user experience in the task of city-wide hybrid order dispatching and fleet management over strong baselines.Comment: CIKM 201

The Repast Simulation/Modelling System for Geospatial Simulation

The use of simulation/modelling systems can simplify the implementation of agent-based models. Repast is one of the few simulation/modelling software systems that supports the integration of geospatial data especially that of vector-based geometries. This paper provides details about Repast specifically an overview, including its different development languages available to develop agent-based models. Before describing Repast’s core functionality and how models can be developed within it, specific emphasis will be placed on its ability to represent dynamics and incorporate geographical information. Once these elements of the system have been covered, a diverse list of Agent-Based Modelling (ABM) applications using Repast will be presented with particular emphasis on spatial applications utilizing Repast, in particular, those that utilize geospatial data

Learning Representations in Model-Free Hierarchical Reinforcement Learning

Common approaches to Reinforcement Learning (RL) are seriously challenged by large-scale applications involving huge state spaces and sparse delayed reward feedback. Hierarchical Reinforcement Learning (HRL) methods attempt to address this scalability issue by learning action selection policies at multiple levels of temporal abstraction. Abstraction can be had by identifying a relatively small set of states that are likely to be useful as subgoals, in concert with the learning of corresponding skill policies to achieve those subgoals. Many approaches to subgoal discovery in HRL depend on the analysis of a model of the environment, but the need to learn such a model introduces its own problems of scale. Once subgoals are identified, skills may be learned through intrinsic motivation, introducing an internal reward signal marking subgoal attainment. In this paper, we present a novel model-free method for subgoal discovery using incremental unsupervised learning over a small memory of the most recent experiences (trajectories) of the agent. When combined with an intrinsic motivation learning mechanism, this method learns both subgoals and skills, based on experiences in the environment. Thus, we offer an original approach to HRL that does not require the acquisition of a model of the environment, suitable for large-scale applications. We demonstrate the efficiency of our method on two RL problems with sparse delayed feedback: a variant of the rooms environment and the first screen of the ATARI 2600 Montezuma's Revenge game