Learning in Real-Time Search: A Unifying Framework

Real-time search methods are suited for tasks in which the agent is interacting with an initially unknown environment in real time. In such simultaneous planning and learning problems, the agent has to select its actions in a limited amount of time, while sensing only a local part of the environment centered at the agents current location. Real-time heuristic search agents select actions using a limited lookahead search and evaluating the frontier states with a heuristic function. Over repeated experiences, they refine heuristic values of states to avoid infinite loops and to converge to better solutions. The wide spread of such settings in autonomous software and hardware agents has led to an explosion of real-time search algorithms over the last two decades. Not only is a potential user confronted with a hodgepodge of algorithms, but he also faces the choice of control parameters they use. In this paper we address both problems. The first contribution is an introduction of a simple three-parameter framework (named LRTS) which extracts the core ideas behind many existing algorithms. We then prove that LRTA*, epsilon-LRTA*, SLA*, and gamma-Trap algorithms are special cases of our framework. Thus, they are unified and extended with additional features. Second, we prove completeness and convergence of any algorithm covered by the LRTS framework. Third, we prove several upper-bounds relating the control parameters and solution quality. Finally, we analyze the influence of the three control parameters empirically in the realistic scalable domains of real-time navigation on initially unknown maps from a commercial role-playing game as well as routing in ad hoc sensor networks

A Smart Contract-based BPMN Choreography Execution for Management of Construction Processes

Construction management can be grouped into two different levels: strategic early planning, that provides the baseline for project monitoring, and short time initiatives, based on objectives and self-organization from actors who are involved in on-site processes. The latter can be considered as a complex system management issue since it presents emergent behaviors thus it can not be handled in a traditional way. The passage from project scheduling to on site operations management requires a change of perspective. On site short time planning is a process of forecasting future outcomes therefore it deals with uncertainty and indeterminacy. At present this is managed through the representation of many separate orchestrations and this does not allow to eliminate the inefficiencies that arise at the level of synchronization of the individual tasks performed by organizations with contractually separate management. Efficiency in construction management implies to take into consideration choreographies because they better reflect synchronization of different organizations management processes. On the other hand, information processed as a trigger for distributed activities on different management does not guarantee process traceability while smart contracts linked to single task execution assure both promptness and irreversible tracking at single task level. The actual execution of the processes depends both on what happens and on the information that flows between the subjects who actually carry out processes asynchronous to each other, so the only possibility to synchronize them is information. This research aims to describe a framework for applying BPMN choreographies to construction site processes in order to better modeling processes for smart contracts application. The choice of applying BPMN instead of CPM lays in the fact that it allows to model the information flows as well as the preparatory aspects and in addition it allows to represent decision-making moments. Every single activity in the baseline can be modelled as a choreography at a lower level. On the other hand, process performance monitoring can be performed thanks to blockchain tasks notarization. Concrete casting quality assessment process has been chosen as use case. BPMN choreography of this process has been modelled and blockchain application for tasks and information notarization has been development and tested on a construction site

Temoral Difference Learning in Complex Domains

Submitted to the University of London for the Degree of Doctor of Philosophy in Computer Scienc