A Rewriting-based, Parameterized Exploration Scheme for the Dynamic Analysis of Complex Software Systems

Los sistemas software actuales son artefactos complejos cuyo comportamiento es a menudo extremadamente difícil de entender. Este hecho ha llevado al desarrollo de metodologías formales muy sofisticadas para el análisis, comprensión y depuración de programas. El análisis de trazas de ejecución consiste en la búsqueda dinámica de contenidos específicos dentro de las trazas de ejecución de un cierto programa. La búsqueda puede llevarse a cabo hacia adelante o hacia atrás. Si bien el análisis hacia adelante se traduce en una forma de análisis de impacto que identifica el alcance y las posibles consecuencias de los cambios en la entrada del programa, el análisis hacia atrás permite llevar a cabo un rastreo de la procedencia; es decir, muestra como (partes de) la salida del programa depende de (partes de) su entrada y ayuda a estimar qué dato de la entrada es necesario modificar para llevar a cabo un cambio en el resultado. En esta tesis se investiga una serie de metodologías de análisis de trazas que son especialmente adecuadas para el análisis de trazas de ejecución largas y complejas en la lógica de reescritura, que es un marco lógico y semántico especialmente adecuado para la formalización de sistemas altamente concurrentes. La primera parte de la tesis se centra en desarrollar una técnica de análisis de trazas hacia atrás que alcanza enormes reducciones en el tamaño de la traza. Esta metodología se basa en la fragmentación incremental y favorece un mejor análisis y depuración ya que la mayoría de las inspecciones, tediosas e irrelevantes, que se realizan rutinariamente en el diagnostico y la localización de errores se pueden eliminar de forma automática. Esta técnica se ilustra por medio de varios ejemplos que ejecutamos mediante el sistema iJulienne, una herramienta interactiva de fragmentación que hemos desarrollado y que implementa la técnica de análisis de trazas hacia atrás. En la segunda parte de la tesis se formaliza un sistema paramétrico, flexible y dinámico, para la exploración de computaciones en la lógica de reescritura. El esquema implementa un algoritmo de animación gen érico que permite la ejecución indeterminista de una teoría de reescritura condicional dada y que puede ser objeto de seguimiento mediante el uso de diferentes modalidades, incluyendo una ejecución gradual paso a paso y una fragmentación automática hacia adelante y/o hacia atrás, lo que reduce drásticamente el tamaño y la complejidad de las trazas bajo inspección y permite a los usuarios evaluar de forma aislada los efectos de una declaración o instrucción dada, el seguimiento de los efectos del cambio de la entrada, y obtener información sobre el comportamiento del programa (o mala conducta del mismo). Por otra parte, la fragmentación de la traza de ejecución puede identificar nuevas oportunidades de optimización del programa. Con esta metodología, un analista puede navegar, fragmentar, filtrar o buscar en la traza durante la ejecución del programa. El marco de análisis de trazas gen érico se ha implementado en el sistema Anima y describimos una profunda evaluación experimental de este que demuestra la utilidad del enfoque propuesto.Frechina Navarro, F. (2014). A Rewriting-based, Parameterized Exploration Scheme for the Dynamic Analysis of Complex Software Systems [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/44234TESI

Multi-target Extension for Beacon Foraging Methods

Robotic foraging is a complex problem that encompasses both the problem of exploring an area and retrieval of targets. To solve this, biologically inspired algorithms have been proposed that handle the scenario when only one target exists, but requires extension to multiple targets. In both scenarios there exists problems of robot allocation and congestion, and we analyze ways of optimizing both allocation and minimizing congestion for our algorithms. We demonstrate the results through parameterized metrics and compare the improvements in each scenario. This is a technical report on artificial intelligence, 2017, written by Sanford, Jiao, and Oh

Inspecting rewriting logic computations (in a parametric and stepwise way)

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-54624-2_12Trace inspection is concerned with techniques that allow the trace content to be searched for specific components. This paper presents a rich and highly dynamic, parameterized technique for the trace inspection of Rewriting Logic theories that allows the non-deterministic execution of a given unconditional rewrite theory to be followed up in different ways. Using this technique, an analyst can browse, slice, filter, or search the traces as they come to life during the program execution. Starting from a selected state in the computation tree, the navigation of the trace is driven by a user-defined, inspection criterion that specifies the required exploration mode. By selecting different inspection criteria, one can automatically derive a family of practical algorithms such as program steppers and more sophisticated dynamic trace slicers that facilitate the dynamic detection of control and data dependencies across the computation tree. Our methodology, which is implemented in the Anima graphical tool, allows users to capture the impact of a given criterion thereby facilitating the detection of improper program behaviors.This work has been partially supported by the EU (FEDER), the Spanish MEC project ref. TIN2010-21062-C02-02, the Spanish MICINN complementary action ref. TIN2009-07495-E, and by Generalitat Valenciana ref. PROMETEO2011/052. This work was carried out during the tenure of D. Ballis’ ERCIM “Alain Bensoussan ”Postdoctoral Fellowship. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n. 246016. F. Frechina was supported by FPU-ME grant AP2010-5681.Alpuente Frasnedo, M.; Ballis, D.; Frechina, F.; Sapiña Sanchis, J. (2014). Inspecting rewriting logic computations (in a parametric and stepwise way). En Specification, algebra, and software: essays dedicated to Kokichi Futatsugi. Springer Verlag (Germany). 229-255. https://doi.org/10.1007/978-3-642-54624-2_12S229255Alpuente, M., Ballis, D., Baggi, M., Falaschi, M.: A Fold/Unfold Transformation Framework for Rewrite Theories extended to CCT. In: Proc. PEPM 2010, pp. 43–52. ACM (2010)Alpuente, M., Ballis, D., Espert, J., Romero, D.: Model-checking Web Applications with Web-TLR. In: Bouajjani, A., Chin, W.-N. (eds.) ATVA 2010. LNCS, vol. 6252, pp. 341–346. Springer, Heidelberg (2010)Alpuente, M., Ballis, D., Espert, J., Romero, D.: Backward Trace Slicing for Rewriting Logic Theories. In: Bjørner, N., Sofronie-Stokkermans, V. (eds.) CADE 2011. LNCS, vol. 6803, pp. 34–48. Springer, Heidelberg (2011)Alpuente, M., Ballis, D., Frechina, F., Sapiña, J.: Slicing-Based Trace Analysis of Rewriting Logic Specifications with iJulienne. In: Felleisen, M., Gardner, P. (eds.) ESOP 2013. LNCS, vol. 7792, pp. 121–124. Springer, Heidelberg (2013)Alpuente, M., Ballis, D., Frechina, F., Romero, D.: Using Conditional Trace Slicing for improving Maude programs. Science of Computer Programming (2013) (to appear)Alpuente, M., Ballis, D., Romero, D.: A Rewriting Logic Approach to the Formal Specification and Verification of Web applications. Science of Computer Programming (2013) (to appear)Baggi, M., Ballis, D., Falaschi, M.: Quantitative Pathway Logic for Computational Biology. In: Degano, P., Gorrieri, R. (eds.) CMSB 2009. LNCS, vol. 5688, pp. 68–82. Springer, Heidelberg (2009)Bruni, R., Meseguer, J.: Semantic Foundations for Generalized Rewrite Theories. Theoretical Computer Science 360(1-3), 386–414 (2006)Clavel, M., Durán, F., Eker, S., Lincoln, P., Martí-Oliet, N., Meseguer, J., Talcott, C.: Maude Manual (Version 2.6). Technical report, SRI Int’l Computer Science Laboratory (2011), http://maude.cs.uiuc.edu/maude2-manual/Clements, J., Flatt, M., Felleisen, M.: Modeling an Algebraic Stepper. In: Sands, D. (ed.) ESOP 2001. LNCS, vol. 2028, pp. 320–334. Springer, Heidelberg (2001)Durán, F., Meseguer, J.: A Maude Coherence Checker Tool for Conditional Order-Sorted Rewrite Theories. In: Ölveczky, P.C. (ed.) WRLA 2010. LNCS, vol. 6381, pp. 86–103. Springer, Heidelberg (2010)Eker, S.: Associative-Commutative Matching via Bipartite Graph Matching. The Computer Journal 38(5), 381–399 (1995)Eker, S.: Associative-Commutative Rewriting on Large Terms. In: Nieuwenhuis, R. (ed.) RTA 2003. LNCS, vol. 2706, pp. 14–29. Springer, Heidelberg (2003)Klop, J.W.: Term Rewriting Systems. In: Abramsky, S., Gabbay, D., Maibaum, T. (eds.) Handbook of Logic in Computer Science, vol. I, pp. 1–112. Oxford University Press (1992)Martí-Oliet, N., Meseguer, J.: Rewriting Logic: Roadmap and Bibliography. Theoretical Computer Science 285(2), 121–154 (2002)Meseguer, J.: Conditional Rewriting Logic as a Unified Model of Concurrency. Theoretical Computer Science 96(1), 73–155 (1992)Meseguer, J.: The Temporal Logic of Rewriting: A Gentle Introduction. In: Degano, P., De Nicola, R., Meseguer, J. (eds.) Montanari Festschrift. LNCS, vol. 5065, pp. 354–382. Springer, Heidelberg (2008)Plotkin, G.D.: The Origins of Structural Operational Semantics. The Journal of Logic and Algebraic Programming 60-61(1), 3–15 (2004)Riesco, A., Verdejo, A., Caballero, R., Martí-Oliet, N.: Declarative Debugging of Rewriting Logic Specifications. In: Corradini, A., Montanari, U. (eds.) WADT 2008. LNCS, vol. 5486, pp. 308–325. Springer, Heidelberg (2009)Riesco, A., Verdejo, A., Martí-Oliet, N.: Declarative Debugging of Missing Answers for Maude. In: Proc. RTA 2010. LIPIcs, vol. 6, pp. 277–294 (2010)TeReSe. Term Rewriting Systems. Cambridge University Press (2003

Dispatching AGVs with Battery Constraints using Deep Reinforcement Learning

This paper considers the problem of real-time dispatching of a fleet of automated guided vehicles (AGVs) with battery constraints. AGVs must be immediately assigned to transport requests, which arrive randomly. In addition, the AGVs must be repositioned and recharged, awaiting future transport requests. Each transport request has a soft time window with late delivery incurring a tardiness cost. This research aims to minimize the total costs, consisting of tardiness costs of transport requests and travel costs of AGVs. We extend the existing literature by making a distinction between parking and charging nodes, where AGVs wait idle for incoming transporting requests and satisfy their charging needs, respectively. Also, we formulate this online decision-making problem as a Markov decision process and propose a solution approach based on deep reinforcement learning. To assess the quality of the proposed approach, we compare it with the optimal solution of a mixed-integer linear programming model that assumes full knowledge of transport requests in hindsight and hence serves as a lower-bound on the costs. We also compare our solution with a heuristic policy used in practice. We assess the performance of the proposed solutions in an industry case study using real-world data

Improving Intrinsic Exploration with Language Abstractions

Reinforcement learning (RL) agents are particularly hard to train when rewards are sparse. One common solution is to use intrinsic rewards to encourage agents to explore their environment. However, recent intrinsic exploration methods often use state-based novelty measures which reward low-level exploration and may not scale to domains requiring more abstract skills. Instead, we explore natural language as a general medium for highlighting relevant abstractions in an environment. Unlike previous work, we evaluate whether language can improve over existing exploration methods by directly extending (and comparing to) competitive intrinsic exploration baselines: AMIGo (Campero et al., 2021) and NovelD (Zhang et al., 2021). These language-based variants outperform their non-linguistic forms by 47-85% across 13 challenging tasks from the MiniGrid and MiniHack environment suites.Comment: NeurIPS 202

Adaptive reinforcement learning with active state-specific exploration for engagement maximization during simulated child-robot interaction

International audienceUsing assistive robots for educational applications requires robots to be able to adapt their behavior specifically for each child with whom they interact. Among relevant signals, non-verbal cues such as the child's gaze can provide the robot with important information about the child's current engagement in the task, and whether the robot should continue its current behavior or not. Here we propose a reinforcement learning algorithm extended with active state-specific exploration and show its applicability to child engagement maximization as well as more classical tasks such as maze navigation. We first demonstrate its adaptive nature on a continuous maze problem as an enhancement of the classic grid world. There, parame-terized actions enable the agent to learn single moves until the end of a corridor, similarly to "options" but without explicit hierarchical representations. We then apply the algorithm to a series of simulated scenarios, such as an extended Tower of Hanoi where the robot should find the appropriate speed of movement for the interacting child, and to a pointing task where the robot should find the child-specific appropriate level of expressivity of action. We show that the algorithm enables to cope with both global and local non-stationarities in the state space while preserving a stable behavior in other stationary portions of the state space. Altogether, these results suggest a promising way to enable robot learning based on non-verbal cues and the high degree of non-stationarities that can occur during interaction with children

Online Learning of Non-stationary Sequences

We consider an online learning scenario in which the learner can make predictions on the basis of a fixed set of experts. The performance of each expert may change over time in a manner unknown to the learner. We formulate a class of universal learning algorithms for this problem by expressing them as simple Bayesian algorithms operating on models analogous to Hidden Markov Models (HMMs). We derive a new performance bound for such algorithms which is considerably simpler than existing bounds. The bound provides the basis for learning the rate at which the identity of the optimal expert switches over time. We find an analytic expression for the a priori resolution at which we need to learn the rate parameter. We extend our scalar switching-rate result to models of the switching-rate that are governed by a matrix of parameters, i.e. arbitrary homogeneous HMMs. We apply and examine our algorithm in the context of the problem of energy management in wireless networks. We analyze the new results in the framework of Information Theory

On sequentializing concurrent programs

We propose a general framework for compositional under-approximate concurrent program analyses by reduction to sequential program analyses—so-called sequentializations. We notice the existing sequentializations—based on bounding the number of execution contexts, execution rounds, or delays from a deterministic task-schedule—rely on three key features for scalable concurrent program analyses: (i) reduction to the sequential program model, (ii) compositional reasoning to avoid expensive task-product constructions, and (iii) parameterized exploration bounds. To understand how those sequentializations can be unified and generalized, we define a general framework which preserves their key features, and in which those sequentializations are particular instances. We also identify a most general instance which considers more executions, by composing the rounds of different tasks in any order, restricted only by the unavoidable program and task-creation causality orders. In fact, we show this general instance is fundamentally more powerful by identifying an infinite family of state-reachability problems (to states g1,g2,...) which can be answered precisely with a fixed exploration bound, whereas the existing sequentializations require an increasing bound k to reach each gk. Our framework applies to a general class of shared-memory concurrent programs, with dynamic task-creation and arbitrary preemption