Software agents & human behavior

People make important decisions in emergencies. Often these decisions involve high stakes in terms of lives and property. Bhopal disaster (1984), Piper Alpha disaster (1988), Montara blowout (2009), and explosion on Deepwater Horizon (2010) are a few examples among many industrial incidents. In these incidents, those who were in-charge took critical decisions under various ental stressors such as time, fatigue, and panic. This thesis presents an application of naturalistic decision-making (NDM), which is a recent decision-making theory inspired by experts making decisions in real emergencies. This study develops an intelligent agent model that can be programed to make human-like decisions in emergencies. The agent model has three major components: (1) A spatial learning module, which the agent uses to learn escape routes that are designated routes in a facility for emergency evacuation, (2) a situation recognition module, which is used to recognize or distinguish among evolving emergency situations, and (3) a decision-support module, which exploits modules in (1) and (2), and implements an NDM based decision-logic for producing human-like decisions in emergencies. The spatial learning module comprises a generalized stochastic Petri net-based model of spatial learning. The model classifies routes into five classes based on landmarks, which are objects with salient spatial features. These classes deal with the question of how difficult a landmark turns out to be when an agent observes it the first time during a route traversal. An extension to the spatial learning model is also proposed where the question of how successive route traversals may impact retention of a route in the agent’s memory is investigated. The situation awareness module uses Markov logic network (MLN) to define different offshore emergency situations using First-order Logic (FOL) rules. The purpose of this module is to give the agent the necessary experience of dealing with emergencies. The potential of this module lies in the fact that different training samples can be used to produce agents having different experience or capability to deal with an emergency situation. To demonstrate this fact, two agents were developed and trained using two different sets of empirical observations. The two are found to be different in recognizing the prepare-to-abandon-platform alarm (PAPA ), and similar to each other in recognition of an emergency using other cues. Finally, the decision-support module is proposed as a union of spatial-learning module, situation awareness module, and NDM based decision-logic. The NDM-based decision-logic is inspired by Klein’s (1998) recognition primed decision-making (RPDM) model. The agent’s attitudes related to decision-making as per the RPDM are represented in the form of belief, desire, and intention (BDI). The decision-logic involves recognition of situations based on experience (as proposed in situation-recognition module), and recognition of situations based on classification, where ontological classification is used to guide the agent in cases where the agent’s experience about confronting a situation is inadequate. At the planning stage, the decision-logic exploits the agent’s spatial knowledge (as proposed in spatial-learning module) about the layout of the environment to make adjustments in the course of actions relevant to a decision that has already been made as a by-product of situation recognition. The proposed agent model has potential to be used to improve virtual training environment’s fidelity by adding agents that exhibit human-like intelligence in performing tasks related to emergency evacuation. Notwithstanding, the potential to exploit the basis provided here, in the form of an agent representing human fallibility, should not be ignored for fields like human reliability analysis

A graphical environment and applications for discrete event and hybrid systems in robotics and automation

technical reportIn this paper we present an overview for the development of a graphical environment for simulating, analyzing, synthesizing, monitoring, and controlling complex discrete event and hybrid systems within the robotics, automation, and intelligent system domain. We start by presenting an overview of discrete event and hybrid systems, and then discuss the proposed framework. We also present two applications within the robotics and automation domain for such complex systems. The first is for formulating an observer for manipulating agents, and the second is for designing sensing strategies for the inspection of machine parts

Evaluating Resilience of Cyber-Physical-Social Systems

Nowadays, protecting the network is not the only security concern. Still, in cyber security, websites and servers are becoming more popular as targets due to the ease with which they can be accessed when compared to communication networks. Another threat in cyber physical social systems with human interactions is that they can be attacked and manipulated not only by technical hacking through networks, but also by manipulating people and stealing users’ credentials. Therefore, systems should be evaluated beyond cy- ber security, which means measuring their resilience as a piece of evidence that a system works properly under cyber-attacks or incidents. In that way, cyber resilience is increas- ingly discussed and described as the capacity of a system to maintain state awareness for detecting cyber-attacks. All the tasks for making a system resilient should proactively maintain a safe level of operational normalcy through rapid system reconfiguration to detect attacks that would impact system performance. In this work, we broadly studied a new paradigm of cyber physical social systems and defined a uniform definition of it. To overcome the complexity of evaluating cyber resilience, especially in these inhomo- geneous systems, we proposed a framework including applying Attack Tree refinements and Hierarchical Timed Coloured Petri Nets to model intruder and defender behaviors and evaluate the impact of each action on the behavior and performance of the system.Hoje em dia, proteger a rede não é a única preocupação de segurança. Ainda assim, na segurança cibernética, sites e servidores estão se tornando mais populares como alvos devido à facilidade com que podem ser acessados quando comparados às redes de comu- nicação. Outra ameaça em sistemas sociais ciberfisicos com interações humanas é que eles podem ser atacados e manipulados não apenas por hackers técnicos através de redes, mas também pela manipulação de pessoas e roubo de credenciais de utilizadores. Portanto, os sistemas devem ser avaliados para além da segurança cibernética, o que significa medir sua resiliência como uma evidência de que um sistema funciona adequadamente sob ataques ou incidentes cibernéticos. Dessa forma, a resiliência cibernética é cada vez mais discutida e descrita como a capacidade de um sistema manter a consciência do estado para detectar ataques cibernéticos. Todas as tarefas para tornar um sistema resiliente devem manter proativamente um nível seguro de normalidade operacional por meio da reconfi- guração rápida do sistema para detectar ataques que afetariam o desempenho do sistema. Neste trabalho, um novo paradigma de sistemas sociais ciberfisicos é amplamente estu- dado e uma definição uniforme é proposta. Para superar a complexidade de avaliar a resiliência cibernética, especialmente nesses sistemas não homogéneos, é proposta uma estrutura que inclui a aplicação de refinamentos de Árvores de Ataque e Redes de Petri Coloridas Temporizadas Hierárquicas para modelar comportamentos de invasores e de- fensores e avaliar o impacto de cada ação no comportamento e desempenho do sistema

A Methodology for Extracting Human Bodies from Still Images

Monitoring and surveillance of humans is one of the most prominent applications of today and it is expected to be part of many future aspects of our life, for safety reasons, assisted living and many others. Many efforts have been made towards automatic and robust solutions, but the general problem is very challenging and remains still open. In this PhD dissertation we examine the problem from many perspectives. First, we study the performance of a hardware architecture designed for large-scale surveillance systems. Then, we focus on the general problem of human activity recognition, present an extensive survey of methodologies that deal with this subject and propose a maturity metric to evaluate them. One of the numerous and most popular algorithms for image processing found in the field is image segmentation and we propose a blind metric to evaluate their results regarding the activity at local regions. Finally, we propose a fully automatic system for segmenting and extracting human bodies from challenging single images, which is the main contribution of the dissertation. Our methodology is a novel bottom-up approach relying mostly on anthropometric constraints and is facilitated by our research in the fields of face, skin and hands detection. Experimental results and comparison with state-of-the-art methodologies demonstrate the success of our approach

Dagstuhl News January - December 2000

"Dagstuhl News" is a publication edited especially for the members of the Foundation "Informatikzentrum Schloss Dagstuhl" to thank them for their support. The News give a summary of the scientific work being done in Dagstuhl. Each Dagstuhl Seminar is presented by a small abstract describing the contents and scientific highlights of the seminar as well as the perspectives or challenges of the research topic

Second Workshop on Modelling of Objects, Components and Agents

This report contains the proceedings of the workshop Modelling of Objects, Components, and Agents (MOCA'02), August 26-27, 2002.The workshop is organized by the 'Coloured Petri Net' Group at the University of Aarhus, Denmark and the 'Theoretical Foundations of Computer Science' Group at the University of Hamburg, Germany. The homepage of the workshop is: http://www.daimi.au.dk/CPnets/workshop02

Model-based engineering of animated interactive systems for the interactive television environment

Les interfaces graphiques étaient la plupart du temps statiques, et représentaient une succession d'états logiciels les uns après les autres. Cependant, les transitions animées entre ces états statiques font partie intégrante des interfaces utilisateurs modernes, et leurs processus de design et d'implémentations constituent un défi pour les designers et les développeurs. Cette thèse propose un processus de conception de systèmes interactifs centré sur les animations, ainsi qu'une architecture pour la définition et l'implémentation d'animations au sein des interfaces graphiques. L'architecture met en avant une approche à deux niveaux pour définir une vue haut niveau d'une animation (avec un intérêt particulier pour les objets animés, leurs propriétés à être animé et la composition d'animations) ainsi qu'une vue bas niveau traitant des aspects détaillés des animations tels que les timings et les optimisations. Concernant les spécifications formelles de ces deux niveaux, nous utilisons une approche qui facilite les réseaux de Petri orientés objets pour la conception, l'implémentation et la validation d'interfaces utilisateurs animées en fournissant une description complète et non-ambiguë de l'ensemble de l'interface utilisateur, y compris les animations. Enfin, nous décrivons la mise en pratique du processus présenté, illustré par un cas d'étude d'un prototype haute-fidélité d'une interface utilisateur, pour le domaine de la télévision interactive. Ce processus conduira à une spécification formelle et détaillée du système interactif, et incluera des animations utilisant des réseaux de Petri orientés objet (conçus avec l'outil PetShop CASE).Graphical User Interfaces used to be mostly static, representing one software state after the other. However, animated transitions between these static states are an integral part in modern user interfaces and processes for both their design and implementation remain a challenge for designers and developers. This thesis proposes a process for designing interactive systems focusing on animations, along with an architecture for the definition and implementation of animation in user interfaces. The architecture proposes a two levels approach for defining a high-level view of an animation (focusing on animated objects, their properties to be animated and on the composition of animations) and a low-level one dealing with detailed aspects of animations such as timing and optimization. For the formal specification of these two levels, we are using an approach facilitating object-oriented Petri nets to support the design, implementation and validation of animated user interfaces by providing a complete and unambiguous description of the entire user interface including animations. Finally, we describe the application of the presented process exemplified by a case study for a high-fidelity prototype of a user interface for the interactive Television domain. This process will lead to a detailed formal specification of the interactive system, including animations using object-oriented Petri nets (designed with the PetShop CASE tool)

Extra Functional Properties Evaluation of Self-managed Software Systems with Formal Methods

Multitud de aplicaciones software actuales están abocadas a operar en contextos dinámicos. Estos pueden manifestarse en términos de cambios en el entorno de ejecución de la aplicación, cambios en los requisitos de la aplicación, cambios en la carga de trabajo recibida por la aplicación, o cambios en cualquiera de los elementos que la aplicación software pueda percibir y verse afectada. Además, estos contextos dinámicos no están restringidos a un dominio particular de aplicaciones sino que se pueden encontrar en múltiples dominios, tales como: sistemas empotrados, arquitecturas orientadas a servicios, clusters para computación de altas prestaciones, dispositivos móviles o software para el funcionamiento de la red. La existencia de estas características disuade a los ingenieros de desarrollar software que no sea capaz de cambiar de modo alguno su ejecución para acomodarla al contexto en el que se está ejecutando el software en cada momento. Por lo tanto, con el objetivo de que el software pueda satisfacer sus requisitos en todo momento, este debe incluir mecanismos para poder cambiar su configuración de ejecución. Además, debido a que los cambios de contexto son frecuentes y afectan a múltiples dispositivos de la aplicación, la intervención humana que cambie manualmente la configuración del software no es una solución factible. Para enfrentarse a estos desafíos, la comunidad de Ingeniería del Software ha propuesto nuevos paradigmas que posibilitan el desarrollo de software que se enfrenta a contextos cambiantes de un modo automático; por ejemplo las propuestas Autonomic Computing y Self-* Software. En tales propuestas es el propio software quien gestiona sus mecanismos para cambiar la configuración de ejecución, sin requerir por lo tanto intervención humana alguna. Un aspecto esencial del software auto-adaptativo (Self-adaptive Software es uno de los términos más generales para referirse a Self-* Software) es el de planear sus cambios o adaptaciones. Los planes de adaptación determinan tanto el modo en el que se adaptará el software como los momentos oportunos para ejecutar tales adaptaciones. Hay un gran conjunto de situaciones para las cuales la propiedad de auto- adaptación es una solución. Una de esas situaciones es la de mantener al sistema satisfaciendo sus requisitos extra funcionales, tales como la calidad de servicio (Quality of Service, QoS) y su consumo de energía. Esta tesis ha investigado esa situación mediante el uso de métodos formales. Una de las contribuciones de esta tesis es la propuesta para asentar en una arquitectura software los sistemas que son auto-adaptativos respecto a su QoS y su consumo de energía. Con este objetivo, esta parte de la investigación la guía una arquitectura de tres capas de referencia para sistemas auto-adaptativos. La bondad del uso de una arquitectura de referencia es que muestra fácilmente los nuevos desafíos en el diseño de este tipo de sistemas. Naturalmente, la planificación de la adaptación es una de las actividades consideradas en la arquitectura. Otra de las contribuciones de la tesis es la propuesta de métodos para la creación de planes de adaptación. Los métodos formales juegan un rol esencial en esta actividad, ya que posibilitan el estudio de las propiedades extra funcionales de los sistemas en diferentes configuraciones. El método formal utilizado para estos análisis es el de las redes de Petri markovianas. Una vez que se ha creado el plan de adaptación, hemos investigado la utilización de los métodos formales para la evaluación de QoS y consumo de energía de los sistemas auto-adaptativos. Por lo tanto, se ha contribuido a la comunidad de análisis de QoS con el análisis de un nuevo y particularmente complejo tipo de sistemas software. Para llevar a cabo este análisis se requiere el modelado de los cambios din·micos del contexto de ejecución, para lo que se han utilizado una variedad de métodos formales, como los Markov modulated Poisson processes para estimar los parámetros de las variaciones en la carga de trabajo recibida por la aplicación, o los hidden Markov models para predecir el estado del entorno de ejecución. Estos modelos han sido usados junto a las redes de Petri para evaluar sistemas auto-adaptativos y obtener resultados sobre su QoS y consumo de energía. El trabajo de investigación anterior sacó a la luz el hecho de que la adaptabilidad de un sistema no es una propiedad tan fácilmente cuantificable como las propiedades de QoS -por ejemplo, el tiempo de respuesta- o el consumo de energÌa. En consecuencia, se ha investigado en esa dirección y, como resultado, otra de las contribuciones de esta tesis es la propuesta de un conjunto de métricas para la cuantificación de la propiedad de adaptabilidad de sistemas basados en servicios. Para conseguir las anteriores contribuciones se realiza un uso intensivo de modelos y transformaciones de modelos; tarea para la que se han seguido las mejores prácticas en el campo de investigación de la Ingeniería orientada a modelos (Model-driven Engineering, MDE). El trabajo de investigación de esta tesis en el campo MDE ha contribuido con: el aumento de la potencia de modelado de un lenguaje de modelado de software propuesto anteriormente y métodos de transformación desde dos lenguajes de modelado de software a redes de Petri estocasticas

A Novel Prediction Method for Early Recognition of Global Human Behaviour in Image Sequences

Human behaviour recognition has been, and still remains, a challenging problem that involves different areas of computational intelligence. The automated understanding of people activities from video sequences is an open research topic in which the computer vision and pattern recognition areas have made big efforts. In this paper, the problem is studied from a prediction point of view. We propose a novel method able to early detect behaviour using a small portion of the input, in addition to the capabilities of it to predict behaviour from new inputs. Specifically, we propose a predictive method based on a simple representation of trajectories of a person in the scene which allows a high level understanding of the global human behaviour. The representation of the trajectory is used as a descriptor of the activity of the individual. The descriptors are used as a cue of a classification stage for pattern recognition purposes. Classifiers are trained using the trajectory representation of the complete sequence. However, partial sequences are processed to evaluate the early prediction capabilities having a specific observation time of the scene. The experiments have been carried out using the three different dataset of the CAVIAR database taken into account the behaviour of an individual. Additionally, different classic classifiers have been used for experimentation in order to evaluate the robustness of the proposal. Results confirm the high accuracy of the proposal on the early recognition of people behaviours.This work was supported in part by the University of Alicante, Valencian Government and Spanish government under grants GRE11-01, GV/2013/005 and DPI2013-40534-R