Conversational Sensing

Recent developments in sensing technologies, mobile devices and context-aware user interfaces have made it possible to represent information fusion and situational awareness as a conversational process among actors - human and machine agents - at or near the tactical edges of a network. Motivated by use cases in the domain of security, policing and emergency response, this paper presents an approach to information collection, fusion and sense-making based on the use of natural language (NL) and controlled natural language (CNL) to support richer forms of human-machine interaction. The approach uses a conversational protocol to facilitate a flow of collaborative messages from NL to CNL and back again in support of interactions such as: turning eyewitness reports from human observers into actionable information (from both trained and untrained sources); fusing information from humans and physical sensors (with associated quality metadata); and assisting human analysts to make the best use of available sensing assets in an area of interest (governed by management and security policies). CNL is used as a common formal knowledge representation for both machine and human agents to support reasoning, semantic information fusion and generation of rationale for inferences, in ways that remain transparent to human users. Examples are provided of various alternative styles for user feedback, including NL, CNL and graphical feedback. A pilot experiment with human subjects shows that a prototype conversational agent is able to gather usable CNL information from untrained human subjects

Customer-engineer relationship management for converged ICT service companies

Thanks to the advent of converged communications services (often referred to as ‘triple play’), the next generation Service Engineer will need radically different skills, processes and tools from today’s counterpart. Why? in order to meet the challenges of installing and maintaining services based on multi-vendor software and hardware components in an IP-based network environment. The converged services environment is likely to be ‘smart’ and support flexible and dynamic interoperability between appliances and computing devices. These radical changes in the working environment will inevitably force managers to rethink the role of Service Engineers in relation to customer relationship management. This paper aims to identify requirements for an information system to support converged communications service engineers with regard to customer-engineer relationship management. Furthermore, an architecture for such a system is proposed and how it meets these requirements is discussed

Combining heterogeneous inputs for the development of adaptive and multimodal interaction systems

In this paper we present a novel framework for the integration of visual sensor networks and speech-based interfaces. Our proposal follows the standard reference architecture in fusion systems (JDL), and combines different techniques related to Artificial Intelligence, Natural Language Processing and User Modeling to provide an enhanced interaction with their users. Firstly, the framework integrates a Cooperative Surveillance Multi-Agent System (CS-MAS), which includes several types of autonomous agents working in a coalition to track and make inferences on the positions of the targets. Secondly, enhanced conversational agents facilitate human-computer interaction by means of speech interaction. Thirdly, a statistical methodology allows modeling the user conversational behavior, which is learned from an initial corpus and improved with the knowledge acquired from the successive interactions. A technique is proposed to facilitate the multimodal fusion of these information sources and consider the result for the decision of the next system action.This work was supported in part by Projects MEyC TEC2012-37832-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS S2009/TIC-1485Publicad

A multi-agent architecture to combine heterogeneous inputs in multimodal interaction systems

Actas de: CAEPIA 2013, Congreso federado Agentes y Sistemas Multi-Agente: de la Teoría a la Práctica (ASMas). Madrid, 17-20 Septiembre 2013.In this paper we present a multi-agent architecture for the integration of visual sensor networks and speech-based interfaces. The proposed architecture combines different techniques related to Artificial Intelligence, Natural Language Processing and User Modeling to provide an enhanced interaction with their users. Firstly, the architecture integrates a Cooperative Surveillance Multi-Agent System (CS-MAS), which includes several types of autonomous agents working in a coalition to track and make inferences on the positions of the targets. Secondly, the proposed architecture incorporates enhanced conversational agents to facilitate human-computer interaction by means of speech interaction. Thirdly, a statistical methodology allows to model the user conversational behavior, which is learned from an initial corpus and posteriorly improved with the knowledge acquired from the successive interactions. A technique is proposed to facilitate the multimodal fusion of these information sources and consider the result for the decision of the next system action.This work was supported in part by Projects MINECO TEC2012-37832-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485).Publicad

A web-based collaboration approach for teaching in medicine

Teaching medicine requires developing a vast range of manual, intellectual, visual and tactile skills as well as taking into account large amounts of factual information. Traditional medical teaching and individual learning in particular, can be complemented with electronic web based systems. One of the main impacts of e-Teaching in education resides in the fact that it provides opportunities to create resources that turn the learning process flexible. This implies a different relation between teachers and students and even between institutions, in the sense that the students participate on their own formation and the vertical hierarchy tends to become increasingly more horizontal. Awareness of the knowledge constructing process is increased, and consequently more satisfaction gained from learning. In this paper we describe a webbased collaboration approach for teaching that is being developed to simulate conversational dialogue in the area of Medicine, that enables the integration of highly heterogeneous sources of information into a coherent knowledge base accessed from web-based interfaces, either from the tutor’s point of view or the development of the discipline in itself, i.e. the system’s content is created automatically by the physicians as their daily work goes on

mPower: A component-based development framework for multi-agent systems to support business processes

One of the obstacles preventing the widespread adoption of multi-agent systems in industry is the difficulty of implementing heterogeneous interactions among participating agents via asynchronous messages. This difficulty arises from the need to understand how to combine elements of various content languages, ontologies, and interaction protocols in order to construct meaningful and appropriate messages. In this paper mPower, a component-based layered framework for easing the development of multi-agent systems, is described, and the facility for customising the components for reuse in similar domains is explained. The framework builds on the JADE-LEAP platform, which provides a homogeneous layer over diverse operating systems and hardware devices, and allows ubiquitous deployment of applications built on multi-agent systems both in wired and wireless environments. The use of the framework to develop mPowermobile , a multi-agent system to support mobile workforces, is reported

Multimodal agent interfaces and system architectures for health and fitness companions

Multimodal conversational spoken dialogues using physical and virtual agents provide a potential interface to motivate and support users in the domain of health and fitness. In this paper we present how such multimodal conversational Companions can be implemented to support their owners in various pervasive and mobile settings. In particular, we focus on different forms of multimodality and system architectures for such interfaces