A Formal Architecture of Shared Mental Models for Computational Improvisational Agents

This paper proposes a formal approach of constructing shared mental models between computational improvisational agents (improv agents) and human interactors based on our socio-cognitive studies of human improvisers. Creating shared mental models helps improv agents co-create stories with each other and interactors in real-time interactive narrative experiences. The approach described here allows flexible modeling of non-Boolean (i.e. fuzzy) knowledge about scene and background concepts through the use of fuzzy rules and confidence factors in order to allow reasoning under uncertainty. It also allows improv agents to infer new knowledge about a scene from existing knowledge, recognize when new knowledge may be divergent from the other actor’s mental model, and attempt to resolve this divergence to reach cognitive consensus despite the absence of explicit goals in the story environment

An Argument for the Improvisational Design of Customer Service Behaviours

The thesis examines the role of improvisation in the theory and practice of design for service and management-oriented design thinking. This examination reveals several features that design thinking and improvisation share, which has significance for service innovation - as it suggests that these features might be incorporated into ‘real-time’ service encounters to make service workers’ improvised behaviours more designerly. Manzini’s ‘action platform’ concept is explored, and used to illustrate how design and improvisation might be combined

Towards a framework for socially interactive robots

250 p.En las últimas décadas, la investigación en el campo de la robótica social ha crecido considerablemente. El desarrollo de diferentes tipos de robots y sus roles dentro de la sociedad se están expandiendo poco a poco. Los robots dotados de habilidades sociales pretenden ser utilizados para diferentes aplicaciones; por ejemplo, como profesores interactivos y asistentes educativos, para apoyar el manejo de la diabetes en niños, para ayudar a personas mayores con necesidades especiales, como actores interactivos en el teatro o incluso como asistentes en hoteles y centros comerciales.El equipo de investigación RSAIT ha estado trabajando en varias áreas de la robótica, en particular,en arquitecturas de control, exploración y navegación de robots, aprendizaje automático y visión por computador. El trabajo presentado en este trabajo de investigación tiene como objetivo añadir una nueva capa al desarrollo anterior, la capa de interacción humano-robot que se centra en las capacidades sociales que un robot debe mostrar al interactuar con personas, como expresar y percibir emociones, mostrar un alto nivel de diálogo, aprender modelos de otros agentes, establecer y mantener relaciones sociales, usar medios naturales de comunicación (mirada, gestos, etc.),mostrar personalidad y carácter distintivos y aprender competencias sociales.En esta tesis doctoral, tratamos de aportar nuestro grano de arena a las preguntas básicas que surgen cuando pensamos en robots sociales: (1) ¿Cómo nos comunicamos (u operamos) los humanos con los robots sociales?; y (2) ¿Cómo actúan los robots sociales con nosotros? En esa línea, el trabajo se ha desarrollado en dos fases: en la primera, nos hemos centrado en explorar desde un punto de vista práctico varias formas que los humanos utilizan para comunicarse con los robots de una maneranatural. En la segunda además, hemos investigado cómo los robots sociales deben actuar con el usuario.Con respecto a la primera fase, hemos desarrollado tres interfaces de usuario naturales que pretenden hacer que la interacción con los robots sociales sea más natural. Para probar tales interfaces se han desarrollado dos aplicaciones de diferente uso: robots guía y un sistema de controlde robot humanoides con fines de entretenimiento. Trabajar en esas aplicaciones nos ha permitido dotar a nuestros robots con algunas habilidades básicas, como la navegación, la comunicación entre robots y el reconocimiento de voz y las capacidades de comprensión.Por otro lado, en la segunda fase nos hemos centrado en la identificación y el desarrollo de los módulos básicos de comportamiento que este tipo de robots necesitan para ser socialmente creíbles y confiables mientras actúan como agentes sociales. Se ha desarrollado una arquitectura(framework) para robots socialmente interactivos que permite a los robots expresar diferentes tipos de emociones y mostrar un lenguaje corporal natural similar al humano según la tarea a realizar y lascondiciones ambientales.La validación de los diferentes estados de desarrollo de nuestros robots sociales se ha realizado mediante representaciones públicas. La exposición de nuestros robots al público en esas actuaciones se ha convertido en una herramienta esencial para medir cualitativamente la aceptación social de los prototipos que estamos desarrollando. De la misma manera que los robots necesitan un cuerpo físico para interactuar con el entorno y convertirse en inteligentes, los robots sociales necesitan participar socialmente en tareas reales para las que han sido desarrollados, para así poder mejorar su sociabilida

Diagnostic reasoning approaches and success rates in bomb disposal

As professions, medicine and bomb disposal have many similarities, with one easily recognizable commonality being that practitioners in both disciplines rely on decision-making that is objective, dispassionate, and to the largest extent possible, grounded in scientific theory. Using research methodologies honed over decades in the medical community, this study investigates diagnostic reasoning approaches and success rates in the bomb disposal community, as viewed through the lens of improvised explosive device (IED) circuit analysis, which includes component identification, hazard assessment, and circuit type-by-function determination. The population for this study consisted of current and former military and civilian bomb technicians, and factors such as years of bomb disposal experience, length of initial training, and specialized IED training were analyzed to determine effects on success rates. A convergent mixed-methods design with a pragmatistic worldview was used, and the data gathered suggests that overall, no variables assessed had any effect on a bomb technician’s ability to successfully perform component identification, assessment of associated hazards, and determination of circuit type-by-function. Quantitatively, average success rates for study participants, by independent variable, showed no statistically significant differences, except for those who attended specific bomb disposal schools for their initial training, and only for circuit type-by-function determinations. Average success rates for study participants were 20% for component identification; 16% for associated hazards; and 51% for circuit type-by-function. Qualitatively, over 90% of participants used Type 1 decision-making (i.e., heuristics and pattern matching) as their diagnostic reasoning approach, and focused on component identification and circuit configurations in determining hazards associated with devices, and circuit type-by-function. Additionally, an analysis of component and hazard selections clearly suggests that bomb technicians key in on specific components, and these selections drive their further analysis. Self-assessed confidence-level data also suggests that study participants significantly over-rated their ability to recognize components, assess hazards, and determine circuit type-by-function. The results of this study can be used by thought leaders and trainers in the bomb disposal community to push for fostering and improving diagnostic reasoning skills, problem-solving, and critical thinking, which in turn should lead to a reduction in operational errors during IED response operations

Automated Discovery and Interpretation of ADA-Compliant Door Placards

A familiar difficulty to any new student on campus is making one’s way from classroom A to classroom B. Facilities with different wings, multiple floors, and irregular floorplans can magnify this challenge, while students with vision impairments are impacted even more by the challenge of identifying the destination. This thesis explored different methods of discovering Americans with Disabilities Act (ADA)- compliant room identifying placards (“plaques”) and identifying the text on the sign. The plaque detection was accomplished with both standard image manipulation techniques and a Histogram of Oriented Gradients (HOG) (Dalal & Triggs, 2005) object detector. The text reading utilized both standard image manipulation tools as well as an implementation of the Efficient and Accurate Scene Text detector (EAST) (Zhou et al., 2017) to isolate text, while Tesseract (Smith, 2007) was used to interpret the text. Different methods of dataset generation were utilized to train the detectors, including manual gathering, internet search scraping, and dataset generation. Results of testing these different methods on a dataset of image frames gathered from filming the Computer Science/Information Technology (CSIT) hallway of Kutztown University’s Old Main building proved the combination of HOG and EAST to be an effective method for identifying and transcribing room identification plaques. In the case of consistent visual design of rooms signs, the generated dataset proved to be nearly as effective as training the detector on real annotated images

Context Aided Tracking with Adaptive Hyperspectral Imagery

A methodology for the context-aided tracking of ground vehicles in remote airborne imagery is developed in which a background model is inferred from hyperspectral imagery. The materials comprising the background of a scene are remotely identified and lead to this model. Two model formation processes are developed: a manual method, and method that exploits an emerging adaptive, multiple-object-spectrometer instrument. A semi-automated background modeling approach is shown to arrive at a reasonable background model with minimal operator intervention. A novel, adaptive, and autonomous approach uses a new type of adaptive hyperspectral sensor, and converges to a 66% correct background model in 5% the time of the baseline {a 95% reduction in sensor acquisition time. A multiple-hypothesis-tracker is incorporated, which utilizes background statistics to form track costs and associated track maintenance thresholds. The context-aided system is demonstrated in a high- fidelity tracking testbed, and reduces track identity error by 30%

Live Sampling in Improvised Musical Performance: Three Approaches and a Discussion of Aesthetics

Three original software programs utilizing improvisation and live sampling are presented here, along with a discussion of aesthetic issues raised by each. They are entitled Zur Elektrodynamik bewegter Musiker, Motet, and Gamepad Sampler. These programs vary in the degree of required interaction and in the kind of user control. They are each studies in imitative counterpoint through live sampling, with an approach seeking elegance before solutions. Because of the improvisational nature of these works, there is no standard musical score. Instead the complete Max/MSP source code and a sound recording of performances making use of these programs in varied situations are included. A discussion of issues raised by these works includes aesthetics, ontology, performance, and the role of the composer. Non-interactive indeterminate compositions are ontologically thin, because some composerly agency is required of the performer. An interactive work can be ontologically substantial if it makes distinct and significant contributions to performance, even though it may not make sound on its own. Although reproducibility reduces ontology and eliminates aura, live sampling within a performance can deepen the ontology of the performance by recontextualizing previous events, reframing the original event as the first reference to an abstract musical idea that lies outside the musical performance. Reproducibility also diminishes the aura or stage presence in live performance with computers. Complex feedback systems can be used to create computation instruments: musical instruments whose unique structure resonates in ways not explicit in their programs. As the human condition and the situation of the composer change, definitions of the composer and performer must be revised. Composition is shifting away from the creation of static artifacts toward the design of dynamic systems

Development and Characterization of a Chromotomosynthetic Hyperspectral Imaging System

A chromotomosynthetic imaging (CTI) methodology based upon mathematical reconstruction of a set of 2-D spectral projections to collect high-speed (100 Hz) 3-D hyperspectral data cube has been proposed. The CTI system can simultaneously provide usable 3-D spatial and spectral information, provide high-frame rate slitless 1-D spectra, and generate 2-D imagery equivalent to that collected with no prism in the optical system. The wavelength region where prism dispersion is highest (500 nm) is most sensitive to loss of spectral resolution in the presence of systematic error, while wavelengths 600 nm suffer mostly from a shift of the spectral peaks. The quality of the spectral resolution in the reconstructed hyperspectral imagery was degraded by as much as a factor of two in the blue spectral region with less than 1° total angular error in mount alignment in the two axes of freedom. Even with no systematic error, spatial artifacts from the reconstruction limit the ability to provide adequate spectral imagery without specialized image reconstruction techniques as targets become more spatially and spectrally uniform