Working and Learning with Knowledge in the Lobes of a Humanoid's Mind

Humanoid class robots must have sufficient dexterity to assist people and work in an environment designed for human comfort and productivity. This dexterity, in particular the ability to use tools, requires a cognitive understanding of self and the world that exceeds contemporary robotics. Our hypothesis is that the sense-think-act paradigm that has proven so successful for autonomous robots is missing one or more key elements that will be needed for humanoids to meet their full potential as autonomous human assistants. This key ingredient is knowledge. The presented work includes experiments conducted on the Robonaut system, a NASA and the Defense Advanced research Projects Agency (DARPA) joint project, and includes collaborative efforts with a DARPA Mobile Autonomous Robot Software technical program team of researchers at NASA, MIT, USC, NRL, UMass and Vanderbilt. The paper reports on results in the areas of human-robot interaction (human tracking, gesture recognition, natural language, supervised control), perception (stereo vision, object identification, object pose estimation), autonomous grasping (tactile sensing, grasp reflex, grasp stability) and learning (human instruction, task level sequences, and sensorimotor association)

Workflow-based Context-aware Control of Surgical Robots

Surgical assistance system such as medical robots enhanced the capabilities of medical procedures in the last decades. This work presents a new perspective on the use of workflows with surgical robots in order to improve the technical capabilities and the ease of use of such systems. This is accomplished by a 3D perception system for the supervision of the surgical operating room and a workflow-based controller, that allows to monitor the surgical process using workflow-tracking techniques

Human Resource Management in Emergency Situations

The dissertation examines the issues related to the human resource management in emergency situations and introduces the measures helping to solve these issues. The prime aim is to analyse complexly a human resource management, built environment resilience management life cycle and its stages for the purpose of creating an effective Human Resource Management in Emergency Situations Model and Intelligent System. This would help in accelerating resilience in every stage, managing personal stress and reducing disaster-related losses. The dissertation consists of an Introduction, three Chapters, the Conclusions, References, List of Author’s Publications and nine Appendices. The introduction discusses the research problem and the research relevance, outlines the research object, states the research aim and objectives, overviews the research methodology and the original contribution of the research, presents the practical value of the research results, and lists the defended propositions. The introduction concludes with an overview of the author’s publications and conference presentations on the topic of this dissertation. Chapter 1 introduces best practice in the field of disaster and resilience management in the built environment. It also analyses disaster and resilience management life cycle ant its stages, reviews different intelligent decision support systems, and investigates researches on application of physiological parameters and their dependence on stress. The chapter ends with conclusions and the explicit objectives of the dissertation. Chapter 2 of the dissertation introduces the conceptual model of human resource management in emergency situations. To implement multiple criteria analysis of the research object the methods of multiple criteria analysis and mahematics are proposed. They should be integrated with intelligent technologies. In Chapter 3 the model developed by the author and the methods of multiple criteria analysis are adopted by developing the Intelligent Decision Support System for a Human Resource Management in Emergency Situations consisting of four subsystems: Physiological Advisory Subsystem to Analyse a User’s Post-Disaster Stress Management; Text Analytics Subsystem; Recommender Thermometer for Measuring the Preparedness for Resilience and Subsystem of Integrated Virtual and Intelligent Technologies. The main statements of the thesis were published in eleven scientific articles: two in journals listed in the Thomson Reuters ISI Web of Science, one in a peer-reviewed scientific journal, four in peer-reviewed conference proceedings referenced in the Thomson Reuters ISI database, and three in peer-reviewed conference proceedings in Lithuania. Five presentations were given on the topic of the dissertation at conferences in Lithuania and other countries

Novel robust computer vision algorithms for micro autonomous systems

People detection and tracking are an essential component of many autonomous platforms, interactive systems and intelligent vehicles used in various search and rescues operations and similar humanitarian applications. Currently, researchers are focusing on the use of vision sensors such as cameras due to their advantages over other sensor types. Cameras are information rich, relatively inexpensive and easily available. Additionally, 3D information is obtained from stereo vision, or by triangulating over several frames in monocular configurations. Another method to obtain 3D data is by using RGB-D sensors (e.g. Kinect) that provide both image and depth data. This method is becoming more attractive over the past few years due to its affordable price and availability for researchers. The aim of this research was to find robust multi-target detection and tracking algorithms for Micro Autonomous Systems (MAS) that incorporate the use of the RGB-D sensor. Contributions include the discovery of novel robust computer vision algorithms. It proposed a new framework for human body detection, from video file, to detect a single person adapted from Viola and Jones framework. The 2D Multi Targets Detection and Tracking (MTDT) algorithm applied the Gaussian Mixture Model (GMM) to reduce noise in the pre-processing stage. Blob analysis was used to detect targets, and Kalman filter was used to track targets. The 3D MTDT extends beyond 2D with the use of depth data from the RGB-D sensor in the pre-processing stage. Bayesian model was employed to provide multiple cues. It includes detection of the upper body, face, skin colour, motion and shape. Kalman filter proved for speed and robustness of the track management. Simultaneous Localisation and Mapping (SLAM) fusing with 3D information was investigated. The new framework introduced front end and back end processing. The front end consists of localisation steps, post refinement and loop closing system. The back-end focus on the post-graph optimisation to eliminate errors.The proposed computer vision algorithms proved for better speed and robustness. The frameworks produced impressive results. New algorithms can be used to improve performances in real time applications including surveillance, vision navigation, environmental perception and vision-based control system on MAS

Communicative humanoids : a computational model of psychosocial dialogue skills

Thesis (Ph. D.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1996.Includes bibliographical references (p. [223]-238).Kristinn Rúnar Thórisson.Ph.D

Context-based multimedia semantics modelling and representation

The evolution of the World Wide Web, increase in processing power, and more network bandwidth have contributed to the proliferation of digital multimedia data. Since multimedia data has become a critical resource in many organisations, there is an increasing need to gain efficient access to data, in order to share, extract knowledge, and ultimately use the knowledge to inform business decisions. Existing methods for multimedia semantic understanding are limited to the computable low-level features; which raises the question of how to identify and represent the high-level semantic knowledge in multimedia resources.In order to bridge the semantic gap between multimedia low-level features and high-level human perception, this thesis seeks to identify the possible contextual dimensions in multimedia resources to help in semantic understanding and organisation. This thesis investigates the use of contextual knowledge to organise and represent the semantics of multimedia data aimed at efficient and effective multimedia content-based semantic retrieval.A mixed methods research approach incorporating both Design Science Research and Formal Methods for investigation and evaluation was adopted. A critical review of current approaches for multimedia semantic retrieval was undertaken and various shortcomings identified. The objectives for a solution were defined which led to the design, development, and formalisation of a context-based model for multimedia semantic understanding and organisation. The model relies on the identification of different contextual dimensions in multimedia resources to aggregate meaning and facilitate semantic representation, knowledge sharing and reuse. A prototype system for multimedia annotation, CONMAN was built to demonstrate aspects of the model and validate the research hypothesis, H₁.Towards providing richer and clearer semantic representation of multimedia content, the original contributions of this thesis to Information Science include: (a) a novel framework and formalised model for organising and representing the semantics of heterogeneous visual data; and (b) a novel S-Space model that is aimed at visual information semantic organisation and discovery, and forms the foundations for automatic video semantic understanding