Human perception capabilities for socially intelligent domestic service robots

The daily living activities for an increasing number of frail elderly people represent a continuous struggle both for them as well as for their extended families. These people have difficulties coping at home alone but are still sufficiently fit not to need the round-the-clock care provided by a nursing home. Their struggle can be alleviated by the deployment of a mechanical helper in their home, i.e. a service robot that can execute a range of simple object manipulation tasks. Such a robotic application promises to extend the period of independent home living for elderly people, while providing them with a better quality of life. However, despite the recent technological advances in robotics, there are still some remaining challenges, mainly related to the human factors. Arguably, the lack of consistently dependable human detection, localisation, position and pose tracking information and insufficiently refined processing of sensor information makes the close range physical interaction between a robot and a human a high-risk task. The work described in this thesis addresses the deficiencies in the processing of the human information of today’s service robots. This is achieved through proposing a new paradigm for the robot’s situational awareness in regard to people as well as a collection of methods and techniques, operating at the lower levels of the paradigm, i.e. perception of new human information. The collection includes methods for obtaining and processing of information about the presence, location and body pose of the people. In addition to the availability of reliable human perception information, the integration between the separate levels of paradigm is considered to be a critically important factor for achieving the human-aware control of the robot. Improving the cognition, judgment and decision making action links between the paradigm’s layers leads to enhanced capability of the robot to engage in a natural and more meaningful interaction with people and, therefore, to a more enjoyable user experience. Therefore, the proposed paradigm and methodology are envisioned to contribute to making the prolonged assisted living of elderly people at home a more feasible and realistic task. In particular, this thesis proposes a set of methods for human presence detection, localisation and body pose tracking that are operating on the perception level of the paradigm. Also, the problem of having only limited visibility of a person from the on-board sensors of the robot is addressed by the proposed classifier fusion method that combines information from several types of sensors. A method for improved real-time human body pose tracking is also investigated. Additionally, a method for estimation of the multiple human tracks from noisy detections, as well as analysis of the computed human tracks for cognition about the social interactions within the social group, operating at the comprehension level of the robot’s situational awareness paradigm, is proposed. Finally, at the human-aware planning layer, a method that utilises the human related information, generated by the perception and comprehension layers to compute a minimally intrusive navigation path to a target person within a human group, is proposed. This method demonstrates how the improved human perception capabilities of the robot, through its judgement activity, ii ABSTRACT can be utilised by the highest level of the paradigm, i.e. the decision making layer, to achieve user friendly human-robot interactions. Overall, the research presented in this work, drawing on recent innovation in statistical learning, data fusion and optimisation methods, improves the overall situational awareness of the robot in regard to people with the main focus placed on human sensing capabilities of service robots. The improved overall situational awareness of the robot regarding people, as defined by the proposed paradigm, enables more meaningful human-robot interactions

Portable decision support for diagnosis of traumatic brain injury

Early detection and diagnosis of Traumatic Brain Injury (TBI) could reduce significantly the death rate and improve the quality of life of the people affected if emergency services are equipped with tools for TBI diagnosis at the place of the accident. This problem is addressed here by proposing a portable decision support system called EmerEEG, which is based on Quantitative Electroencephalography (qEEG). The contributions of the paper are the proposed system concept, architecture and decision support for TBI diagnosis. By the virtue of its easily operable mobile system, the proposed solution for emergency TBI diagnosis provides valuable decision support at a very early stage after an accident, thereby enabling a short response time in critical situations and better prospects for the people affected

Automatic EEG processing for the early diagnosis of traumatic brain injury

Traumatic Brain Injury (TBI) is recognized as an important cause of death and disabilities after an accident. The availability a tool for the early diagnosis of brain dysfunctions could greatly improve the quality of life of people affected by TBI and even prevent deaths. The contribution of the paper is a process including several methods for the automatic processing of electroencephalography (EEG) data, in order to provide a fast and reliable diagnosis of TBI. Integrated in a portable decision support system called EmerEEG, the TBI diagnosis is obtained using discriminant analysis based on quantitative EEG (qEEG) features extracted from data recordings after the automatic removal of artifacts. The proposed algorithm computes the TBI diagnosis on the basis of a model extracted from clinically-labelled EEG records. The system evaluations have confirmed the speed and reliability of the processing algorithms as well as the system's ability to deliver accurate diagnosis. The developed algorithms have achieved 79.1% accuracy in removing artifacts, and 87.85% accuracy in TBI diagnosis. Therefore, the developed system enables a short response time in emergency situations and provides a tool the emergency services could base their decision upon, thus preventing possibly miss-diagnosed injuries

Human detection and tracking in an assistive living service robot through multimodal data fusion

A new method is proposed for using a combination of measurements from a laser range finder and a depth camera in a data fusion process that benefits from each modality's strong side. The combination leads to a significantly improved performance of the human detection and tracking in comparison with what is achievable from the singular modalities. The useful information from both laser and depth camera is automatically extracted and combined in a Bayesian formulation that is estimated using a Markov Chain Monte Carlo (MCMC) sampling framework. The experiments show that this algorithm can track robustly multiple people in real world assistive robotics applications

Challenges for service robots operating in non-industrial environments

The concept of service robotics has grown considerably over the past two decades with many robots being used in non-industrial environments such homes, hospitals and airports. Many of these environments were never designed to have mobile service robots deployed within them. This paper describes some of the challenges that are faced and need to be overcome in order for robots to successfully work in non-industrial environments (specifically homes and hospitals). These include the problems caused by an environment not having been designed to be robot-friendly, the unstructured nature of the environment and finally the challenges presented by certain user populations who may have difficulties interacting with a robot

Integration of symbolic task planning into operations within an unstructured environment

To ensure a robot capable of robust task execution in unstructured environments, task planners need to have a high-level understanding of the nature of the world, reasoning for deliberate actions, and reacting to environment changes. Proposed is a practical task planning approach that seamlessly integrating deeper domain knowledge, real time perception and symbolic planning for robot operation. A higher degree of autonomy under unstructured environment will be endowed to the robot with the proposed approach