Acoustic event characterization for service robot using convolutional networks

This paper presents and discusses the creation of a sound event classification model using deep learning. In the design of service robots, it is necessary to include routines that improve the response of both the robot and the human being throughout the interaction. These types of tasks are critical when the robot is taking care of children, the elderly, or people in vulnerable situations. Certain dangerous situations are difficult to identify and assess by an autonomous system, and yet, the life of the users may depend on these robots. Acoustic signals correspond to events that can be detected at a great distance, are usually present in risky situations, and can be continuously sensed without incurring privacy risks. For the creation of the model, a customized database is structured with seven categories that allow to categorize a problem, and eventually allow the robot to provide the necessary help. These audio signals are processed to produce graphical representations consistent with human acoustic identification. These images are then used to train three convolutional models identified as high-performing in this type of problem. The three models are evaluated with specific metrics to identify the best-performing model. Finally, the results of this evaluation are discussed and analyzed

Collaborative Artificial Intelligence Development for Social Robots

The main aim of this doctoral thesis was to investigate on how to involve a community for collaborative artificial intelligence (AI) development of a social robot. The work was initiated by the author’s personal interest in developing the Sony AIBO robots that have been unavailable on the retail markets, however, user communities with special interests in these robots remained on the internet. At first, to attract people’s attention, the author developed three specific features for the robot. These consisted of teaching the robot 1) sound event recognition in order to react to environmental audio stimuli, 2) a method to detect the underlying surface under the robot, and 3) of how to recognize its own body states. As this AI development proved to be very challenging, the author decided to start a community project for artificial intelligence development. Community involvement has a long history in open-source software projects and some robotics companies tried to benefit from their userbase in product development. An active online community of Sony AIBO owners was approached to investigate factors to engage its members in the creative processes. For this purpose, 78 Sony AIBO owners were recruited online to fill a questionnaire and their data were analyzed with respect to age, gender, culture, length of ownership, user contribution, and model preference. The results revealed the motives to own these robots for many years and how these heavy users perceived their social robots after a long period in the robot acceptance phase. For example, female participants tended to have more emotional relation to their robots than male who had more technically oriented long-term engagement motivation. The user expectations were also explored by analyzing the answers to this questionnaire to discover the key needs of this user group. The results revealed that the most-wanted skills were the interaction with humans and the autonomous operation. The integration with the AI agents and Internet services was important, but the long-term memory and learning capabilities were not so relevant for the participants. The diverse preferences for robot skills led to creating a prioritized recommendation list to complement the design guidelines for social robots in the literature. In sum, the findings of this thesis showed that developing AI features for an outdated robot is possible but takes a lot of time and shared community efforts. To involve a specific community, one needs first to build up trust by working with and for the community. Also, the trust for the long-term endurance of the development project was found as a precondition for the community commitment. The discoveries of this thesis can be applied to similar types of collaborative AI developments in the future. There are significant contributions in this dissertation to robotics. First, the long-term robot usage was not studied on a years-long scale before and the most extended human-robot interactions analyzed test subjects for only a few months. A questionnaire investigated the robot owners with 1-10+ years-long ownership in this work and their attitude towards robot acceptance. The survey results helped to understand the viable strategies to engage users for a long time. Second, innovative ways were explored to involve online communities in robotics development. The past approaches introduced the community ideas and opinions into product design and innovation iterations. The community in this dissertation tested the developed AI engine, provided inputs for further development directions, created content for the actual AI and gave their feedback about product quality. These contributions advance the social robotics field

The State of Lifelong Learning in Service Robots: Current Bottlenecks in Object Perception and Manipulation

Service robots are appearing more and more in our daily life. The development of service robots combines multiple fields of research, from object perception to object manipulation. The state-of-the-art continues to improve to make a proper coupling between object perception and manipulation. This coupling is necessary for service robots not only to perform various tasks in a reasonable amount of time but also to continually adapt to new environments and safely interact with non-expert human users. Nowadays, robots are able to recognize various objects, and quickly plan a collision-free trajectory to grasp a target object in predefined settings. Besides, in most of the cases, there is a reliance on large amounts of training data. Therefore, the knowledge of such robots is fixed after the training phase, and any changes in the environment require complicated, time-consuming, and expensive robot re-programming by human experts. Therefore, these approaches are still too rigid for real-life applications in unstructured environments, where a significant portion of the environment is unknown and cannot be directly sensed or controlled. In such environments, no matter how extensive the training data used for batch learning, a robot will always face new objects. Therefore, apart from batch learning, the robot should be able to continually learn about new object categories and grasp affordances from very few training examples on-site. Moreover, apart from robot self-learning, non-expert users could interactively guide the process of experience acquisition by teaching new concepts, or by correcting insufficient or erroneous concepts. In this way, the robot will constantly learn how to help humans in everyday tasks by gaining more and more experiences without the need for re-programming