Framework of active robot learning

A thesis submitted to the University of Bedfordshire, in fulfilment of the requirements for the degree of Master of Science by researchIn recent years, cognitive robots have become an attractive research area of Artificial Intelligent (AI). High-order beliefs for cognitive robots regard the robots' thought about their users' intention and preference. The existing approaches to the development of such beliefs through machine learning rely on particular social cues or specifically defined award functions . Therefore, their applications can be limited. This study carried out primary research on active robot learning (ARL) which facilitates a robot to develop high-order beliefs by actively collecting/discovering evidence it needs. The emphasis is on active learning, but not teaching. Hence, social cues and award functions are not necessary. In this study, the framework of ARL was developed. Fuzzy logic was employed in the framework for controlling robot and for identifying high-order beliefs. A simulation environment was set up where a human and a cognitive robot were modelled using MATLAB, and ARL was implemented through simulation. Simulations were also performed in this study where the human and the robot tried to jointly lift a stick and keep the stick level. The simulation results show that under the framework a robot is able to discover the evidence it needs to confirm its user's intention

Test moment determination design in active robot learning

A thesis submitted to the University of Bedfordshire, in fulfilment of the requirements for the degree of Master of Science by researchIn recent years, service robots have been increasingly used in people's daily live. These robots are autonomous or semiautonomous and are able to cooperate with their human users. Active robot learning (ARL) is an approach to the development of beliefs for the robots on their users' intention and preference, which is needed by the robots to facilitate the seamless cooperation with humans. This approach allows a robot to perform tests on its users and to build up the high-order beliefs according to the users' responses. This study carried out primary research on designing the test moment determination component in ARL framework. The test moment determination component is used to decide right moment of taking a test action. In this study, an action plan theory was suggested to synthesis actions into a sequence, that is, an action plan, for a given task. All actions are defined in a special format of precondition, action, post-condition and testing time. Forward chaining reasoning was introduced to establish connection between the actions and to synthesis individual actions into an action plan, corresponding to the given task. A simulation environment was set up where a human user and a service robot were modelled using MATLAB. Fuzzy control was employed for controlling the robot to carry out the cooperative action. In order to examine the effect of test moment determination component, simulations were performed to execute a scenario where a robot passes on an object to a human user. The simulation results show that an action plan can be formed according to provided conditions and executed by simulated models properly. Test actions were taken at the moment determined by the test moment determination component to find the human user's intention

The development of test action bank for active robot learning

A thesis submitted to the University of Bedfordshire, in fulfilment of the requirements for the degree of Master of Science by researchIn the rapidly expanding service robotics research area, interactions between robots and humans become increasingly cornmon as more and more jobs will require cooperation between the robots and their human users. It is important to address cooperation between a robot and its user. ARL is a promising approach which facilitates a robot to develop high-order beliefs by actively performing test actions in order to obtain its user's intention from his responses to the actions. Test actions are crucial to ARL. This study carried out primary research on developing a Test Action Bank (TAB) to provide test actions for ARL. In this study, a verb-based task classifier was developed to extract tasks from user's commands. Taught tasks and their corresponding test actions were proposed and stored in database to establish the TAB. A backward test actions retrieval method was used to locate a task in a task tree and retrieve its test actions from TAB. A simulation environment was set up with a service robot model and a user model to test TAB and demonstrate some test actions. Simulations were also perfonned in this study, the simulation results proved TAB can successfully provide test actions according to different tasks and the proposed service robot model can demonstrate test actions

Continual Active Robot Learning using Self-organizing Neural Network

학위논문 (석사) -- 서울대학교 대학원 : 공과대학 컴퓨터공학부, 2021. 2. 장병탁.이 논문에서는 인공지능 로봇이 실제 환경에 적응하면서 주변에서 접하는 대상의 개념을 지속적이고 능동적으로 학습하는 방법을 제안한다. 최근 딥러닝이 비약적으로 발전하면서 인공지능 가전, 스피커 등이 개발되고 있으나, 이런 제품 대부분은 일괄적으로 학습된 음성 인식이나 얼굴 인식 같은 기능을 이용하기 때문에 개별 동작 환경이 학습 환경과 다르다면 성능이 크게 저하될 수 있다. 또한, 여기에 활용되는 딥러닝 모델은 대량의 데이터로 오랜 시간 학습시켜야 하고 입력 순서에 따라 파괴적 망각이 나타날 수 있다는 한계가 있다. 인공지능 로봇은 새로 감지한 소수의 데이터를 계속 학습해 나가는 것이 필요하며, 이 연구에서는 이를 위해 사람의 학습 방식을 모사하는 데 초점을 맞추었다. 자기조직화 신경망, 온라인 준지도 능동 학습을 비롯하여 사람의 학습 방식을 모사한 기존 머신러닝 기법을 모델 구조와 학습 기제 측면에서 분석하고 이들의 장점을 종합할 수 있는 새로운 모델인 CARLSON을 개발했다. CARLSON은 로봇이 관측한 물체 이미지를 입력받아 물체 개념을 학습하며, 새로운 데이터를 기존 개념과 대조하면서 지식을 확장해 나가는 자기조직화 신경망 구조로 되어 있다. 물체 이미지는 차원이 높고 잡음을 포함하므로, 효율적이고 안정적인 학습을 위하여 이미지에서 핵심적인 표상을 우선 추출하도록 했다. 표상 추출은 모델의 인코더(encoder) 부분이 수행하며, 이는 표상을 이미지로 복원하는 디코더(decoder)와 함께 훈련된다. 인코더에서 추출된 표상들은 상호 유사도에 따라 여러 개념으로 나뉘고 각 개념은 대표 표상을 가지는 하나의 노드(node)로 군집화된다. 군집화 과정의 노드 추가와 조정은 적응 공명 이론(Grossberg 1987)에서처럼 하향식 예측과 상향식 활성화를 통해 이루어진다. 인코더와 디코더를 포함한 전체 모델은 데이터가 입력될 때마다 학습하며, 표지 전파 기법을 통해 유사한 노드 간에 정보를 전달하고 불확실한 개념에 대해서는 능동적 질의를 통해 정보를 보충함으로써 데이터가 적고 정답 표지가 드물 때도 효과적으로 학습할 수 있다. 이 연구에서는 실제 로봇에서 모델의 성능을 검증하기 위하여 휴머노이드 로봇인 NAO로 연속적인 물체 이미지를 수집하고 시각 객체 인식 실험을 수행했다. CARLSON은 일반적인 딥러닝 모델인 합성곱 신경망(CNN)보다 확연히 높은 분류 정확도를 보였으며, 데이터 수와 표지가 적고 각 데이터를 한 번씩만 학습할 수 있는 제약하에서도 안정적으로 동작하는 것을 검증할 수 있었다. 추가로 잘 알려진 숫자 및 물체 인식 데이터셋인 MNIST, SVHN, Fashion-MNIST, CIFAR-10에서 온라인 준지도 학습 시나리오를 설정하고 모델을 시험했으며, 마찬가지로 CARLSON이 CNN보다 높은 성능을 보이는 것을 확인했다.In this thesis, a continual and active machine learning method is proposed to make artificial intelligence (AI) robots adapt to real environments and form concepts of nearby objects. Recent advances in the field of AI have led to the development of smart home appliances or AI speakers, but most of these products may suffer performance degradation in actual use. This is because they use functions such as voice or face recognition without adjusting them to the individual operation environments. The deep learning techniques used for these functions need to be trained repeatedly with big data for a long time, and they have a risk of catastrophic forgetting when encountering increasingly diverse objects. Meanwhile, AI robots need to continuously learn skills and concepts from a relatively small number of newly acquired data. Since humans are the most well-known agents that learn this way, imitating human learning would be one of the most effective ways to achieve the desired robot learning. The proposed model, CARLSON, integrates the strengths of the previous human-like machine learning methods. CARLSON is a self-organizing neural network that can expand the knowledge by comparing the incoming object image to the learned concepts. In order to increase the efficiency and stability of learning, the model first reduces the size and noise of high-dimensional input images by extracting informative features, or representations, from them. The feature extraction is carried out by an encoder which is jointly trained with a decoder that reconstructs images from representations. CARLSON divides the representations into groups in such a way that each group represents a single kind of objects, or an individual concept. The groups are implemented as nodes with means and variances that are created or adjusted by considering both top-down prediction and bottom-up activation as in Adaptive Resonance Theory (Grossberg 1987). The whole model including the encoder and decoder is trained in an end-to-end manner, and updated upon every new input. Using a label propagation method, CARLSON makes the similar nodes share information so that it can infer the object categories even when the labels are provided rarely. It can also actively ask a human operator about uncertain concepts to further make up for insufficient information. To evaluate the model, a visual object dataset was constructed by collecting images with a humanoid robot NAO, and was used for object recognition experiments. CARLSON clearly outperformed a convolutional neural network (CNN) model and showed a stable performance even when the labels were given rarely and each data could be accessed only once during training. It also performed better than CNN in online semi-supervised recognition tasks using well-known digit and object classification datasets: MNIST, SVHN, Fashion-MNIST, and CIFAR-10.제 1장 서 론 1 제 2장 사람의 학습 방식을 모사한 머신러닝 5 2.1. 자기조직화 신경망 5 2.1.1. 승자 독식과 k-평균 군집화 6 2.1.2. 자기조직화 지도 6 2.1.3. 뉴럴 가스 네트워크 7 2.2. 효율적인 학습 기제 9 2.2.1. 데이터가 적을 때의 학습 방법 9 2.2.2. 표지가 적을 때의 학습 방법 11 2.3. 적응 공명 이론을 통한 지속 학습 12 제 3장 지속적 능동 학습 자기조직화 신경망 15 3.1. 표상 추출과 데이터 재생성 15 3.2. 가우시안 군집화와 노드 간 정보 전달 16 3.2.1. 개념 생성과 조정 17 3.2.2. 군집 특성을 활용한 인코더 학습 18 3.2.3. 노드 간 표지 정보 전파 21 3.3. 데이터 생성을 통한 능동 학습 22 3.4. CARLSON 학습 및 추론 23 제 4장 일반화된 CARLSON 모델 25 4.1. 유기적 개념 형성 26 4.1.1. 명시적 개념 조정 27 4.1.2. 개념 병합 및 세분화 27 4.2. 계층적 표상화 28 4.3. 능동적 데이터 질의 29 제 5장 시각 객체 인식 실험 31 5.1. 시각 객체 데이터셋 31 5.1.1. 로봇을 이용한 데이터 수집 31 5.1.2. 숫자 및 물체 인식 데이터셋 33 5.2. 온라인 준지도 학습 실험 33 5.2.1. 모델 구현 상세 33 5.2.2. 실험 설정 34 5.2.3. 실험 결과 및 논의 35 제 6장 결 론 36 참고 문헌 37 Abstract 43Maste

Multimodal Hierarchical Dirichlet Process-based Active Perception

In this paper, we propose an active perception method for recognizing object categories based on the multimodal hierarchical Dirichlet process (MHDP). The MHDP enables a robot to form object categories using multimodal information, e.g., visual, auditory, and haptic information, which can be observed by performing actions on an object. However, performing many actions on a target object requires a long time. In a real-time scenario, i.e., when the time is limited, the robot has to determine the set of actions that is most effective for recognizing a target object. We propose an MHDP-based active perception method that uses the information gain (IG) maximization criterion and lazy greedy algorithm. We show that the IG maximization criterion is optimal in the sense that the criterion is equivalent to a minimization of the expected Kullback--Leibler divergence between a final recognition state and the recognition state after the next set of actions. However, a straightforward calculation of IG is practically impossible. Therefore, we derive an efficient Monte Carlo approximation method for IG by making use of a property of the MHDP. We also show that the IG has submodular and non-decreasing properties as a set function because of the structure of the graphical model of the MHDP. Therefore, the IG maximization problem is reduced to a submodular maximization problem. This means that greedy and lazy greedy algorithms are effective and have a theoretical justification for their performance. We conducted an experiment using an upper-torso humanoid robot and a second one using synthetic data. The experimental results show that the method enables the robot to select a set of actions that allow it to recognize target objects quickly and accurately. The results support our theoretical outcomes.Comment: submitte

A Dataset of Anatomical Environments for Medical Robots: Modeling Respiratory Deformation

Anatomical models of a medical robot's environment can significantly help guide design and development of a new robotic system. These models can be used for benchmarking motion planning algorithms, evaluating controllers, optimizing mechanical design choices, simulating procedures, and even as resources for data generation. Currently, the time-consuming task of generating these environments is repeatedly performed by individual research groups and rarely shared broadly. This not only leads to redundant efforts, but also makes it challenging to compare systems and algorithms accurately. In this work, we present a collection of clinically-relevant anatomical environments for medical robots operating in the lungs. Since anatomical deformation is a fundamental challenge for medical robots operating in the lungs, we describe a way to model respiratory deformation in these environments using patient-derived data. We share the environments and deformation data publicly by adding them to the Medical Robotics Anatomical Dataset (Med-RAD), our public dataset of anatomical environments for medical robots

Human-Machine Collaborative Optimization via Apprenticeship Scheduling

Coordinating agents to complete a set of tasks with intercoupled temporal and resource constraints is computationally challenging, yet human domain experts can solve these difficult scheduling problems using paradigms learned through years of apprenticeship. A process for manually codifying this domain knowledge within a computational framework is necessary to scale beyond the ``single-expert, single-trainee" apprenticeship model. However, human domain experts often have difficulty describing their decision-making processes, causing the codification of this knowledge to become laborious. We propose a new approach for capturing domain-expert heuristics through a pairwise ranking formulation. Our approach is model-free and does not require enumerating or iterating through a large state space. We empirically demonstrate that this approach accurately learns multifaceted heuristics on a synthetic data set incorporating job-shop scheduling and vehicle routing problems, as well as on two real-world data sets consisting of demonstrations of experts solving a weapon-to-target assignment problem and a hospital resource allocation problem. We also demonstrate that policies learned from human scheduling demonstration via apprenticeship learning can substantially improve the efficiency of a branch-and-bound search for an optimal schedule. We employ this human-machine collaborative optimization technique on a variant of the weapon-to-target assignment problem. We demonstrate that this technique generates solutions substantially superior to those produced by human domain experts at a rate up to 9.5 times faster than an optimization approach and can be applied to optimally solve problems twice as complex as those solved by a human demonstrator.Comment: Portions of this paper were published in the Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) in 2016 and in the Proceedings of Robotics: Science and Systems (RSS) in 2016. The paper consists of 50 pages with 11 figures and 4 table