Robustness of Utilizing Feedback in Embodied Visual Navigation

This paper presents a framework for training an agent to actively request help in object-goal navigation tasks, with feedback indicating the location of the target object in its field of view. To make the agent more robust in scenarios where a teacher may not always be available, the proposed training curriculum includes a mix of episodes with and without feedback. The results show that this approach improves the agent's performance, even in the absence of feedback.Comment: Accepted at the ICRA Workshop for Communicating Robot Learning across Human-Robot Interactio

A Survey of Embodied AI: From Simulators to Research Tasks

There has been an emerging paradigm shift from the era of "internet AI" to "embodied AI", where AI algorithms and agents no longer learn from datasets of images, videos or text curated primarily from the internet. Instead, they learn through interactions with their environments from an egocentric perception similar to humans. Consequently, there has been substantial growth in the demand for embodied AI simulators to support various embodied AI research tasks. This growing interest in embodied AI is beneficial to the greater pursuit of Artificial General Intelligence (AGI), but there has not been a contemporary and comprehensive survey of this field. This paper aims to provide an encyclopedic survey for the field of embodied AI, from its simulators to its research. By evaluating nine current embodied AI simulators with our proposed seven features, this paper aims to understand the simulators in their provision for use in embodied AI research and their limitations. Lastly, this paper surveys the three main research tasks in embodied AI -- visual exploration, visual navigation and embodied question answering (QA), covering the state-of-the-art approaches, evaluation metrics and datasets. Finally, with the new insights revealed through surveying the field, the paper will provide suggestions for simulator-for-task selections and recommendations for the future directions of the field.Comment: Under Review for IEEE TETC

Survey and design of embodied AI simulator for the research of generalizing task-planning in 3D environment via ActioNet

With the emerging paradigm shift from “internet AI” to “embodied AI”, AI algorithms and agents are no longer just learning from images, videos, or curated text-based datasets from the internet. Instead, learning has been through physical interactions with a dynamic environment, whether real or simulated. Hence, this project aims to further advance the research effort in embodied AI through its three different portions. The project first presented ActioNet, an interactive end-to-end platform for data collection and augmentation of a task-based dataset in a 3D environment. The ActioNet platform and dataset help facilitate the learning of hierarchical task planning for artificial agents in embodied AI simulators. Afterwhich, to further deepen the understanding of the field, the project proposed a survey of embodied AI from its simulators to research tasks. This survey paper is the first modern and extensive survey of this field. It provides a detailed benchmarking of nine modern embodied AI simulators and further introduced a pyramidal hierarchy that delves into the embodied AI research tasks while giving new insight into the field. Lastly, with the new insights and knowledge gained from the previous portions, the project further proposed SPECIAL, Simulator for Physics Enriched Conditions in Artificially synthesised environments for causal Learning. SPECIAL is a state-of-the-art embodied AI simulation framework that can synthesis three new research task datasets; containment, stability, and contact, which are all fundamental physical interaction. To my knowledge, the SPECIAL dataset is the largest complex physics scenario dataset, consisting of over 60k individual scene instances, with up to 8 million frames. The project also proposed and constructed a SPECIAL model to train AI systems to learn causal reasoning and intuitive physics in a virtual environment. The first portion of the project on ActioNet has been published in the International Conference on Image Processing (ICIP 2020), while the second portion of the project has been submitted to the Computer Vision and Image Understanding Journal. The dataset and results curated from the third portion of the project are also being used to prepare for submitting to the British Machine Vision Conference 2021. Notably, this project has been shortlisted as one of the top 7 finalists for the EEE FYP Challenge 2021.Bachelor of Engineering (Electrical and Electronic Engineering

ABCDE: An Agent-Based Cognitive Development Environment

Children's cognitive abilities are sometimes cited as AI benchmarks. How can the most common 1,000 concepts (89\% of everyday use) be learnt in a naturalistic children's setting? Cognitive development in children is about quality, and new concepts can be conveyed via simple examples. Our approach of knowledge scaffolding uses simple objects and actions to convey concepts, like how children are taught. We introduce ABCDE, an interactive 3D environment modeled after a typical playroom for children. It comes with 300+ unique 3D object assets (mostly toys), and a large action space for child and parent agents to interact with objects and each other. ABCDE is the first environment aimed at mimicking a naturalistic setting for cognitive development in children; no other environment focuses on high-level concept learning through learner-teacher interactions. The simulator can be found at https://pypi.org/project/ABCDESim/1.0.0/Comment: Accepted to CVPRW 2022,Embodied AI Workshop (Extended Abstract

A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories

Recent work in computer vision and cognitive reasoning has given rise to an increasing adoption of the Violation-of-Expectation (VoE) paradigm in synthetic datasets. Inspired by infant psychology, researchers are now evaluating a model’s ability to label scenes as either expected or surprising with knowledge of only expected scenes. However, existing VoE-based 3D datasets in physical reasoning provide mainly vision data with little to no heuristics or inductive biases. Cognitive models of physical reasoning reveal infants create high-level abstract representations of objects and interactions. Capitalizing on this knowledge, we established a benchmark to study physical reasoning by curating a novel large-scale synthetic 3D VoE dataset armed with ground-truth heuristic labels of causally relevant features and rules. To validate our dataset in five event categories of physical reasoning, we benchmarked and analyzed human performance. We also proposed the Object File Physical Reasoning Network (OFPR-Net) which exploits the dataset's novel heuristics to outperform our baseline and ablation models. The OFPR-Net is also flexible in learning an alternate physical reality, showcasing its ability to learn universal causal relationships in physical reasoning to create systems with better interpretability

Current status of diagnosis and treatment of pulmonary hypertension in Chinese tertiary hospitals: A nationwide survey

Abstract We intended to evaluate the diagnosis and treatment status of pulmonary hypertension (PH) in China and provide the basis for the design of the Chinese PH centers system. A questionnaire survey was conducted by sampling from Chinese Class A tertiary hospitals that have carried out the clinical work of PH, including the composition of PH clinical team, the current application of examinations related to PH diagnosis, the availability of PAH‐specific medicine and the implementation of PH‐related intervention and surgery. A total of 44 valid questionnaires from 20 provinces were collected in this survey. In the vast majority of centers (83.33%, n = 35), pulmonary artery catheterization was routinely performed under X‐ray guidance. In 19.05% (n = 8) of centers, pressure measurements were determined at the right time (the end of normal expiration). Only 73.81% (n = 31) centers have carried out acute vasoreactivity testing. Prostacyclin analogues and prostaglandin receptor agonists were just prescribed in 45.45% (n = 20) of the centers. 19 centers (43.18%) were capable of performing balloon pulmonary angioplasty (BPA) and pulmonary endarterectomy (PEA), while 25% (n = 11) were able to perform BPA, PEA, and lung transplantation. There was no significant difference in the diagnosis and treatment of PH between economic regions. The majority of Chinese tertiary hospitals were well equipped with the corresponding personnel, examinations and medicines related to PH, but the standardization and specialization of the management of PH need to be strengthened