44 research outputs found
AI2-THOR: An Interactive 3D Environment for Visual AI
We introduce The House Of inteRactions (THOR), a framework for visual AI
research, available at http://ai2thor.allenai.org. AI2-THOR consists of near
photo-realistic 3D indoor scenes, where AI agents can navigate in the scenes
and interact with objects to perform tasks. AI2-THOR enables research in many
different domains including but not limited to deep reinforcement learning,
imitation learning, learning by interaction, planning, visual question
answering, unsupervised representation learning, object detection and
segmentation, and learning models of cognition. The goal of AI2-THOR is to
facilitate building visually intelligent models and push the research forward
in this domain
Vision-based deep execution monitoring
Execution monitor of high-level robot actions can be effectively improved by
visual monitoring the state of the world in terms of preconditions and
postconditions that hold before and after the execution of an action.
Furthermore a policy for searching where to look at, either for verifying the
relations that specify the pre and postconditions or to refocus in case of a
failure, can tremendously improve the robot execution in an uncharted
environment. It is now possible to strongly rely on visual perception in order
to make the assumption that the environment is observable, by the amazing
results of deep learning. In this work we present visual execution monitoring
for a robot executing tasks in an uncharted Lab environment. The execution
monitor interacts with the environment via a visual stream that uses two DCNN
for recognizing the objects the robot has to deal with and manipulate, and a
non-parametric Bayes estimation to discover the relations out of the DCNN
features. To recover from lack of focus and failures due to missed objects we
resort to visual search policies via deep reinforcement learning