Rescan: Inductive Instance Segmentation for Indoor RGBD Scans

In depth-sensing applications ranging from home robotics to AR/VR, it will be common to acquire 3D scans of interior spaces repeatedly at sparse time intervals (e.g., as part of regular daily use). We propose an algorithm that analyzes these "rescans" to infer a temporal model of a scene with semantic instance information. Our algorithm operates inductively by using the temporal model resulting from past observations to infer an instance segmentation of a new scan, which is then used to update the temporal model. The model contains object instance associations across time and thus can be used to track individual objects, even though there are only sparse observations. During experiments with a new benchmark for the new task, our algorithm outperforms alternate approaches based on state-of-the-art networks for semantic instance segmentation.Comment: IEEE International Conference on Computer Vision 201

Entraining a Robot to its Environment with an Artificial Circadian System

As robots become persistent agents in a complex and dynamic world, they must deal with changing environments. This challenge has grown research in long-term autonomy, with particular focus on localization and mapping in dynamic environments. Less attention has been paid to learning these dynamics to adapt or entrain an agent’s behavior to them. Inspired by circadian systems in nature, this dissertation seeks to answer how a robotic agent can both learn the regular cycles and patterns that exist in many environments, and how it can exploit that knowledge. In this work, relevant environmental states are modeled as time series and forecasted into the future. These forecasts are used to estimate the utility of executing some behavior at any point in time during the dominant environmental cycle. The relative utility of executing a behavior at the current time, compared to the potential utility for waiting until later, is passed as one component in an activation-based action selection system. While other components still impact what behaviors execute when, the ‘circadian’ component derived from forecasts biases the behavior to execute at the better times in the environmental cycle. This approach was dubbed the artificial circadian system. As forecasting the future is inherently unreliable, methods to make the approach robust to degrading forecast accuracy are presented. A unitless error measure is used to adapt the weight of the forecasting component in action selection, allowing an autonomous agent to leverage forecasts when they are good, and fall back onto reactive strategies when forecasts fail. As time series models rely on the history for predictions, a temporary disruption to the environment can potentially degrade forecast accuracy for many cycles. Methods to create modified forecasts which exclude potential outlier data are presented, and these forecasts are leveraged only if they improve accuracy. The ideas in this work were validated using an experimental test bed designed to approximate a precision agricultural task, where a solar powered robot monitored individual plants for pests and weeds. The artificial circadian system was shown to effectively entrain the agent’s behavior to the dynamics of its environment in some cases, improving performance. It was also noted that dynamics not on the same time-scale as the robot’s actions could not be exploited, even with forecasted knowledge. The artificial circadian system was reliably able to detect deviations in the environment and remove the influence of forecasts when their accuracy degraded. Successfully removing outlier data to generate better forecasts was less consistent, but achieved in a significant portion of trials.Ph.D