Predictive World Models from Real-World Partial Observations

Cognitive scientists believe adaptable intelligent agents like humans perform reasoning through learned causal mental simulations of agents and environments. The problem of learning such simulations is called predictive world modeling. Recently, reinforcement learning (RL) agents leveraging world models have achieved SOTA performance in game environments. However, understanding how to apply the world modeling approach in complex real-world environments relevant to mobile robots remains an open question. In this paper, we present a framework for learning a probabilistic predictive world model for real-world road environments. We implement the model using a hierarchical VAE (HVAE) capable of predicting a diverse set of fully observed plausible worlds from accumulated sensor observations. While prior HVAE methods require complete states as ground truth for learning, we present a novel sequential training method to allow HVAEs to learn to predict complete states from partially observed states only. We experimentally demonstrate accurate spatial structure prediction of deterministic regions achieving 96.21 IoU, and close the gap to perfect prediction by 62% for stochastic regions using the best prediction. By extending HVAEs to cases where complete ground truth states do not exist, we facilitate continual learning of spatial prediction as a step towards realizing explainable and comprehensive predictive world models for real-world mobile robotics applications. Code is available at https://github.com/robin-karlsson0/predictive-world-models.Comment: Accepted for IEEE MOST 202

ViCE: Visual Concept Embedding Discovery and Superpixelization

Recent self-supervised computer vision methods have demonstrated equal or better performance to supervised methods, opening for AI systems to learn visual representations from practically unlimited data. However, these methods are classification-based and thus ineffective for learning dense feature maps required for unsupervised semantic segmentation. This work presents a method to effectively learn dense semantically rich visual concept embeddings applicable to high-resolution images. We introduce superpixelization as a means to decompose images into a small set of visually coherent regions, allowing efficient learning of dense semantics by swapped prediction. The expressiveness of our dense embeddings is demonstrated by significantly improving the SOTA representation quality benchmarks on COCO (+16.27 mIoU) and Cityscapes (+19.24 mIoU) for both low- and high-resolution images

Learning to Predict Navigational Patterns from Partial Observations

Human beings cooperatively navigate rule-constrained environments by adhering to mutually known navigational patterns, which may be represented as directional pathways or road lanes. Inferring these navigational patterns from incompletely observed environments is required for intelligent mobile robots operating in unmapped locations. However, algorithmically defining these navigational patterns is nontrivial. This paper presents the first self-supervised learning (SSL) method for learning to infer navigational patterns in real-world environments from partial observations only. We explain how geometric data augmentation, predictive world modeling, and an information-theoretic regularizer enables our model to predict an unbiased local directional soft lane probability (DSLP) field in the limit of infinite data. We demonstrate how to infer global navigational patterns by fitting a maximum likelihood graph to the DSLP field. Experiments show that our SSL model outperforms two SOTA supervised lane graph prediction models on the nuScenes dataset. We propose our SSL method as a scalable and interpretable continual learning paradigm for navigation by perception. Code released upon publication.Comment: Under revie

Learning Large Causal Structures from Inverse Covariance Matrix via Matrix Decomposition

Learning causal structures from observational data is a fundamental yet highly complex problem when the number of variables is large. In this paper, we start from linear structural equation models (SEMs) and investigate ways of learning causal structures from the inverse covariance matrix. The proposed method, called $\mathcal{O}$-ICID (for {\it Independence-preserving} Decomposition from Oracle Inverse Covariance matrix), is based on continuous optimization of a type of matrix decomposition that preserves the nonzero patterns of the inverse covariance matrix. We show that $\mathcal{O}$-ICID provides an efficient way for identifying the true directed acyclic graph (DAG) under the knowledge of noise variances. With weaker prior information, the proposed method gives directed graph solutions that are useful for making more refined causal discovery. The proposed method enjoys a low complexity when the true DAG has bounded node degrees, as reflected by its time efficiency in experiments in comparison with state-of-the-art algorithms

Photoinduced electron transport in dye-containing titania gel films

Amorphous dye-containing titania gel films were prepared on ITO electrodes coated with a crystalline titania foundation from titanium alkoxide sols containing a dye at room temperature. Photoinduced electron transport in the amorphous titania gel film was investigated by spectroscopic and photovoltaic measurements. Influences of the structure and morphology of the multilayered film on the photoelectron transport and electrically conductive properties were discussed. The photocurrent was observed from only the layer contacting the crystalline titania foundation. The electron transport from the amorphous upper layers was limited. Steam treatment of the electrodes improved the electron transport due to crystallization of the amorphous titania to anatase accompanied by enhancement of its electrical conductivity. The efficiency of the dye-sensitized electron transport in the steam-treated titania film was close to that of the anatase film prepared by heating at 773 K. The dye-containing titania layers functioned as efficient sensitizers.ArticleRSC ADVANCES. 2(10):4258-4267 (2012)journal articl

Development of a separable search-and-rescue robot composed of a mobile robot and a snake robot

In this study, we propose a new robot system consisting of a mobile robot and a snake robot. The system works not only as a mobile manipulator but also as a multi-agent system by using the snake robot's ability to separate from the mobile robot. Initially, the snake robot is mounted on the mobile robot in the carrying mode. When an operator uses the snake robot as a manipulator, the robot changes to the manipulator mode. The operator can detach the snake robot from the mobile robot and command the snake robot to conduct lateral rolling motions. In this paper, we present the details of our robot and its performance in the World Robot Summit