Predictive World Models from Real-World Partial Observations

Cognitive scientists believe adaptable intelligent agents like humans perform reasoning through learned causal mental simulations of agents and environments. The problem of learning such simulations is called predictive world modeling. Recently, reinforcement learning (RL) agents leveraging world models have achieved SOTA performance in game environments. However, understanding how to apply the world modeling approach in complex real-world environments relevant to mobile robots remains an open question. In this paper, we present a framework for learning a probabilistic predictive world model for real-world road environments. We implement the model using a hierarchical VAE (HVAE) capable of predicting a diverse set of fully observed plausible worlds from accumulated sensor observations. While prior HVAE methods require complete states as ground truth for learning, we present a novel sequential training method to allow HVAEs to learn to predict complete states from partially observed states only. We experimentally demonstrate accurate spatial structure prediction of deterministic regions achieving 96.21 IoU, and close the gap to perfect prediction by 62% for stochastic regions using the best prediction. By extending HVAEs to cases where complete ground truth states do not exist, we facilitate continual learning of spatial prediction as a step towards realizing explainable and comprehensive predictive world models for real-world mobile robotics applications. Code is available at https://github.com/robin-karlsson0/predictive-world-models.Comment: Accepted for IEEE MOST 202

ViCE: Visual Concept Embedding Discovery and Superpixelization

Recent self-supervised computer vision methods have demonstrated equal or better performance to supervised methods, opening for AI systems to learn visual representations from practically unlimited data. However, these methods are classification-based and thus ineffective for learning dense feature maps required for unsupervised semantic segmentation. This work presents a method to effectively learn dense semantically rich visual concept embeddings applicable to high-resolution images. We introduce superpixelization as a means to decompose images into a small set of visually coherent regions, allowing efficient learning of dense semantics by swapped prediction. The expressiveness of our dense embeddings is demonstrated by significantly improving the SOTA representation quality benchmarks on COCO (+16.27 mIoU) and Cityscapes (+19.24 mIoU) for both low- and high-resolution images

Learning to Predict Navigational Patterns from Partial Observations

Human beings cooperatively navigate rule-constrained environments by adhering to mutually known navigational patterns, which may be represented as directional pathways or road lanes. Inferring these navigational patterns from incompletely observed environments is required for intelligent mobile robots operating in unmapped locations. However, algorithmically defining these navigational patterns is nontrivial. This paper presents the first self-supervised learning (SSL) method for learning to infer navigational patterns in real-world environments from partial observations only. We explain how geometric data augmentation, predictive world modeling, and an information-theoretic regularizer enables our model to predict an unbiased local directional soft lane probability (DSLP) field in the limit of infinite data. We demonstrate how to infer global navigational patterns by fitting a maximum likelihood graph to the DSLP field. Experiments show that our SSL model outperforms two SOTA supervised lane graph prediction models on the nuScenes dataset. We propose our SSL method as a scalable and interpretable continual learning paradigm for navigation by perception. Code released upon publication.Comment: Under revie

Discrete chiral organic nanotubes by stacking pillar[5]arenes using covalent linkages

Owing to their unique one-dimensional hollow structures, organic nanotubes have been widely explored in recent years. Covalent organic nanotubes (CONs) can be prepared by stacking building blocks, such as macrocycles, through covalent linkages. However, because of the mismatched covalent connections, controllable synthesis of the discrete CONs with clear structures, such as sidewall and chirality, is a challenging target. In this work, by coupling two pillar[5]arenes through dynamic covalent bonds, thermodynamically stable discrete CONs with 5-fold symmetry are successfully prepared. Three different chiral CONs are separated, including homo-CONs, consisting of two enantiomers (pR, pR and pS, pS), and hetero-CON, consisting of the meso form (pR, pS). These CONs show negative allosteric binding affinities toward guest molecules, which are not observed in individual pillar[5]arenes

Real-time chirality transfer monitoring from statistically random to discrete homochiral nanotubes

Real time monitoring of chirality transfer processes is necessary to better understand their kinetic properties. Herein, we monitor an ideal chirality transfer process from a statistically random distribution to a diastereomerically pure assembly in real time. The chirality transfer is based on discrete trimeric tubular assemblies of planar chiral pillar[5]arenes, achieving the construction of diastereomerically pure trimers of pillar[5]arenes through synergistic effect of ion pairing between a racemic rim-differentiated pillar[5]arene pentaacid bearing five benzoic acids on one rim and five alkyl chains on the other, and an optically resolved pillar[5]arene decaamine bearing ten amines. When the decaamine is mixed with the pentaacid, the decaamine is sandwiched by two pentaacids through ten ion pairs, initially producing a statistically random mixture of a homochiral trimer and two heterochiral trimers. The heterochiral trimers gradually dissociate and reassemble into the homochiral trimers after unit flipping of the pentaacid, leading to chirality transfer from the decaamine and producing diastereomerically pure trimers

CPL on/off control of an assembled system by water soluble macrocyclic chiral sources with planar chirality

Herein, we report the synthesis and planar chiral properties of a pair of water-soluble cationic pillar[5]arenes with stereogenic carbons. Interestingly, although units of the molecules were rotatable, only one planar chiral diastereomer existed in water in both cases. As a new type of chiral source, these molecules transmitted chiral information from the planar chiral cavities to the assembly of a water-soluble extended π-conjugated compound, affording circularly polarized luminescence (CPL). The chirality transfer process and resulting CPL were extremely sensitive to the feed ratio of the chiral pillar[5]arenes owing to the combined action of their planar chirality, bulkiness, and strong binding properties. When a limited amount of chiral source was added, further assembly of the extended π-conjugated compound into helical fibers with CPL was triggered. Unexpectedly, larger amounts of chiral source destroyed the helical fiber assemblies, resulting in elimination of the chirality and CPL properties from the assembled structures