Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Symphonize 3D Semantic Scene Completion with Contextual Instance Queries

3D Semantic Scene Completion (SSC) has emerged as a nascent and pivotal task for autonomous driving, as it involves predicting per-voxel occupancy within a 3D scene from partial LiDAR or image inputs. Existing methods primarily focus on the voxel-wise feature aggregation, while neglecting the instance-centric semantics and broader context. In this paper, we present a novel paradigm termed Symphonies (Scene-from-Insts) for SSC, which completes the scene volume from a sparse set of instance queries derived from the input with context awareness. By incorporating the queries as the instance feature representations within the scene, Symphonies dynamically encodes the instance-centric semantics to interact with the image and volume features while avoiding the dense voxel-wise modeling. Simultaneously, it orchestrates a more comprehensive understanding of the scenario by capturing context throughout the entire scene, contributing to alleviating the geometric ambiguity derived from occlusion and perspective errors. Symphonies achieves a state-of-the-art result of 13.02 mIoU on the challenging SemanticKITTI dataset, outperforming existing methods and showcasing the promising advancements of the paradigm. The code is available at \url{https://github.com/hustvl/Symphonies}.Comment: Technical report. Code and models at: https://github.com/hustvl/Symphonie

A Framework for Augmenting Building Performance Models Using Machine Learning and Immersive Virtual Environment

Building performance models (BPMs), such as building energy simulation models, have been widely used in building design. Existing BPMs are mainly derived using data from existing buildings. They may not be able to effectively address human-building interactions and lack the capability to address specific contextual factors in buildings under design. The lack of such capability often contributes to the existence of building performance discrepancies, i.e., differences between predicted performance during design and the actual performance. To improve the prediction accuracy of existing BPMs, a computational framework is developed in this dissertation. It combines an existing BPM with context-aware design-specific data involving human-building interactions in new designs by using a machine learning approach. Immersive virtual environments (IVEs) are used to acquire data describing design-specific human-building interactions, a machine learning technique is used to combine data obtained from an existing BPM, and IVEs are used to generate an augmented BPM. The potential of the framework is investigated and evaluated. An artificial neural network (ANN)-based greedy algorithm combines context-aware design-specific data obtained from IVEs with an existing BPM to enhance the simulations of human-building interactions in new designs. The results of the application show the potential of the framework to improve the prediction accuracy of an existing BPM evaluated against data obtained from the physical environment. However, it lacks the ability to determine the appropriate combination between context-aware design-specific data and data of the existing BPM. Consequently, the framework is improved to have ability to determine an appropriate combination based on a specified performance target. A generative adversarial network (GAN) is used to combine context-aware design-specific data and data of an existing BPM using the performance target as guide to generate an augmented BPM. The results confirm the effectiveness of this new framework. The performance of the augmented BPMs generated using the GAN-based framework is significantly better than the updated BPMs generated using the ANN-based greedy algorithm. The framework is completed by incorporating a robustness analysis to assist investigations of robustness of the GAN regarding the uncertainty involved in the input parameters (i.e., an existing BPM and context-aware design-specific data). Overall, this dissertation shows the promising potential of the framework in enhancing performance of BPMs and reducing performance discrepancies between estimations made during design and in performance in actual buildings