Holistic Temporal Situation Interpretation for Traffic Participant Prediction

For a profound understanding of traffic situations including a prediction of traf- fic participants’ future motion, behaviors and routes it is crucial to incorporate all available environmental observations. The presence of sensor noise and depen- dency uncertainties, the variety of available sensor data, the complexity of large traffic scenes and the large number of different estimation tasks with diverging requirements require a general method that gives a robust foundation for the de- velopment of estimation applications. In this work, a general description language, called Object-Oriented Factor Graph Modeling Language (OOFGML), is proposed, that unifies formulation of esti- mation tasks from the application-oriented problem description via the choice of variable and probability distribution representation through to the inference method definition in implementation. The different language properties are dis- cussed theoretically using abstract examples. The derivation of explicit application examples is shown for the automated driv- ing domain. A domain-specific ontology is defined which forms the basis for four exemplary applications covering the broad spectrum of estimation tasks in this domain: Basic temporal filtering, ego vehicle localization using advanced interpretations of perceived objects, road layout perception utilizing inter-object dependencies and finally highly integrated route, behavior and motion estima- tion to predict traffic participant’s future actions. All applications are evaluated as proof of concept and provide an example of how their class of estimation tasks can be represented using the proposed language. The language serves as a com- mon basis and opens a new field for further research towards holistic solutions for automated driving

BAT: Behavior-Aware Human-Like Trajectory Prediction for Autonomous Driving

The ability to accurately predict the trajectory of surrounding vehicles is a critical hurdle to overcome on the journey to fully autonomous vehicles. To address this challenge, we pioneer a novel behavior-aware trajectory prediction model (BAT) that incorporates insights and findings from traffic psychology, human behavior, and decision-making. Our model consists of behavior-aware, interaction-aware, priority-aware, and position-aware modules that perceive and understand the underlying interactions and account for uncertainty and variability in prediction, enabling higher-level learning and flexibility without rigid categorization of driving behavior. Importantly, this approach eliminates the need for manual labeling in the training process and addresses the challenges of non-continuous behavior labeling and the selection of appropriate time windows. We evaluate BAT's performance across the Next Generation Simulation (NGSIM), Highway Drone (HighD), Roundabout Drone (RounD), and Macao Connected Autonomous Driving (MoCAD) datasets, showcasing its superiority over prevailing state-of-the-art (SOTA) benchmarks in terms of prediction accuracy and efficiency. Remarkably, even when trained on reduced portions of the training data (25%), our model outperforms most of the baselines, demonstrating its robustness and efficiency in predicting vehicle trajectories, and the potential to reduce the amount of data required to train autonomous vehicles, especially in corner cases. In conclusion, the behavior-aware model represents a significant advancement in the development of autonomous vehicles capable of predicting trajectories with the same level of proficiency as human drivers. The project page is available at https://github.com/Petrichor625/BATraj-Behavior-aware-Model

Visual Preferences and Human Interactions with Shading and Electric Lighting Systems

Buildings in the United States are responsible for 40% of the primary energy use and 30% of carbon dioxide emissions. As awareness is being raised for the energy consumption and environmental impacts of buildings, it is not surprising that improving building performance has gained significant attention over the past years. Increasing the energy efficiency and reducing the emissions associated with buildings is possible through the use of high-performance building design and implementation of advanced building controls. Moreover, as part of the modern life style, people in developed countries spend most of their time inside the buildings. This fact necessitates consideration of two important requirements. First that energy saving achieved by efficiency methods in practice should not compromise occupants’ comfort. Second, energy impacts of building users and their indoor environment preferences should be taken into account at both design and operation phases. Therefore, understanding and modeling human-building interactions and their links to energy consumption and occupant satisfaction with the indoor environment is the main goal of this research. To this end and with a focus on the visual environment, systematic data collection from a large number of participants is undertaken and novel probabilistic modeling approaches are explored to provide new insights towards human-centered sustainable buildings. The specific research objectives of this thesis are: 1. Study human interactions with motorized roller shades and dimmable electric lights as well as human perception and satisfaction with the luminous environment in private offices with variable daylight and electric light conditions. 2. Develop a novel Bayesian approach to model the interrelated human interactions with window shades and electric lights. 3. Develop a Bayesian classification and inference modeling framework for occupants’ visual preferences in daylit perimeter offices. To this end, four identical private offices in a high performance building located in West Lafayette, IN were equipped with sensing network and online survey questionnaires to study almost 300 occupants during the two sets of field studies conducted for this thesis. The first field study extends the knowledge of human-building interactions to advanced building systems such as motorized roller shades and dimmable electric lights and reveals behavioral patterns enabled through side-by-side comparisons of different environmental controls and user interfaces ranging from fully automated to fully manual and from low to high levels of accessibility (wall switch, remote controller and graphical web interface). Results of the field study reveal: (a) occupational dynamics and human variables as two key features, in addition to environmental variables, in describing human-shading and -electric lighting interactions; (b) higher daylight utilization in offices with easy-to-access controls; (c) strong preference for customized indoor climate, along with a relationship between occupant perception of control and acceptability of a wider range of visual conditions. With the insights gained from the first field study, the research extends to exploit the resulted dataset as a basis for the development of a hierarchical Bayesian approach which is used, for the first time, to model human interactions with motorized roller shades and dimmable electric lights. Bayesian multivariate binary-choice logit models have been constructed to predict shade raising/lowering and electric light increasing actions while Bayesian regression models with built-in physical constraints to estimate the magnitude of shading and electric lighting actions. The proposed models, in their structure, account for (a) intermediate operating states of the systems; (b) interrelated operation of shades and lights; (c) personal characteristics and human attributes. Moreover, the developed human-building interaction modeling framework benefits from the advantages of the Bayesian formalism as it (a) provides a systematic approach to identify significant features in describing the human-building interactions; (b) incorporates prior beliefs about the systems; (c) captures the epistemic uncertainty, which is important when dealing with small-sized datasets, a ubiquitous issue in human data collection in actual buildings. The second field study was designed and conducted to collect data for occupants’ satisfaction with the visual environment when exposed to different combinations of daylight and electric light conditions, along with data from room sensors, shading and light dimming states. The resulted dataset is then used as a basis to model occupants’ visual preferences such as prefer darker, prefer brighter, or satisfied with current conditions. Bayesian multinomial logistic regression is augmented with Dirichlet process prior to encode within the model structure that occupants’ visual preferences are influenced by a combination of environmental and control state variables as well as individual visual characteristics. The latter is treated as a hidden random variable which is used to cluster occupants with similar visual preference characteristics and to determine the optimal number of clusters among the observed population. Modeling results based on observations from 75 occupants in glare-free conditions suggest work plane illuminance, window unshaded area, and electric light ratio as significant features of the general visual preference model and reveal the existence of three distinct clusters with physical interpretation; preference for bright, moderate, and dark conditions. In the final step, a method for learning the visual preferences of new occupants is deployed which uses a mixture of the general probabilistic sub-models to infer new occupants’ cluster values and personalized preference profiles. The proposed approach proves to be efficient as it is shown to predict personalized profiles with 81% prediction accuracy with very few observations (less than 16) from each new occupant. In summary, the systematic data collection methods and prototype interfaces used in this dissertation establish a consistent and reliable approach for studying human interactions with building systems and satisfaction with the indoor environment. Unique datasets for human attributes towards the visual environment in perimeter building zones have been generated especially for the occupants’ direct preference votes with different visual conditions which is currently lacked in the literature. The probabilistic models for human interactions with shading and lighting systems and occupants’ visual preferences incorporate individual characteristics and account for uncertainties associated with limited data, thus, are to increase prediction accuracy when implemented in Building Performance Simulation tools. The research presented herein facilitates an effective pathway towards implementation of adaptive personalized environments and is a necessary precursor for future investigation and expansion to human-centered building controls

An Overview about Emerging Technologies of Autonomous Driving

Since DARPA started Grand Challenges in 2004 and Urban Challenges in 2007, autonomous driving has been the most active field of AI applications. This paper gives an overview about technical aspects of autonomous driving technologies and open problems. We investigate the major fields of self-driving systems, such as perception, mapping and localization, prediction, planning and control, simulation, V2X and safety etc. Especially we elaborate on all these issues in a framework of data closed loop, a popular platform to solve the long tailed autonomous driving problems

Interactive Prediction and Planning for Autonomous Driving: from Algorithms to Fundamental Aspects

Inevitably, autonomous vehicles need to interact with other road participants in a variety of highly complex or critical driving scenarios. It is still an extremely challenging task even for the forefront companies or institutes to enable autonomous vehicles to interactively predict the behavior of others, and plan safe and high-quality motions accordingly. The major obstacles are not just originated from prediction and planning algorithms with insufficient performances. Several fundamental problems in the fields of interactive prediction and planning still remain open, such as formulation, representation and evaluation of interactive prediction methods, motion dataset with densely interactive driving behavior, as well as interface of interactive prediction and planning algorithms. The aforementioned fundamental aspects of interactive prediction and planning are addressed in this dissertation along with various kinds of algorithms. First, generic environmental representation for various scenarios with topological decomposition is constructed, and a corresponding planning algorithm is designed by combining graph search and optimization. Hard constraints in optimization-based planners are also incorporated into the training loss of imitation learning so that the policy net can generate safe and feasible motions in highly constrained scenarios. Unified problem formulation and motion representation are designed for different paradigms of interactive predictors such as planning-based prediction (inverse reinforcement learning), as well as probabilistic graphical models (hidden Markov model) and deep neural networks (mixture density network), which are utilized for the prediction/planning interface design and prediction benchmark. A framework combing decision network and graph-search/optimization/sample-based planner is proposed to achieve a driving strategy which is defensive to potential violations of others, but not overly conservatively to threats of low probabilities. Such driving strategy is achieved via experiments based on the aforementioned interactive prediction and planning algorithms with proper interface designed. These predictors are also evaluated from closed loop perspective considering planning fatality when using the prediction results instead of pure data approximation metrics. Finally, INTERACTION (INTERnational, Adversarial and Cooperative moTION) dataset with highly interactive driving scenarios and behavior from international locations is constructed with interaction density metric defined to compare different datasets. The dataset has been utilized for various behavior-related research areas such as prediction, planning, imitation learning and behavior modeling, and is inspiring new research fields such as representation learning, interaction extraction and scenario generation

Sensorimotor Representation Learning for an “Active Self” in Robots: A Model Survey

Safe human-robot interactions require robots to be able to learn how to behave appropriately in spaces populated by people and thus to cope with the challenges posed by our dynamic and unstructured environment, rather than being provided a rigid set of rules for operations. In humans, these capabilities are thought to be related to our ability to perceive our body in space, sensing the location of our limbs during movement, being aware of other objects and agents, and controlling our body parts to interact with them intentionally. Toward the next generation of robots with bio-inspired capacities, in this paper, we first review the developmental processes of underlying mechanisms of these abilities: The sensory representations of body schema, peripersonal space, and the active self in humans. Second, we provide a survey of robotics models of these sensory representations and robotics models of the self; and we compare these models with the human counterparts. Finally, we analyze what is missing from these robotics models and propose a theoretical computational framework, which aims to allow the emergence of the sense of self in artificial agents by developing sensory representations through self-exploration.Deutsche Forschungsgemeinschaft http://dx.doi.org/10.13039/501100001659Deutsche Forschungsgemeinschaft http://dx.doi.org/10.13039/501100001659Deutsche Forschungsgemeinschaft http://dx.doi.org/10.13039/501100001659Deutsche Forschungsgemeinschaft http://dx.doi.org/10.13039/501100001659Deutsche Forschungsgemeinschaft http://dx.doi.org/10.13039/501100001659Deutsche Forschungsgemeinschaft http://dx.doi.org/10.13039/501100001659Projekt DEALPeer Reviewe

Modifiability Of Strategy Use In Probabilistic Categorization By Rhesus Macaques (Macaca Mulatta) And Capuchin Monkeys (Cebus [Sapajus] Apella)

Humans and nonhuman animals categorize the natural world, and their behaviors can reveal how they use the stimulus information they encounter in service of these categorizations. Rigorous psychological study of categorization has offered many insights into the processes of categorization and their relative strengths and weaknesses across species. Probabilistic categorization, in which the relationships among stimulus information and category membership that are observed by an individual are fundamentally probabilistic, presents unique challenges both to the categorizer and to the psychologist attempting to model their behavior. Challenges notwithstanding, probabilistic categorization is an exceptionally ecologically relevant problem to human and nonhuman animal cognition alike. This dissertation reports the effects of many manipulations of theoretical interest on computer-trained rhesus macaques’ and capuchin monkeys’ inferred cognitive strategy use in a computerized version of a classic probabilistic categorization task. Experiment 1 probed cognitive strategy use across five variants of the same task in which the probability structure was constant, but the appearances and onscreen locations of cues and responses changed. Experiment 2 presented a series of manipulations of theoretical interest to the animals by changing the probability and reward structures of the task. Experiment 3 manipulated the stimuli of the task in ways motivated by findings across perceptual psychology literature. Experiment 4 extended the reward rate manipulations of Experiment 2 even further. Across four experiments, inferred strategy use was remarkably stable. Those animals that used cue-based strategies often returned to the same specific strategy experiment after experiment, as the cues, responses, probabilities, and contingencies changed around them. This finding is discussed in relation to questions of a real or functional ceiling on sophistication of strategy use, the robustness of cognitive individual differences in nonhuman primates, and future directions for comparative study of cognitive strategy use in probabilistic categorization