Work Zone Safety: Virtual Reality-Based Traffic Co-simulation Platform for Workforce Training and Pedestrian Behavior Analysis

First, an assessment was conducted to evaluate the suitability of VR in replicating reality and to compare worker behaviors in real-world and virtual environments. Second, an existing VR-based traffic co- simulation platform was enhanced by incorporating realistic worker and pedestrian behavior. This improvement allows for more accurate simulations of movement, including restrictions, tasks, and random movements. The platform also enables users to modify variables to adjust worker behavior. Finally, VR-based training modules were implemented and evaluated against traditional methods to enhance worker preparedness for hazards

A Testing and Experimenting Environment for Microscopic Traffic Simulation Utilizing Virtual Reality and Augmented Reality

Microscopic traffic simulation (MTS) is the emulation of real-world traffic movements in a virtual environment with various traffic entities. Typically, the movements of the vehicles in MTS follow some predefined algorithms, e.g., car-following models, lane changing models, etc. Moreover, existing MTS models only provide a limited capability of two- and/or three-dimensional displays that often restrict the user’s viewpoint to a flat screen. Their downscaled scenes neither provide a realistic representation of the environment nor allow different users to simultaneously experience or interact with the simulation model from different perspectives. These limitations neither allow the traffic engineers to effectively disseminate their ideas to various stakeholders of different backgrounds nor allow the analysts to have realistic data about the vehicle or pedestrian movements. This dissertation intends to alleviate those issues by creating a framework and a prototype for a testing environment where MTS can have inputs from user-controlled vehicles and pedestrians to improve their traffic entity movement algorithms as well as have an immersive M3 (multi-mode, multi-perspective, multi-user) visualization of the simulation using Virtual Reality (VR) and Augmented Reality (AR) technologies. VR environments are created using highly realistic 3D models and environments. With modern game engines and hardware available on the market, these VR applications can provide a highly realistic and immersive experience for a user. Different experiments performed by real users in this study prove that utilizing VR technology for different traffic related experiments generated much more favorable results than the traditional displays. Moreover, using AR technologies for pedestrian studies is a novel approach that allows a user to walk in the real world and the simulation world at a one-to-one scale. This capability opens a whole new avenue of user experiment possibilities. On top of that, the in-environment communication chat system will allow researchers to perform different Advanced Driver Assistance System (ADAS) studies without ever needing to leave the simulation environment. Last but not least, the distributed nature of the framework enables users to participate from different geographic locations with their choice of display device (desktop, smartphone, VR, or AR). The prototype developed for this dissertation is readily available on a test webpage, and a user can easily download the prototype application without needing to install anything. The user also can run the remote MTS server and then connect their client application to the server

Control and communication systems for automated vehicles cooperation and coordination

Mención Internacional en el título de doctorThe technological advances in the Intelligent Transportation Systems (ITS) are exponentially improving over the last century. The objective is to provide intelligent and innovative services for the different modes of transportation, towards a better, safer, coordinated and smarter transport networks. The Intelligent Transportation Systems (ITS) focus is divided into two main categories; the first is to improve existing components of the transport networks, while the second is to develop intelligent vehicles which facilitate the transportation process. Different research efforts have been exerted to tackle various aspects in the fields of the automated vehicles. Accordingly, this thesis is addressing the problem of multiple automated vehicles cooperation and coordination. At first, 3DCoAutoSim driving simulator was developed in Unity game engine and connected to Robot Operating System (ROS) framework and Simulation of Urban Mobility (SUMO). 3DCoAutoSim is an abbreviation for "3D Simulator for Cooperative Advanced Driver Assistance Systems (ADAS) and Automated Vehicles Simulator". 3DCoAutoSim was tested under different circumstances and conditions, afterward, it was validated through carrying-out several controlled experiments and compare the results against their counter reality experiments. The obtained results showed the efficiency of the simulator to handle different situations, emulating real world vehicles. Next is the development of the iCab platforms, which is an abbreviation for "Intelligent Campus Automobile". The platforms are two electric golf-carts that were modified mechanically, electronically and electrically towards the goal of automated driving. Each iCab was equipped with several on-board embedded computers, perception sensors and auxiliary devices, in order to execute the necessary actions for self-driving. Moreover, the platforms are capable of several Vehicle-to-Everything (V2X) communication schemes, applying three layers of control, utilizing cooperation architecture for platooning, executing localization systems, mapping systems, perception systems, and finally several planning systems. Hundreds of experiments were carried-out for the validation of each system in the iCab platform. Results proved the functionality of the platform to self-drive from one point to another with minimal human intervention.Los avances tecnológicos en Sistemas Inteligentes de Transporte (ITS) han crecido de forma exponencial durante el último siglo. El objetivo de estos avances es el de proveer de sistemas innovadores e inteligentes para ser aplicados a los diferentes medios de transporte, con el fin de conseguir un transporte mas eficiente, seguro, coordinado e inteligente. El foco de los ITS se divide principalmente en dos categorías; la primera es la mejora de los componentes ya existentes en las redes de transporte, mientras que la segunda es la de desarrollar vehículos inteligentes que hagan más fácil y eficiente el transporte. Diferentes esfuerzos de investigación se han llevado a cabo con el fin de solucionar los numerosos aspectos asociados con la conducción autónoma. Esta tesis propone una solución para la cooperación y coordinación de múltiples vehículos. Para ello, en primer lugar se desarrolló un simulador (3DCoAutoSim) de conducción basado en el motor de juegos Unity, conectado al framework Robot Operating System (ROS) y al simulador Simulation of Urban Mobility (SUMO). 3DCoAutoSim ha sido probado en diferentes condiciones y circunstancias, para posteriormente validarlo con resultados a través de varios experimentos reales controlados. Los resultados obtenidos mostraron la eficiencia del simulador para manejar diferentes situaciones, emulando los vehículos en el mundo real. En segundo lugar, se desarrolló la plataforma de investigación Intelligent Campus Automobile (iCab), que consiste en dos carritos eléctricos de golf, que fueron modificados eléctrica, mecánica y electrónicamente para darle capacidades autónomas. Cada iCab se equipó con diferentes computadoras embebidas, sensores de percepción y unidades auxiliares, con la finalidad de transformarlos en vehículos autónomos. Además, se les han dado capacidad de comunicación multimodal (V2X), se les han aplicado tres capas de control, incorporando una arquitectura de cooperación para operación en modo tren, diferentes esquemas de localización, mapeado, percepción y planificación de rutas. Innumerables experimentos han sido realizados para validar cada uno de los diferentes sistemas incorporados. Los resultados prueban la funcionalidad de esta plataforma para realizar conducción autónoma y cooperativa con mínima intervención humana.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Francisco Javier Otamendi Fernández de la Puebla.- Secretario: Hanno Hildmann.- Vocal: Pietro Cerr

Mobile Delivery Robots: Mixed Reality-Based Simulation Relying on ROS and Unity 3D

In the context of Intelligent Transportation Systems and the delivery of goods, new technology approaches need to be developed in order to cope with certain challenges that last mile delivery entails, such as navigation in an urban environment. Autonomous delivery robots can help overcome these challenges. We propose a method for performing mixed reality (MR) simulation with ROS-based robots using Unity, which synchronizes the real and virtual environment, and simultaneously uses the sensor information of the real robots to locate themselves and project them into the virtual environment, so that they can use their virtual doppelganger to perceive the virtual world. Using this method, real and virtual robots can perceive each other and the environment in which the other party is located, thereby enabling the exchange of information between virtual and real objects. Through this approach a more realistic and reliable simulation can be obtained. Results of the demonstrated use-cases verified the feasibility and efficiency as well as the stability of implementing MR using Unity for Robot Operating System (ROS)-based robots. © 2020 IEEE.This work was supported by the Austrian Ministry for Climate Action, Environment, Energy, Mobility, Innovation and Technology (BMK) Endowed Professorship for Sustainable Transport Logistics 4.0

A game theory-based approach for modeling autonomous vehicle behavior in congested, urban lane-changing scenarios

Autonomous vehicles are expected to display human-like behavior, at least to the extent that their decisions can be intuitively understood by other road users. If this is not the case, the coexistence of manual and autonomous vehicles in a mixed environment might affect road user interactions negatively and might jeopardize road safety. To this end, it is highly important to design algorithms that are capable of analyzing human decision-making processes and of reproducing them. In this context, lane-change maneuvers have been studied extensively. However, not all potential scenarios have been considered, since most works have focused on highway rather than urban scenarios. We contribute to the field of research by investigating a particular urban traffic scenario in which an autonomous vehicle needs to determine the level of cooperation of the vehicles in the adjacent lane in order to proceed with a lane change. To this end, we present a game theory-based decision-making model for lane changing in congested urban intersections. The model takes as input driving-related parameters related to vehicles in the intersection before they come to a complete stop. We validated the model by relying on the Co-AutoSim simulator. We compared the prediction model outcomes with actual participant decisions, i.e., whether they allowed the autonomous vehicle to drive in front of them. The results are promising, with the prediction accuracy being 100% in all of the cases in which the participants allowed the lane change and 83.3% in the other cases. The false predictions were due to delays in resuming driving after the traffic light turned green. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.This work was supported by the Austrian Ministry for Climate Action, Environment, Energy, Mobility, Innovation, and Technology (BMK) Endowed Professorship for Sustainable Transport Logistics 4.0., IAV France S.A.S.U., IAV GmbH, Austrian Post AG, and the UAS Technikum Wien. It was additionally supported by the Zero Emission Roll-Out?Cold Chain Distribution_877493

Serious Game Engineering and Lighting Models for the Realistic Emulation of 5G Systems

[ES] La quinta generación de comunicaciones móviles, 5G, promete ser una revolución tecnológica que vaya más allá de multiplicar la velocidad de transmisión de datos de sus predecesoras. Pretende soportar una gran cantidad de dispositivos y alcanzar latencias muy cercanas a 1 milisegundo. Para satisfacer estos ambiciosos requisitos, se han investigado nuevas tecnologías habilitadoras. Una de ellas es el uso de las bandas de ondas milimétricas (mmW) en las cuales hay una gran cantidad de espectro disponible. Para predecir las características del canal radio y evaluar las prestaciones de la 5G de forma fiable en las bandas mmW se requieren modelos de canal complejos. Concretamente, los modelos de propagación más precisos son los basados en trazado de rayos, pero su alto costo computacional los hacen inviables para la caracterización del canal radio en escenarios complejos. Por otro lado, en los últimos años, la tecnología de videojuegos ha desarrollado potentes herramientas para modelar la propagación de la luz en escenarios superrealistas. Dada la cercanía espectral entre el espectro visible y las ondas mmW, la presente Tesis ha estudiado la aplicación de las herramientas de modelado de propagación de la luz de los motores de juego para el modelado del canal radio en mmW. Esta Tesis propone un modelo de estimación de las pérdidas de propagación en mmW llamado "Modelo de Intensidad de Luz'' (LIM). Usando este modelo, basado en los procesos de iluminación realizados por los motores de juego, los transmisores de señal se sustituyen por focos de luz y la intensidad lumínica recibida en un punto se traduce a potencia de señal en milimétricas a través de una función polinómica sencilla. Una de las ventajas de usar los motores de juego es su gran capacidad y la facilidad que tiene el usuario para crear escenarios superrealistas que representen fielmente la geometría de escenarios donde se quiera evaluar el canal radio. De esta forma se pueden obtener estimaciones precisas de las pérdidas de propagación. La estimación de las pérdidas de propagación con LIM ha sido comparada con campañas de medida en las bandas de 28 GHz y 73 GHz y con otros modelos de propagación. Como resultado, el error de estimación de LIM es menor que los modelos estocásticos actuales y es comparable con el modelo de trazado de rayos. Y, además, el coste computacional de LIM comparado con el trazado de rayos es 130 veces menor, lo que posibilita el uso de LIM en escenarios altamente complejos para la estimación del canal radio en tiempo real. Los motores de juego permiten caracterizar de forma diferente la interacción de los materiales con la luz configurando el mapa de normales de sus superficies y sus funciones de dispersión y reflexión. En esta Tesis se ha determinado la caracterización de varios materiales que mejor se ajusta a medidas de laboratorio realizadas en un escenario controlado en la banda de 28 GHz. El modelo de LIM empleando materiales con esta caracterización óptima reduce más de un 50\% su error de estimación con respecto a la aplicación de LIM con los materiales por defecto, mientras que su coste computacional sigue siendo 26 veces menor que el modelo de trazado de rayos. Finalmente, se ha desarrollado sobre un motor de juego una primera versión de plataforma para la emulación de los sistemas 5G que es el punto de partida para un emulador completo de 5G. Esta plataforma no sólo contiene el modelo de LIM sino que incluye varios casos de uso de la 5G en entornos superrealistas. La plataforma, que se basa en el concepto de "Serious Game Engineering", rompe las limitaciones de los simuladores de redes móviles en cuanto a las capacidades de visualización e interacción del usuario con los componentes de la red en tiempo real.[CA] La cinquena generació de comunicacions mòbils, 5G, promet ser una revolució tecnològica que vaja més enllà de multiplicar la velocitat de transmissió de dades de les seues predecessores. Pretén suportar una gran quantitat de dispositius i aconseguir latències molt pròximes a 1 mil·lisegon. Per a satisfer aquests ambiciosos requisits, s'han investigat noves tecnologies habilitadores. Una d'elles és l'ús de les bandes d'ones mil·limètriques (mmW) en les quals hi ha una gran quantitat d'espectre disponible. Per a predir les característiques del canal ràdio i avaluar les prestacions de la 5G de forma fiable en les bandes mmW es requereixen models de canal complexos. Concretament, els models de propagació més precisos són els basats en traçat de rajos, però el seu alt cost computacional els fan inviables per a la caracterització del canal ràdio en escenaris complexos. D'altra banda, en els últims anys, la tecnologia de videojocs ha desenvolupat potents eines per a modelar la propagació de la llum en escenaris superrealistes. Donada la proximitat espectral entre l'espectre visible i les ones mmW, la present Tesi ha estudiat l'aplicació de les eines de modelatge de propagació de la llum dels motors de joc per al modelatge del canal radie en mmW. Aquesta Tesi proposa un model d'estimació de les pèrdues de propagació en mmW anomenat "Model d'Intensitat de Llum'' (LIM). Usant aquest model, basat en els processos d'il·luminació realitzats pels motors de joc, els transmissors de senyal se substitueixen per focus de llum i la intensitat lumínica rebuda en un punt es tradueix a potència de senyal en mil·limètriques a través d'una funció polinòmica senzilla. Una dels avantatges d'usar els motors de joc és la seua gran capacitat i la facilitat que té l'usuari per a crear escenaris superrealistes que representen fidelment la geometria d'escenaris on es vulga avaluar el canal ràdio. D'aquesta forma es poden obtindre estimacions precises de les pèrdues de propagació. L'estimació de les pèrdues de propagació amb LIM ha sigut comparada amb campanyes de mesura en les bandes de 28~GHz i 73~GHz i amb altres models de propagació. Com a resultat, l'error d'estimació de LIM és menor que els models estocàstics actuals i és comparable amb el model de traçat de rajos. I, a més, el cost computacional de LIM comparat amb el traçat de rajos és 130 vegades menor, la qual cosa possibilita l'ús de LIM en escenaris altament complexos per a l'estimació del canal ràdio en temps real. Els motors de joc permeten caracteritzar de forma diferent la interacció dels materials amb la llum configurant el mapa de normals de les seues superfícies i les seues funcions de dispersió i reflexió. En aquesta Tesi s'ha determinat la caracterització de diversos materials que s'ajusta millor a mesures de laboratori realitzades en un escenari controlat en la banda de 28 GHz. El model de LIM emprant materials amb aquesta caracterització òptima redueix més d'un 50 % el seu error d'estimació respecte a l'aplicació de LIM amb els materials per defecte, mentre que el seu cost computacional continua sent 26 vegades menor que el model de traçat de rajos. Finalment, s'ha desenvolupat sobre un motor de joc una primera versió de plataforma per a l'emulació dels sistemes 5G que és el punt de partida per a un emulador complet de 5G. Aquesta plataforma no solament conté el model de LIM sinó que inclou diversos casos d'ús de la 5G en entorns superrealistes. La plataforma, que es basa en el concepte de "Serious Game Engineering", trenca les limitacions dels simuladors de xarxes mòbils quant a les capacitats de visualització i interacció de l'usuari amb els components de la xarxa en temps real.[EN] The fifth generation of mobile communications, 5G, promises to be a technological revolution that goes beyond multiplying the data transmission speed of its predecessors. It aims to support a large number of devices and reach latencies very close to 1 millisecond. To meet these ambitious requirements, new enabling technologies have been researched. One of these is the use of millimetre-wave bands (mmW) in which a large amount of spectrum is available. Complex channel models are required to predict radio channel characteristics and reliably evaluate 5G performance in the mmW bands. Specifically, the most accurate propagation models are those based on ray tracing, but their high computational cost makes them unfeasible for radio channel characterization in complex scenarios. On the other hand, in recent years, video game technology has developed powerful tools to model the propagation of light in super realistic scenarios. Given the spectral closeness between the visible spectrum and the mmW waves, the present Thesis has studied the application of light propagation modeling tools from game engines for radio channel modeling in mmW. This Thesis proposes a model for estimating propagation losses in mmW called "Light Intensity Model'' (LIM). Using this model, based on the lighting processes performed by the game engines, the signal transmitters are replaced by light sources and the light intensity received at a point is translated into signal strength in mmW through a simple polynomial function. One of the advantages of using the game engines is their great capacity and the ease with which the user can create super realistic scenarios that faithfully represent the geometry of scenarios where the radio channel is to be evaluated. In this way, accurate estimates of propagation losses can be obtained. The estimation of propagation losses with LIM has been compared with measurement campaigns in the 28 GHz and 73 GHz bands and with other propagation models. As a result, the LIM estimation error is smaller than the current stochastic models and is comparable with the ray tracing model. In addition, the computational cost of LIM compared to ray tracing is 130 times lower, allowing the use of LIM in highly complex scenarios for real-time radio channel estimation. The game engines allow to characterize in a different way the interaction of the materials with the light configuring the normal map of their surfaces and their scattering and reflection functions. In this Thesis it has been determined the characterization of several materials that best fits to laboratory measurements made in a controlled scenario in the 28 GHz band. The LIM model using materials with this optimal characterization reduces by more than 50% its estimation error with respect to the application of LIM with default materials, while its computational cost remains 26 times lower than the ray tracing model. Finally, a first version of a platform for the emulation of 5G systems has been developed on a game engine, which is the starting point for a complete 5G emulator. This platform not only contains the LIM model but also includes several 5G use cases in super realistic environments. The platform, which is based on the concept of "`Serious Game Engineering", breaks the limitations of mobile network simulators in terms of visualization capabilities and user interaction with network components in real time.Inca Sánchez, SA. (2019). Serious Game Engineering and Lighting Models for the Realistic Emulation of 5G Systems [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/132695TESI

Procedural modelling techniques to configure driving serious game scenes

Esta dissertação tem o objetivo de tratar do problema da segurança rodoviária, a fim de evitar mais acidentes e mortes tanto de motoristas como de pedestres através da criação de uma ferramenta que é capaz de carregar dados do mundo real a partir de localizações selecionadas pelo utilizador e transformá-los em modelos tridimensionais para uso posterior em jogos de condução sérios. Estes modelos são então povoados com os pedestres que andam nos passeios e a capacidade de conduzir um veículo é dada ao utilizador. Além disso, a ferramenta deve ser flexível o suficiente para permitir que os utilizadores configurem diferentes condições, tais como o tempo, hora do dia, opções de renderização, os danos do veículo e densidade de pedestres, a fim de realizar estudos em diferentes condições. O projeto também deve ser de código aberto, para que qualquer pessoa pode editá-lo e expandi-lo para atender as suas necessidades e realizar estudos específicos. Possui integração Oculus Rift, que estende ainda mais a possibilidade de realização de estudos para o motorista humano, através da expanção dessa integração para avaliar os comportamentos do motorista. Outro aspeto importante é a possibilidade de exportar toda a cena procedimentalmente gerada para um formato de ficheiro 3D que pode ser editado numa aplicação externa.Ao fornecer esta ferramenta de forma gratuita, não só marca o início de um gerador do mundo em 3D de código aberto, mas também uma ferramenta capaz de permitir diversos usos diferentes, tais como a realização de estudos, a construção de cenários de jogos de vídeo ou ser usado como uma ferramenta de aprendizagem por uma escola de condução. Esperemos que isto seja capaz de aumentar a segurança rodoviária, se usado com cuidado como uma ferramenta séria.Para fazer isto vamos usar o motor de jogo Unity 5 para desenvolver o projeto, CGIAR-CSI para baixar dados de elevação, o Google Static Maps para as imagens de satélite, OpenStreetMap para os dados de localização e tudo o mais é construído dentro do Unity. Também é usado o UnitySlippyMap, que é um mapa do mundo que trabalha com vários fornecedores de "tiles" que foi integrado no contexto deste projeto para permitir que os utilizadores selecionem um local dentro do Unity.Ao longo deste documento, irá encontrar uma revisão da literatura sobre o tema, incluindo o trabalho relacionado e tentativas de fazer projetos semelhantes, seguido de uma comparação entre outros projetos e este. Irá também encontrar detalhes sobre o desenvolvimento e a arquitetura do sistema, bem como detalhes profundos sobre a implementação. No final pode encontrar algumas capturas de ecrã dos resultados deste projeto e a conclusão que refere a satisfação dos objetivos e trabalho futuro.Palavras Chave: Modelação Procedimental, Simulação de Condução, Locais do Mundo, Jogos Sérios, Segurança RodoviáriaThis dissertation has the objective of tackling the road safety problem in order to further prevent accidents and casualties for both drivers and pedestrians by creating a tool that is capable of loading real world data from user selected locations and render them in 3 dimensional models for further use in serious driving games. These models are then populated with pedestrians that walk around and the ability to drive a vehicle is given to the user. Also the tool should be flexible enough to allow the users to configure the different conditions such as weather, time of day, rendering options, vehicle damage and pedestrian density, in order to conduct studies on different conditions. The project should also be open source, so anyone can edit it and expand it their own way to suit their needs and conduct specific studies. It features Oculus Rift integration, which further extends the possibility of conducting studies to the human driver by giving the possibility to expand this integration to evaluate the driver's behaviours. Another important aspect is the possibility to export the entire procedurally generated scene to a 3D file format that can be edited by an external application.By providing such tool for free, not only marks the beginning of an open source world 3D generator, but also a framework capable of allowing multiple different usages, such as conducting studies, building video game scenarios or be used as a learning tool by a driving school for instance. Hopefully this will increase road safety if used carefully as a serious tool.To do so we'll use the game engine Unity 5 to develop the project, CGIAR-CSI to download elevation data, Google Static Maps for the satellite imagery, OpenStreetMap for the location data and everything else is built inside Unity. Also UnitySlippyMap, which is a world map that works with various tile providers was used and integrated on the context of this project to allow users to select a location inside Unity.Along this document you will find a literature review on the topic, including related work and attempts to do similar projects followed by a comparison between other project and this one. The reader will also find details about development and the system's architecture as well as deep details about implementation. On the end you can find a few screen shots of the results of this project.Key Words: Procedural Modelling, Driving Simulation, World Locations, Serious Games, Road Safet

Sim2real and Digital Twins in Autonomous Driving: A Survey

Safety and cost are two important concerns for the development of autonomous driving technologies. From the academic research to commercial applications of autonomous driving vehicles, sufficient simulation and real world testing are required. In general, a large scale of testing in simulation environment is conducted and then the learned driving knowledge is transferred to the real world, so how to adapt driving knowledge learned in simulation to reality becomes a critical issue. However, the virtual simulation world differs from the real world in many aspects such as lighting, textures, vehicle dynamics, and agents' behaviors, etc., which makes it difficult to bridge the gap between the virtual and real worlds. This gap is commonly referred to as the reality gap (RG). In recent years, researchers have explored various approaches to address the reality gap issue, which can be broadly classified into two categories: transferring knowledge from simulation to reality (sim2real) and learning in digital twins (DTs). In this paper, we consider the solutions through the sim2real and DTs technologies, and review important applications and innovations in the field of autonomous driving. Meanwhile, we show the state-of-the-arts from the views of algorithms, models, and simulators, and elaborate the development process from sim2real to DTs. The presentation also illustrates the far-reaching effects of the development of sim2real and DTs in autonomous driving