MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving

Autonomous driving requires operation in different behavioral modes ranging from lane following and intersection crossing to turning and stopping. However, most existing deep learning approaches to autonomous driving do not consider the behavioral mode in the training strategy. This paper describes a technique for learning multiple distinct behavioral modes in a single deep neural network through the use of multi-modal multi-task learning. We study the effectiveness of this approach, denoted MultiNet, using self-driving model cars for driving in unstructured environments such as sidewalks and unpaved roads. Using labeled data from over one hundred hours of driving our fleet of 1/10th scale model cars, we trained different neural networks to predict the steering angle and driving speed of the vehicle in different behavioral modes. We show that in each case, MultiNet networks outperform networks trained on individual modes while using a fraction of the total number of parameters.Comment: Published in IEEE WACV 201

Pavement Performance Evaluation Using Connected Vehicles

Roads deteriorate at different rates from weathering and use. Hence, transportation agencies must assess the ride quality of a facility regularly to determine its maintenance needs. Existing models to characterize ride quality produce the International Roughness Index (IRI), the prevailing summary of roughness. Nearly all state agencies use Inertial Profilers to produce the IRI. Such heavily instrumented vehicles require trained personnel for their operation and data interpretation. Resource constraints prevent the scaling of these existing methods beyond 4% of the network. This dissertation developed an alternative method to characterize ride quality that uses regular passenger vehicles. Smartphones or connected vehicles provide the onboard sensor data needed to enable the new technique. The new method provides a single index summary of ride quality for all paved and unpaved roads. The new index is directly proportional to the IRI. A new transform integrates sensor data streams from connected vehicles to produce a linear energy density representation of roughness. The ensemble average of indices from different speed ranges converges to a repeatable characterization of roughness. The currently used IRI is undefined at speeds other than 80 km/h. This constraint mischaracterizes roughness experienced at other speeds. The newly proposed transform integrates the average roughness indices from all speed ranges to produce a speed-independent characterization of ride quality. This property avoids spatial wavelength bias, which is a critical deficiency of the IRI. The new method leverages the emergence of connected vehicles to provide continuous characterizations of ride quality for the entire roadway network. This dissertation derived precision bounds of deterioration forecasting for models that could utilize the new index. The results demonstrated continuous performance improvements with additional vehicle participation. With practical traversal volumes, the achievable precision of forecast is within a few days. This work also quantified capabilities of the new transform to localize roadway anomalies that could pose travel hazards. The methods included derivations of the best sensor settings to achieve the desired performances. Several case studies validated the findings. These new techniques have the potential to save agencies millions of dollars annually by enabling predictive maintenance practices for all roadways, worldwide.Mountain Plains Consortium (MPC

A simulation environment for drone cinematography

In this paper, we present a workflow for the simulation of drone operations exploiting realistic background environments constructed within Unreal Engine 4 (UE4). Methods for environmental image capture, 3D reconstruction (photogrammetry) and the creation of foreground assets are presented along with a flexible and user-friendly simulation interface. Given the geographical location of the selected area and the camera parameters employed, the scanning strategy and its associated flight parameters are first determined for image capture. Source imagery can be extracted from virtual globe software or obtained through aerial photography of the scene (e.g. using drones). The latter case is clearly more time consuming but can provide enhanced detail, particularly where coverage of virtual globe software is limited. The captured images are then used to generate 3D background environment models employing photogrammetry software. The reconstructed 3D models are then imported into the simulation interface as background environment assets together with appropriate foreground object models as a basis for shot planning and rehearsal. The tool supports both free-flight and parameterisable standard shot types along with programmable scenarios associated with foreground assets and event dynamics. It also supports the exporting of flight plans. Camera shots can also be designed to provide suitable coverage of any landmarks which need to appear in-shot. This simulation tool will contribute to enhanced productivity, improved safety (awareness and mitigations for crowds and buildings), improved confidence of operators and directors and ultimately enhanced quality of viewer experience

Automatic Classification and Quantification of Basic Distresses on Urban Flexible Pavement through Convolutional Neural Networks

[EN] Pavement condition assessment is a critical step in road pavement management. In contrast to the automatic and objective methods used for rural roads, the most commonly used method in urban areas is the development of visual surveys usually filled out by technicians that leads to a subjective pavement assessment. While most previous studies on automatic identification of distresses focused on crack detection, this research aims not only to cover the identification and classification of multiple urban flexible pavement distresses (longitudinal and transverse cracking, alligator cracking, raveling, potholes, and patching), but also to quantify them through the application of Convolutional Neural Networks. Additionally, this study also proposes a methodology for an automatic pavement assessment considering the different stages developed in this research. This methodology allows for a more efficient and reliable pavement assessment, minimizing the cost and time required by the current visual surveys.The study presented in this paper is part of the research project titled SIMEPU Sistema Integral de Mantenimiento Eficiente de Pavimentos Urbanos, funded by the Spanish Ministries of Science and Innovation and Universities, as well as the European Regional Development Fund under Grant No. RTC-2017-6148-7. The authors also acknowledge the support of partner companies Pavasal Empresa Constructora, S.A. and CPS Infraestructuras, Movilidad y Medio Ambiente, S.L. and the Valencia City Council.Llopis-Castelló, D.; Paredes Palacios, R.; Parreño-Lara, M.; García-Segura, T.; Pellicer, E. (2021). Automatic Classification and Quantification of Basic Distresses on Urban Flexible Pavement through Convolutional Neural Networks. Journal of Transportation Engineering, Part B: Pavements. 147(4):1-8. https://doi.org/10.1061/JPEODX.000032118147

Metropolitan Segment Traffic Speeds from Massive Floating Car Data in 10 Cities

Traffic analysis is crucial for urban operations and planning, while the availability of dense urban traffic data beyond loop detectors is still scarce. We present a large-scale floating vehicle dataset of per-street segment traffic information, Metropolitan Segment Traffic Speeds from Massive Floating Car Data in 10 Cities (MeTS-10), available for 10 global cities with a 15-minute resolution for collection periods ranging between 108 and 361 days in 2019-2021 and covering more than 1500 square kilometers per metropolitan area. MeTS-10 features traffic speed information at all street levels from main arterials to local streets for Antwerp, Bangkok, Barcelona, Berlin, Chicago, Istanbul, London, Madrid, Melbourne and Moscow. The dataset leverages the industrial-scale floating vehicle Traffic4cast data with speeds and vehicle counts provided in a privacy-preserving spatio-temporal aggregation. We detail the efficient matching approach mapping the data to the OpenStreetMap road graph. We evaluate the dataset by comparing it with publicly available stationary vehicle detector data (for Berlin, London, and Madrid) and the Uber traffic speed dataset (for Barcelona, Berlin, and London). The comparison highlights the differences across datasets in spatio-temporal coverage and variations in the reported traffic caused by the binning method. MeTS-10 enables novel, city-wide analysis of mobility and traffic patterns for ten major world cities, overcoming current limitations of spatially sparse vehicle detector data. The large spatial and temporal coverage offers an opportunity for joining the MeTS-10 with other datasets, such as traffic surveys in traffic planning studies or vehicle detector data in traffic control settings.Comment: Accepted by IEEE Transactions on Intelligent Transportation Systems (T-ITS), DOI: https://doi.org/10.1109/TITS.2023.329173

Analysis of the Efficiency of Traffic Lights Turning Red in Case of Exceeding Speed Limit

Due to the presence of various traffic calming measures (TCM) and traffic lights in urban areas, the speed of vehicles is maintained low. Nevertheless, a problem arises in the frontier between urban and non-urban areas because drivers must adapt their speed and behavior to new conditions. This risk becomes even greater in rural roads that penetrate small villages without a bypass and with a short urban segment, since drivers do not normally speed down in these segments. Various measures can be installed, but traffic lights that turn red if the speed limit is exceeded is not usually considered as a TCM in the literature. Therefore, this paper aims to analyze the efficiency of traffic lights turning red in case of exceeding speed limit. The village of Abalos in Spain was selected for this research, with an urban area of 630 m and this type of traffic lights in both directions. Results showed that drivers do not respect the speed limit - and hence, the red light - when they are placed separately. However, if they are placed next to a crosswalk, their effect is increased. Consequently, it is recommended to place these traffic lights with a crosswalk to reinforce the efficiency of both TCMs.Debido a la presencia de varias medidas de calmado de tráfico (MCT) y semáforos en zonas urbanas, la velocidad de los vehículos semantiene baja. No obstante, un problema aparece en la frontera entre áreas urbanas y no urbanas porque los conductores debenadaptar su velocidad y comportamiento a nuevas condiciones. Este riesgo se vuelve incluso mayor en las carreteras interurbanas quepenetran en pequeñas poblaciones sin circunvalación y con un corto tramo urbano, pues normalmente los conductores no reducensu velocidad en estos tramos. Varias medidas pueden ser instaladas, pero los semáforos que se ponen rojos si se sobrepasa el límitede velocidad no suelen considerarse como MCT en la literatura. Por lo tanto, el objetivo de este artículo es analizar la eficacia delos semáforos que se ponen rojos en caso de exceder el límite de velocidad. La población de Abalos en España fue seleccionadapara esta investigación, con un área urbana de 630 m y con este tipo de semáforos en ambas direcciones. Los resultados muestranque los conductores no respetan el límite de velocidad - y por tanto, el semáforo en rojo - cuando son colocados por separado. Sinembargo, si son colocados junto a un paso de peatones, se aumenta su efecto. En consecuencia, se recomienda disponer este tipo desemáforos junto con un paso de peatones para reforzar la eficacia de ambas MCT.This work was funded by GIRDER Ingenieros, S.L.P. [grant 2019.0478]

A Series of Vertical Deflections, a Promising Traffic Calming Measure: Analysis and Recommendations for Spacing

Traffic calming measures (TCM) are placed in urban areas to improve road safety, and among them, vertical TCMs are widely employed. Many researches are focused on the influence of the geometry of each measure on speed reduction, but it is demonstrated that drivers forget its effect and speed up after it. Therefore, placing consecutive TCMs can help to maintain a safe area. However, scarce literature can be found about the adequate spacing between them. Hence, the aim of this paper is to analyze the adequate distance between TCMs. Various streets with variable distances and different vertical TCMs were evaluated in Poland and Spain, including raised crosswalks, raised intersections, speed humps and speed cushions. The intermediate point between two TCMs was selected as the place where the maximum speed is achieved. Results showed that there was a good correlation between the speeds at intermediate points and the distance between TCMs, with a determination coefficient around 0.80. For an 85th percentile of the speed under 50 km/h, a maximum distance of 200 m between TCMs is recommended, and for a value of 40 km/h, 75 m.This research was funded by “Eramus+ programme—Call 2016—KA1—Mobility of Staff in higher education—Staff mobility for teaching and training activities

A comparative study of monitoring methods in sustainable pavement system development

Pavement management system (PMS) has been receiving increasing attention from both the government and private sectors in the attempt to ensure and keep the roads in good condition. The appropriate level of road maintenance activity is often contingent upon the type of pavement distress. Valid and reliable pavement data would lead to develop a PMS which is more suitable for agencies. Previous studies which attempted to identify modes of monitoring pavements were limited by constraints such as cost, time, and safety. This study was conducted to review some of the pavement monitoring modes introduced in previous studies. After completing a literature review, three mostly used modes, namely manual survey, smart sensor, and optical image processing, are selected for a comparative study to determine which mode is the most effective method in terms of cost, time, safety, accuracy, and sustainability. A data quality guideline was modified to produce a rating system for ranking the modes. In conclusion, the findings of this study could provide a guideline for the government and private sectors in determining the most effective pavement monitoring mode to be used in the sustainable PMS strategy

Optimization of Single and Layered Surface Texturing

In visualization problems, surface shape is often a piece of data that must be shown effectively. One factor that strongly affects shape perception is texture. For example, patterns of texture on a surface can show the surface orientation from foreshortening or compression of the texture marks, and surface depth through size variation from perspective projection. However, texture is generally under-used in the scientific visualization community. The benefits of using texture on single surfaces also apply to layered surfaces. Layering of multiple surfaces in a single viewpoint allows direct comparison of surface shape. The studies presented in this dissertation aim to find optimal methods for texturing of both single and layered surfaces. This line of research starts with open, many-parameter experiments using human subjects to find what factors are important for optimal texturing of layered surfaces. These experiments showed that texture shape parameters are very important, and that texture brightness is critical so that shading cues are available. Also, the optimal textures seem to be task dependent; a feature finding task needed relatively little texture information, but more shape-dependent tasks needed stronger texture cues. In visualization problems, surface shape is often a piece of data that must be shown effectively. One factor that strongly affects shape perception is texture. For example, patterns of texture on a surface can show the surface orientation from foreshortening or compression of the texture marks, and surface depth through size variation from perspective projection. However, texture is generally under-used in the scientific visualization community. The benefits of using texture on single surfaces also apply to layered surfaces. Layering of multiple surfaces in a single viewpoint allows direct comparison of surface shape. The studies presented in this dissertation aim to find optimal methods for texturing of both single and layered surfaces. This line of research starts with open, many-parameter experiments using human subjects to find what factors are important for optimal texturing of layered surfaces. These experiments showed that texture shape parameters are very important, and that texture brightness is critical so that shading cues are available. Also, the optimal textures seem to be task dependent; a feature finding task needed relatively little texture information, but more shape-dependent tasks needed stronger texture cues

JUICINESS IN CITIZEN SCIENCE COMPUTER GAMES: ANALYSIS OF A PROTOTYPICAL GAME

Incorporating the collective problem-solving skills of non-experts could ac- celerate the advancement of scientific research. Citizen science games leverage puzzles to present computationally difficult problems to players. Such games typ- ically map the scientific problem to game mechanics and visual feed-back helps players improve their solutions. Like games for entertainment, citizen science games intend to capture and retain player attention. “Juicy” game design refers to augmented visual feedback systems that give a game personality without modi- fying fundamental game mechanics. A “juicy” game feels alive and polished. This thesis explores the use of “juicy” game design applied to the citizen science genre. We present the results of a user study in its effect on player motivation with a prototypical citizen science game inspired by clustering-based E. coli bacterial strain analysis