Studying Pedestrian’s Unmarked Midblock Crossing Behavior on a Multilane Road When Interacting With Autonomous Vehicles Using Virtual Reality

This dissertation focuses on the challenge of pedestrian interaction with autonomous vehicles (AVs) at unmarked midblock locations where the right-of-way is unspecified. A virtual reality (VR) simulation was developed to replicate an urban unmarked midblock environment where pedestrians cross a four-lane arterial roadway and interact with AVs. One research goal is to investigate the impact of roadway centerline features (undivided, two-way left-turn lane, and median) and AV operational schemes portrayed through on-vehicle signals (no signal, yellow negotiating indication, and yellow/blue negotiating/no-yield indications) on pedestrian crossing behavior. Results demonstrate that both roadway centerline design features and AV operations and signaling show significant impacts on pedestrians\u27 unmarked midblock crossing behavior, including the waiting time at the curb, waiting time in the middle of the road, and the total crossing time. Whereas, only the roadway centerline design features significantly impact the walking time, and only the AV operations and signaling significantly impact the accepted gap. Participants in the undivided centerline scene spent longer time waiting at the curb and walking on the road. Also, pedestrians are more likely to display risky behavior and cross in front of AVs indicating blue signals with non-yielding behavior in the presence of a median centerline scene. The inclusion of a yellow signal, which indicates the detection of pedestrians and signifies that the AVs will negotiate with them, resulted in a significant reduction in pedestrian waiting time both at the curb and in the middle of the road, when compared to AVs without a signal. Interaction effects between roadway centerline design features and AV operations and signaling are significant only for waiting time in the middle of the road. It is also found that older pedestrians tend to wait longer at the curb and are less likely to cross in front of AVs showing a blue signal with non-yielding behavior. Another research goal is to investigate how this VR experience change pedestrians’ perception of AVs. Results demonstrated that both pedestrians’ overall attitude toward AVs and trust in the effectiveness of AV systems significantly improved after the VR experience. It is also found that the more pedestrians trust the yellow signals, the more likely they are to improve their perception of AVs. Further, pedestrians who exhibit more aggressive crossing behavior are less likely to change their perception towards AVs as compared to those pedestrians who display rule-conforming crossing behaviors. Also, if the experiment made pedestrians feel motion sick, they were less likely to experience increased trust in the AV system\u27s effectiveness

Priorities and Potential of Pedestrian Protection - Accident data, Experimental tests and Numerical Simulations of Car-to-Pedestrian Impacts

Pedestrian disability and fatality as a consequence of car crashes is a large global health problem. To introduce maximally effective car-based countermeasures it is important to understand which injuries are most common and from which car parts they originate. It is also important to focus on the most severe injuries resulting in disability or death. The aim of this thesis was therefore to determine priorities for and evaluate the potential of car-mounted safety systems designed to mitigate severe upper-body injuries (including disability and fatality) of pedestrians in car crashes. Accident data was collected from two areas; severe (AIS3+) accidents in Dresden/Hannover in Germany and fatal accidents in Sweden. For the surviving pedestrians an estimate of long-term injury was performed using accident data- derived risk matrices of permanent injury. Results showed that 31% would sustain a permanent impairment of some kind and 5% would sustain a more severe impairment, where the head was most susceptible to severe impairment. The car front frequently caused leg injuries, which is addressed in current regulations. However, current legal tests do not address the most common upper-body injury source, the windshield, which was found to be the dominating cause of head injuries. Chest injuries, frequently caused by both the hood and windshield areas in the severe and fatal crashes in this thesis, are also unaddressed in legal tests. Children are most commonly head-injured from the hood area, which is addressed in current regulations. Further, regulations do not fully consider brain injury with the current head test methods. Therefore, in this thesis focus was on upper-body injury/source combinations not addressed in the regulations, that is, the head-to-windshield area and chest-to-hood/windshield areas, and the evaluation of brain injury in hood and windshield impacts. Experimental head-to-hood component tests with succeeding brain simulations were performed to evaluate the influence of the under-hood distance and head impact speed. A hood designed to minimize linear head loading to acceptable injury levels was also found effective in reducing combined linear/rotational brain loading. Further, in full-scale car-to-pedestrian finite element simulations both a braking and deployable system alone proved efficient in reducing head and chest loading, and an integrated countermeasure of combining the two systems proved to increase the protection potential. While current pedestrian countermeasures focus on the head-to-hood impact, this thesis recommends extending countermeasures to the lower part of the windshield and the A-pillars, and adding brain and chest injury assessment for both hood and windshield areas to effectively minimize disabling and fatal injuries. Since head impact location and head impact speed is dependent on the car design, the introduction of full-scale simulations in the test methods to determine impact conditions for experimental component tests is recommended. If the deployable countermeasures are combined with autonomous braking in an integrated system the most effective system is achieved. Auto-brake systems should, in high speed impacts, aim to reduce speeds to where the secondary countermeasures can effectively mitigate injury. Future pedestrian test methods should therefore evaluate how primary and secondary countermeasures interact

Pedestrian Detection by Computer Vision.

This document describes work aimed at determining whether the detection, bycomputer vision, of pedestrians waiting at signal-controlled road crossings could bemade sufficiently reliable and affordable, using currently available technology, so asto be suitable for widespread use in traffic control systems.The work starts by examining the need for pedestrian detection in traffic controlsystems and then goes onto look at the specific problems of applying a vision systemto the detection task. The most important distinctive features of the pedestriandetection task addressed in this work are:• The operating conditions are an outdoor environment with no constraints onfactors such as variation in illumination, presence of shadows and the effects ofadverse weather.• Pedestrians may be moving or static and are not limited to certain orientations orto movement in a single direction.• The number of pedestrians to be monitored is not restricted such that the visionsystem must cope with the monitoring of multiple targets concurrently.• The background scene is complex and so contains image features that tend todistract a vision system from the successful detection of pedestrians.• Pedestrian attire is unconstrained so detection must occur even when details ofpedestrian shape are hidden by items such as coats and hats.• The camera's position is such that assumptions commonly used by vision systemsto avoid the effects of occlusion, perspective and viewpoint variation are not valid.•The implementation cost of the system, in moderate volumes, must be realistic forwidespread installation.A review of relevant prior art in computer vision with respect to the above demands ispresented. Thereafter techniques developed by the author to overcome thesedifficulties are developed and evaluated over an extensive test set of image sequencesrepresentative of the range of conditions found in the real world.The work has resulted in the development of a vision system which has been shown toattain a useful level of performance under a wide range of environmental andtransportation conditions. This was achieved, in real-time, using low-cost processingand sensor components so demonstrating the viability of developing the results of thiswork into a practical detector

A Context Aware Classification System for Monitoring Driver’s Distraction Levels

Understanding the safety measures regarding developing self-driving futuristic cars is a concern for decision-makers, civil society, consumer groups, and manufacturers. The researchers are trying to thoroughly test and simulate various driving contexts to make these cars fully secure for road users. Including the vehicle’ surroundings offer an ideal way to monitor context-aware situations and incorporate the various hazards. In this regard, different studies have analysed drivers’ behaviour under different case scenarios and scrutinised the external environment to obtain a holistic view of vehicles and the environment. Studies showed that the primary cause of road accidents is driver distraction, and there is a thin line that separates the transition from careless to dangerous. While there has been a significant improvement in advanced driver assistance systems, the current measures neither detect the severity of the distraction levels nor the context-aware, which can aid in preventing accidents. Also, no compact study provides a complete model for transitioning control from the driver to the vehicle when a high degree of distraction is detected. The current study proposes a context-aware severity model to detect safety issues related to driver’s distractions, considering the physiological attributes, the activities, and context-aware situations such as environment and vehicle. Thereby, a novel three-phase Fast Recurrent Convolutional Neural Network (Fast-RCNN) architecture addresses the physiological attributes. Secondly, a novel two-tier FRCNN-LSTM framework is devised to classify the severity of driver distraction. Thirdly, a Dynamic Bayesian Network (DBN) for the prediction of driver distraction. The study further proposes the Multiclass Driver Distraction Risk Assessment (MDDRA) model, which can be adopted in a context-aware driving distraction scenario. Finally, a 3-way hybrid CNN-DBN-LSTM multiclass degree of driver distraction according to severity level is developed. In addition, a Hidden Markov Driver Distraction Severity Model (HMDDSM) for the transitioning of control from the driver to the vehicle when a high degree of distraction is detected. This work tests and evaluates the proposed models using the multi-view TeleFOT naturalistic driving study data and the American University of Cairo dataset (AUCD). The evaluation of the developed models was performed using cross-correlation, hybrid cross-correlations, K-Folds validation. The results show that the technique effectively learns and adopts safety measures related to the severity of driver distraction. In addition, the results also show that while a driver is in a dangerous distraction state, the control can be shifted from driver to vehicle in a systematic manner

A High-Fidelity VR Simulation Study: Do External Warnings Really Improve Pedestrian Safe Crossing Behavior?

To better communicate with pedestrians, adding external displays to autonomous vehicles (AVs) has been proposed as a potential communication method to encourage safe crossing behavior by pedestrians. Whereas, most researchers have conducted intercept interviews, lab studies, or simulation studies to explore the efficacy of these displays, these approaches only studied crossing intention but did not explore crossing behavior. We developed a high-fidelity virtual reality scenario where participants could demonstrate actual crossing behavior within an adequately replicated real-world street. We simulated a local street with scalability of the real world in a VR environment, conducted an experiment in an empty space large enough for participants to move across the road in the VR environment. A mixed-method approach assessed attitudinal and behavioral interactions with potential warning patterns. The results showed that the warning patterns contributed significantly to pedestrians’ perceptual vigilance, as in past studies, but safer crossing behavior was not observed. This suggests that crossing intention measures may not be an adequate substitute for behavioral measures of crossing

Sensory Augmentation for Navigation in Difficult Urban Environments by People With Visual Impairment

Independent mobility in completing such tasks as walking through a town centre is taken for granted by well-bodied individuals. However, for those with a disability such as impairment of vision, mobility and navigation can become challenging tasks not easily undertaken. The barriers to access for blind and partially sighted individuals are increased when familiar navigational cues are removed in difficult urban environments such as Shared Space. The research consisted of investigating methods of navigation employed by people with visual impairment and designing a device to restore confidence to this group so as to lower the barriers of access to such environments. Investigation was carried out through the deployment of a questionnaire; discussions with groups representing blind and partially sighted people; and a site visit to Shared Space environments. Statistical analysis was carried out on the results of the questionnaire to ascertain the navigational habits of blind and partially sighted individuals in different environments. From the analysis and the results of the discussions and site visit it was established that it would be socially acceptable to design a secondary aid to navigation that would complement the primary aids of long cane or guide dog. A concept experiment was carried out to test the idea that knowledge about changes in surface colour could help with navigation. A prototype device that could be used by individuals with visual impairment to increase their confidence when navigating a difficult environment was designed, built and tested. Different programming methods were researched and trialled to effectively use machine vision to provide a solution to analyse video feed from a passive camera and return useful information to a blind or partially sighted user. The device was tested indoors and outdoors and found to be effective at detecting changes in surface colour. Further work is needed to run the software on a more compact platform such as a mobile phone, but initial results show that the concept is viable and that the barriers that present to blind and partially sighted people navigating difficult urban environments can be much reduced through the use of this technology

Radial Basis Function Neural Network in Identifying The Types of Mangoes

Mango (Mangifera Indica L) is part of a fruit plant species that have different color and texture characteristics to indicate its type. The identification of the types of mangoes uses the manual method through direct visual observation of mangoes to be classified. At the same time, the more subjective way humans work causes differences in their determination. Therefore in the use of information technology, it is possible to classify mangoes based on their texture using a computerized system. In its completion, the acquisition process is using the camera as an image processing instrument of the recorded images. To determine the pattern of mango data taken from several samples of texture features using Gabor filters from various types of mangoes and the value of the feature extraction results through artificial neural networks (ANN). Using the Radial Base Function method, which produces weight values, is then used as a process for classifying types of mangoes. The accuracy of the test results obtained from the use of extraction methods and existing learning methods is 100%

Deep Learning Detected Nutrient Deficiency in Chili Plant

Chili is a staple commodity that also affects the Indonesian economy due to high market demand. Proven in June 2019, chili is a contributor to Indonesia's inflation of 0.20% from 0.55%. One factor is crop failure due to malnutrition. In this study, the aim is to explore Deep Learning Technology in agriculture to help farmers be able to diagnose their plants, so that their plants are not malnourished. Using the RCNN algorithm as the architecture of this system. Use 270 datasets in 4 categories. The dataset used is primary data with chili samples in Boyolali Regency, Indonesia. The chili we use are curly chili. The results of this study are computers that can recognize nutrient deficiencies in chili plants based on image input received with the greatest testing accuracy of 82.61% and has the best mAP value of 15.57%

Study on the Implementation of Autonomous Driving Communications with 5G

V zadnjih desetletjih je tehnološki napredek povzročil razvoj številnih novih idej in področij, ki pomembno prispevajo k izboljšanju kakovosti življenja ljudi. Ena od takšnih, ki se obeta v bližnji prihodnosti, je tudi avtonomna vožnja. Čeprav so v preteklosti znanstveniki že poskušali uresničiti izdelavo samovozečega avtomobila, razpoložljive tehnologije do danes niso zmogle izpolnjevati potrebnih zahtev. Peta generacija mobilnih sistemov, poznana kot 5G, je prvo tehnološko področje, ki predstavlja obetaven element pri realizaciji popolnoma avtonomne vožnje. Cilj tega magistrskega dela je bil raziskati, ali je omrežje 5G iz 3GPP Release 15 sposobno zagotoviti potrebna sredstva za popolno podporo avtonomni vožnji in drugih družin primerov uporabe C-V2X, hkrati pa delovati v izjemno zahtevnih (omrežnih) scenarijih cestnega prometa. V teoretičnem delu magistrskega dela smo najprej predstavili nekaj najpomembnejših konceptov v paradigmi avtonomnih vozil, ki nam bodo kasneje pomagali bolje razumeti praktični del. Najprej smo s komunikacijskega stališča definirali avtonomno vozilo in njegove komunikacijske potrebe in storitve. Nato smo našteli nekaj najpomembnejših prednosti avtonomnih vozil, pa tudi nekatere najbolj zaskrbljujoče izzive, s katerimi se srečujejo strokovnjaki, ki se razlikujejo od tehnoloških in okoljskih do zakonodajnih in filozofskih. Na kratko smo predstavili tudi Vehicular ad hoc networks, Intelligent transport systems, Vehicle-to-everything in kako bi nadaljnji razvoj takšnih konceptov pozitivno vplival na človeštvo. V zadnjem delu teoretičnega dela smo podrobneje razdelali tehnologijo C-V2X, kjer smo začeli s kratkim pregledom njenega razvoja skozi zgodovino. Gre za tehnologijo, ki uporablja obstoječe celično omrežje za zagotavljanje povezave med vozili, med vozili in infrastrukturo, med vozili in omrežjem ter med vozili in pešci. Vsak nov korak k popolnoma avtonomnemu vozilu prinaša nove sklope še bolj zapletenih izzivov. Na začetku sta bili najpomembnejši tehnologiji, ki sta bili uporabljeni za V2X, DSRC in LTE-based V2X. Čeprav ima LTE-based V2X več 18 Povzetek prednosti v primerjavi z DSRC, sta bila oba presežena z vidika zmogljivosti, ko je bil 5G NR-V2X predstavljen skupaj s 5G celičnim omrežjem. Aplikacije V2X, kot so kooperativno zaznavanje in manevriranje med vozili, vodenje z visoko gostoto vozil in daljinsko vodenje vozil, podajajo težko dosegljive računalniške in komunikacijske zahteve, ki presegajo LTE in DSRC. Večina varnostnih aplikacij V2X zahteva izjemno majhno zakasnitev (pod 10 ms), izjemno visoko zanesljivost (blizu 100 %) in visoko hitrost prenosa podatkov (v območju Gbps). Celično omrežje 5G ima edino dovolj napredno tehnologijo, ki lahko doseže takšne pragove zmogljivosti. Poleg tega smo predstavili načrt 5GAA, ki prikazuje rezultate in napovedi na ključnih prednostnih področjih, kot je pospešek razvoja celičnih tehnologij iz LTEbased V2X v 5G NR-V2X. Teoretični del se konča s prikazom, kako je 5G omogočil več primerov uporabe C-V2X, ki jih prej ni bilo mogoče vzdrževati z LTE kot nosilno tehnologijo, pa tudi kako lahko podpre novejše, bolj zapletene primere uporabe z višjimi zahtevami. Spoznali smo, da so nekateri primeri uporabe C-V2X izjemno zahtevni za omrežje, in sicer do te mere, da se pojavi dvom, ali bodo sploh sposobni delovati v resničnih situacijah. Teoretične raziskave smo dopolnili z vrsto simulacij, z namenom, da bi spremljali zakasnitev paketov in zanesljivost simuliranega omrežja 5G, in jo kasneje primerjali z zahtevami QoS družin primerov uporabe C-V2X. Simulacije so bile izvedene s pomočjo ogrodja OMNeT ++ 5G-Sim-V2I/N, ki omogoča simulacijo primerov uporabe 5G V2I / V2N z aplikacijami, ki obsegajo celotno uporabniško ravnino 5G. Simulirali smo dva različna scenarija cestnega prometa. V prvem scenariju smo simulirali avtocesto, kjer avtomobili vozijo z večjo hitrostjo, kar močno vpliva na delovanje omrežja. V drugem scenariju je bila pobuda izbrati nekaj povsem drugačnega, z namenom, da bi opazovali različne rezultate med dvema skrajnostnima. Zato je bila izbira za drugi scenarij urbano okolje, kjer je veliko ovir, kot so stavbe ali visoka drevesa, ki bi lahko motila komunikacijo. V obeh scenarijih sta bila izmerjena dva parametra QoS za štiri različne aplikacije UDP, ki delujejo vzporedno na več vozilih. Vzporedno izvajane aplikacije so vključevale aplikacije V2X, VoIP klic, video predvajanje in prenos podatkov od in k uporabniku, medtem ko so merjeni KPI vključevali zakasnitev in zanesljivost paketov. Izziv je bil spremljati zmogljivost omrežja v celotnem simulacijskem času in na koncu primerjati končne rezultate s celotnimi zahtevami glede zakasnitve in zanesljivosti nekaterih najzahtevnejših družin primerov uporabe C-V2X, ki naj bi bile Povzetek 19 s pomočjo 5G omrežja izvedljive. Cilj je bil ugotoviti, za katere od teh družin primerov uporabe simulirano omrežje 5G izpolnjuje zahteve. Simulirali smo omrežje 5G po specifikacijah 3GPP Release 15 in rezultate ocenili glede na zakasnitev na celotni komunikacijski poti in zanesljivost. Rezultati so pokazali, da omrežje ne izpolnjuje zahtev, potrebnih za podporo tako skrajnih primerov uporabe v simuliranih scenarijih. Te ugotovitve so nas pripeljale do zaključka, da simulirano omrežje potrebuje nadaljnje izboljšave zmogljivosti, zlasti z vidika znižanja zakasnitev in povečanja zanesljivosti, kar je bilo tudi obljubljeno za prihodnje izdaje 3GPP.In recent decades, technological advances have led to the development of many new ideas and areas that have made an important contribution to improving people\u27s quality of life. Autonomous driving is one area that has resulted from these advances. While attempts have been made in the past to try and facilitate the realization of the self-driving car, the available technologies have never met the needed requirements. However, the recently introduced 5G is believed to be a promising enabler for the fully autonomous vehicle. The goal of this thesis was to investigate whether the 5G network from 3GPP Release 15 is capable of providing the needed resources to fully support autonomous driving and other C-V2X use-case families while simultaneously operating in extremely demanding (network-wise) road traffic scenarios. In the theoretical part of the thesis (in the first four chapters), we firstly introduced some of the most relevant concepts in the autonomous vehicle paradigm, which would later help us better understand the practical part. We begin by defining the autonomous vehicle from a communication standpoint, and its communication needs and services. We continue by listing some of the most important benefits of autonomous vehicles, as well as some of the most troubling challenges that experts face, varying from technological and environmental challenges to legislative and philosophical ones. We also briefly present Vehicular Ad Hoc Networks, Intelligent Transport Systems, Vehicle-to-Everything and how the further development of such concepts would have a positive impact on humanity. The fourth chapter discusses the C-V2X technology, beginning with a brief overview of its evolution throughout history. It is a technology that uses the existing cellular network to provide the vehicle-to-vehicle, vehicle-to-infrastructure, vehicleto-network, and vehicle-to-pedestrian connections. Every new step towards a fully autonomous vehicle brings about new sets of even more complicated challenges. In the very beginning, the two most relevant technologies used for V2X were DSRC and the LTE-based V2X. Although the LTE- 22 Abstract based V2X has multiple advantages compared to DSRC, they both got outclassed, in terms of performance, once 5G NR-V2X was introduced along with the 5G cellular network. V2X applications, such as cooperative sensing and maneuvering, high-density platooning and teleoperated driving, show hard-to-meet computing and communication demands, well beyond what LTE and DSRC can provide. Most V2X safety applications demand ultra-low latency (below 10 ms), ultra-high reliability (near 100%), and a high data rate (in the Gbps range). The 5G cellular network is the only cellular technology advanced enough to be capable of reaching such performance thresholds. In addition, we presented the 5GAA roadmap which shows the results and predictions in the key priority areas, such as the acceleration of the evolution of cellular technologies from the LTE-based V2X towards the 5G NR-V2X. The theoretical part ends by showing how 5G has enabled more C-V2X use cases that were previously impossible to sustain with LTE as the underlying technology, as well as how it has introduced newer, more complex use cases with higher requirements. It quickly became clear that some of the C-V2X use cases are extremely demanding on the network, to the point where a lot of doubts arose whether they would be able to operate in real-life situations. The theoretical research was complemented with a series of simulations in order to monitor the packet delay and reliability of the simulated 5G network, which would later be compared to the QoS requirements of the C-V2X use-case families. The simulations were executed with the help of the OMNeT++ framework 5G-Sim-V2I/N which enables to simulate 5G V2I/V2N use cases with applications comprising the whole 5G user plane. We have simulated two different road-traffic scenarios. In the first scenario, we simulated a motorway where the cars drive at higher speeds, which can have a huge effect on the network\u27s performance. In the second scenario, the initiative was to choose something completely different from the first one in order to observe different results between two extremes. Hence, the choice for the second scenario was an urban environment with heterogeneous road characteristics and buildings that act as obstacles which interfere with the signals. In both scenarios, two QoS parameters were measured for four different UDP applications running in parallel, on multiple cars. The running parallel applications included a V2X application, a VoIP call, a video stream, and a data download/upload, while the measured KPIs included packet delay and reliability. The challenge was to monitor the network performance throughout the simulation, and in the end compare Abstract 23 the final results with the end-to-end latency and reliability requirements of some of the most demanding C-V2X use-case families, which were said to have been enabled by the 5G network. The objective was to find out for which of these C-V2X use-case families the simulated network met the requirements. We have simulated the 5G network from 3GPP Release 15 and have evaluated our results in terms of end-to-end-latency and reliabilityit quickly became clear that the network did not meet the requirements needed to support such extreme use-case families in the simulated scenarios. These findings brought us to the conclusion that the simulated network is in need of serious performance enhancements, in terms of lowering its latency and increasing its reliability, both of which have been promised for future 3GPP release