Self-Driving Car A Deep-Learning Approach

Nowadays self-directed learning and automation are not restricted to human beings only. If you stare out at the automotive horizon, you can see a new exciting era coming into limelight: the age of self-driving cars. An age when humans will no longer need to keep their eyes on the road. No more concerns about distraction while driving or those stressful rush hour commutes, vehicles will whisk us where we want to go, blazingly fast and efficiently. This paper aims at demonstrating a system, which is able to drive a car on road without any human input. Both software and hardware parts are discussed here. The vehicle would contain certain sensors such as GPS, Ultrasonic Sensor, Camera and would contain an on-board computer for decision making. Waypoint data would be obtained from a nav provider like Google Maps. All of it would be simulated in CARLA, an open-source simulator

Application of Image Processing Techniques for Autonomous Cars

This paper aims to implement different image processing techniques that will help to control an autonomous car. A multistage pre-processing technique is used to detect the lanes, street signs, and obstacles accurately. The images captured from the autonomous car are processed by the proposed system which is used to control the autonomous vehicle. Canny edge detection was applied to the captured image for detecting the edges, Also, Hough transform was used to detect and mark the lanes immediately to the left and right of the car. This work attempts to highlight the importance of autonomous cars which drastically increase road safety and improve the efficiency of driving compared to human drivers. The performance of the proposed system is observed by the implementation of the autonomous car that is able to detect and classify the stop signs and other vehicles

An advanced deep learning model for maneuver prediction in real-time systems using alarming-based hunting optimization

The increasing trend of autonomous driving vehicles in smart cities emphasizes the need for safe travel. However, the presence of obstacles, potholes, and complex road environments, such as poor illumination and occlusion, can cause blurred road images that may impact the accuracy of maneuver prediction in visual perception systems. To address these challenges, a novel ensemble model named ABHO-based deep CNN-BiLSTM has been proposed for traffic sign detection. This model combines a hybrid convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) with the alarming-based hunting optimization (ABHO) algorithm to improve maneuver prediction accuracy. Additionally, a modified hough-enabled lane generative adversarial network (ABHO based HoughGAN) has been proposed, which is designed to be robust to blurred images. The ABHO algorithm, inspired by the defending and social characteristics of starling birds and Canis kojot, allows the model to efficiently search for the optimal solution from the available solutions in the search space. The proposed ensemble model has shown significantly improved accuracy, sensitivity, and specificity in maneuver prediction compared to previously utilized methods, with minimal error during lane detection. Overall, the proposed ensemble model addresses the challenges faced by autonomous driving vehicles in complex and obstructed road environments, offering a promising solution for enhancing safety and reliability in smart cities

Chartopolis - A Self Driving Car Test Bed

abstract: This thesis presents an autonomous vehicle test bed which can be used to conduct studies on the interaction between human-driven vehicles and autonomous vehicles on the road. The test bed will make use of a fleet of robots which is a microcosm of an autonomous vehicle performing all the vital tasks like lane following, traffic signal obeying and collision avoidance with other vehicles on the road. The robots use real-time image processing and closed-loop control techniques to achieve automation. The testbed also features a manual control mode where a user can choose to control the car with a joystick by viewing a video relayed to the control station. Stochastic rogue vehicle processes will be introduced into the system which will emulate random behaviors in an autonomous vehicle. The test bed was experimented to perform a comparative study of driving capabilities of the miniature self-driving car and a human driver.Dissertation/ThesisMasters Thesis Electrical Engineering 201

Lane Detection For Automatic Cars

The first stage in developing an autonomous car is the lane detection system. To help us identify lanes, we've borrowed a pair of ready-made models. As a rule, these two models are very time-consuming and expensive to compute. To lessen the burden on the computer, we developed a technique called the "row anchor based" approach. The computational burden is reduced, and the no-visual-clue issue is addressed by using this technique. It is exceedingly challenging to identify lanes when we are unable to see them clearly, as occurs in inclement weather, when water is on the lanes, or when the lanes are not designated. No-visual-clue is the term for this kind of issue. ResNet-18, which is used for pretrained models, has been utilized. Because of this, velocity will rise. ResNet-34 is another option, but it is too resource-intensive for this particular project. Road detection from one image is used to locate the road in a picture so it can be used as a district in the automation of the driving system within the vehicles for moving the vehicle on the correct road given a picture captured from a camera attached to a vehicle moving on a road, which road may or may not be level, have clearly described edges, or have some previous acknowledged patterns thereon. Here, we apply techniques for vanishing point identification, Hough Transformation Space, area of interest detection, edge detection, and canny edge detection for road recognition to locate the road inside the picture acquired by the vehicle. To train our model to recognize the road in the fresh image processed by the car, we typically use hundreds of images of roads from different locations

Computer Vision System-On-Chip Designs for Intelligent Vehicles

Intelligent vehicle technologies are growing rapidly that can enhance road safety, improve transport efficiency, and aid driver operations through sensors and intelligence. Advanced driver assistance system (ADAS) is a common platform of intelligent vehicle technologies. Many sensors like LiDAR, radar, cameras have been deployed on intelligent vehicles. Among these sensors, optical cameras are most widely used due to their low costs and easy installation. However, most computer vision algorithms are complicated and computationally slow, making them difficult to be deployed on power constraint systems. This dissertation investigates several mainstream ADAS applications, and proposes corresponding efficient digital circuits implementations for these applications. This dissertation presents three ways of software / hardware algorithm division for three ADAS applications: lane detection, traffic sign classification, and traffic light detection. Using FPGA to offload critical parts of the algorithm, the entire computer vision system is able to run in real time while maintaining a low power consumption and a high detection rate. Catching up with the advent of deep learning in the field of computer vision, we also present two deep learning based hardware implementations on application specific integrated circuits (ASIC) to achieve even lower power consumption and higher accuracy. The real time lane detection system is implemented on Xilinx Zynq platform, which has a dual core ARM processor and FPGA fabric. The Xilinx Zynq platform integrates the software programmability of an ARM processor with the hardware programmability of an FPGA. For the lane detection task, the FPGA handles the majority of the task: region-of-interest extraction, edge detection, image binarization, and hough transform. After then, the ARM processor takes in hough transform results and highlights lanes using the hough peaks algorithm. The entire system is able to process 1080P video stream at a constant speed of 69.4 frames per second, realizing real time capability. An efficient system-on-chip (SOC) design which classifies up to 48 traffic signs in real time is presented in this dissertation. The traditional histogram of oriented gradients (HoG) and support vector machine (SVM) are proven to be very effective on traffic sign classification with an average accuracy rate of 93.77%. For traffic sign classification, the biggest challenge comes from the low execution efficiency of the HoG on embedded processors. By dividing the HoG algorithm into three fully pipelined stages, as well as leveraging extra on-chip memory to store intermediate results, we successfully achieved a throughput of 115.7 frames per second at 1080P resolution. The proposed generic HoG hardware implementation could also be used as an individual IP core by other computer vision systems. A real time traffic signal detection system is implemented to present an efficient hardware implementation of the traditional grass-fire blob detection. The traditional grass-fire blob detection method iterates the input image multiple times to calculate connected blobs. In digital circuits, five extra on-chip block memories are utilized to save intermediate results. By using additional memories, all connected blob information could be obtained through one-pass image traverse. The proposed hardware friendly blob detection can run at 72.4 frames per second with 1080P video input. Applying HoG + SVM as feature extractor and classifier, 92.11% recall rate and 99.29% precision rate are obtained on red lights, and 94.44% recall rate and 98.27% precision rate on green lights. Nowadays, convolutional neural network (CNN) is revolutionizing computer vision due to learnable layer by layer feature extraction. However, when coming into inference, CNNs are usually slow to train and slow to execute. In this dissertation, we studied the implementation of principal component analysis based network (PCANet), which strikes a balance between algorithm robustness and computational complexity. Compared to a regular CNN, the PCANet only needs one iteration training, and typically at most has a few tens convolutions on a single layer. Compared to hand-crafted features extraction methods, the PCANet algorithm well reflects the variance in the training dataset and can better adapt to difficult conditions. The PCANet algorithm achieves accuracy rates of 96.8% and 93.1% on road marking detection and traffic light detection, respectively. Implementing in Synopsys 32nm process technology, the proposed chip can classify 724,743 32-by-32 image candidates in one second, with only 0.5 watt power consumption. In this dissertation, binary neural network (BNN) is adopted as a potential detector for intelligent vehicles. The BNN constrains all activations and weights to be +1 or -1. Compared to a CNN with the same network configuration, the BNN achieves 50 times better resource usage with only 1% - 2% accuracy loss. Taking car detection and pedestrian detection as examples, the BNN achieves an average accuracy rate of over 95%. Furthermore, a BNN accelerator implemented in Synopsys 32nm process technology is presented in our work. The elastic architecture of the BNN accelerator makes it able to process any number of convolutional layers with high throughput. The BNN accelerator only consumes 0.6 watt and doesn\u27t rely on external memory for storage