Vision-based vehicle detection and tracking in intelligent transportation system

This thesis aims to realize vision-based vehicle detection and tracking in the Intelligent Transportation System. First, it introduces the methods for vehicle detection and tracking. Next, it establishes the sensor fusion framework of the system, including dynamic model and sensor model. Then, it simulates the traffic scene at a crossroad by a driving simulator, where the research target is one single car, and the traffic scene is ideal. YOLO Neural Network is applied to the image sequence for vehicle detection. Kalman filter method, extended Kalman filter method, and particle filter method are utilized and compared for vehicle tracking. The Following part is the practical experiment where there are multiple vehicles at the same time, and the traffic scene is in real life with various interference factors. YOLO Neural Network combined with OpenCV is adopted to realize real-time vehicle detection. Kalman filter and extended Kalman filter are applied for vehicle tracking; an identification algorithm is proposed to solve the occlusion of the vehicles. The effects of process noise as well as measurement noise are analysed using variable-controlling approach. Additionally, perspective transformation is illustrated and implemented to transfer the coordinates from the image plane to the ground plane. If the vision-based vehicle detection and tracking can be realized and popularized in daily lives, vehicle information can be shared among infrastructures, vehicles, and users, so as to build interactions inside the Intelligent Transportation System

MME-EKF-Based Path-Tracking Control of Autonomous Vehicles Considering Input Saturation

This paper investigates the path-tracking control issue for autonomous ground vehicles with the integral sliding mode control (ISMC) considering the transient performance improvement. The path-tracking control is converted into the yaw stabilization problem, where the sideslip-angle compensation is adopted to reduce the steady-state errors, and then the yaw-rate reference is generated for the path-tracking purpose. The lateral velocity and roll angle are estimated with the measurement of the yaw rate and roll rate. Three contributions have been made in this paper: first, to enhance the estimation accuracy for the vehicle states in the presence of the parametric uncertainties caused by the lateral and roll dynamics, a robust extended Kalman filter is proposed based on the minimum model error algorithm; second, an improved adaptive radial basis function neural network (RBFNN) considering the approximation error adaptation is developed to compensate for the uncertainties caused by the vertical motion; third, the RBFNN and composite nonlinear feedback (CNF) based ISMC is developed to achieve the yaw stabilization and enhance the transient tracking performance considering the input saturation of the front steering angle. The overall stability is proved with Lyapunov function. Finally, the superiority of the developed control strategy is verified by comparing with the traditional CNF with high-fidelity CarSim-MATLAB simulations

Advances in Batteries, Battery Modeling, Battery Management System, Battery Thermal Management, SOC, SOH, and Charge/Discharge Characteristics in EV Applications

The second-generation hybrid and Electric Vehicles are currently leading the paradigm shift in the automobile industry, replacing conventional diesel and gasoline-powered vehicles. The Battery Management System is crucial in these electric vehicles and also essential for renewable energy storage systems. This review paper focuses on batteries and addresses concerns, difficulties, and solutions associated with them. It explores key technologies of Battery Management System, including battery modeling, state estimation, and battery charging. A thorough analysis of numerous battery models, including electric, thermal, and electro-thermal models, is provided in the article. Additionally, it surveys battery state estimations for a charge and health. Furthermore, the different battery charging approaches and optimization methods are discussed. The Battery Management System performs a wide range of tasks, including as monitoring voltage and current, estimating charge and discharge, equalizing and protecting the battery, managing temperature conditions, and managing battery data. It also looks at various cell balancing circuit types, current and voltage stressors, control reliability, power loss, efficiency, as well as their advantages and disadvantages. The paper also discusses research gaps in battery management systems.publishedVersio

Multi-Sensor Data Fusion for Travel Time Estimation

The importance of travel time estimation has increased due to the central role it plays in a number of emerging intelligent transport systems and services including Advanced Traveller Information Systems (ATIS), Urban Traffic Control (UTC), Dynamic Route Guidance (DRG), Active Traffic Management (ATM), and network performance monitoring. Along with the emerging of new sensor technologies, the much greater volumes of near real time data provided by these new sensor systems create opportunities for significant improvement in travel time estimation. Data fusion as a recent technique leads to a promising solution to this problem. This thesis presents the development and testing of new methods of multi-sensor data fusion for the accurate, reliable and robust estimation of travel time. This thesis reviews the state-of-art data fusion approaches and its application in transport domain, and discusses both of opportunities and challenging of applying data fusion into travel time estimation in a heterogeneous real time data environment. For a particular England highway scenario where ILDs and ANPR data are largely available, a simple but practical fusion method is proposed to estimate the travel time based on a novel relationship between space-mean-speed and time-mean-speed. In developing a general fusion framework which is able to fuse ILDs, GPS and ANPR data, the Kalman filter is identified as the most appropriate fundamental fusion technique upon which to construct the required framework. This is based both on the ability of the Kalman filter to flexibly accommodate well-established traffic flow models which describe the internal physical relation between the observed variables and objective estimates and on its ability to integrate and propagate in a consistent fashion the uncertainty associated with different data sources. Although the standard linear Kalman filter has been used for multi-sensor travel time estimation in the previous research, the novelty of this research is to develop a nonlinear Kalman filter (EKF and UKF) fusion framework which improves the estimation performance over those methods based on the linear Kalman filter. This proposed framework is validated by both of simulation and real-world scenarios, and is demonstrated the effectiveness of estimating travel time by fusing multi-sensor sources

Deep Learning Assisted Intelligent Visual and Vehicle Tracking Systems

Sensor fusion and tracking is the ability to bring together measurements from multiple sensors of the current and past time to estimate the current state of a system. The resulting state estimate is more accurate compared with the direct sensor measurement because it balances between the state prediction based on the assumed motion model and the noisy sensor measurement. Systems can then use the information provided by the sensor fusion and tracking process to support more-intelligent actions and achieve autonomy in a system like an autonomous vehicle. In the past, widely used sensor data are structured, which can be directly used in the tracking system, e.g., distance, temperature, acceleration, and force. The measurements\u27 uncertainty can be estimated from experiments. However, currently, a large number of unstructured data sources can be generated from sensors such as cameras and LiDAR sensors, which bring new challenges to the fusion and tracking system. The traditional algorithm cannot directly use these unstructured data, and it needs another method or process to “understand” them first. For example, if a system tries to track a particular person in a video sequence, it needs to understand where the person is in the first place. However, the traditional tracking method cannot finish such a task. The measurement model for unstructured data is usually difficult to construct. Deep learning techniques provide promising solutions to this type of problem. A deep learning method can learn and understand the unstructured data to accomplish tasks such as object detection in images, object localization in LiDAR point clouds, and driver behavior prediction from the current traffic conditions. Deep-learning architectures such as deep neural networks, deep belief networks, recurrent neural networks, and convolutional neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, and machine translation, where they have produced results comparable with human expert performance. How to incorporate information obtained via deep learning into our tracking system is one of the topics of this dissertation. Another challenging task is using learning methods to improve a tracking filter\u27s performance. In a tracking system, many manually tuned system parameters affect the tracking performance, e.g., the process noise covariance and measurement noise covariance in a Kalman Filter (KF). These parameters used to be estimated by running the tracking algorithm several times and selecting the one that gives the optimal performance. How to learn the system parameters automatically from data, and how to use machine learning techniques directly to provide useful information to the tracking systems are critical to the proposed tracking system. The proposed research on the intelligent tracking system has two objectives. The first objective is to make a visual tracking filter smart enough to understand unstructured data sources. The second objective is to apply learning algorithms to improve a tracking filter\u27s performance. The goal is to develop an intelligent tracking system that can understand the unstructured data and use the data to improve itself

State Estimation in Lithium-ion Batteries Using Pulse Perturbation and Feedforward Neural Networks

Predicting battery stored charge, available capacity, and peak power quickly and accurately is important for understanding pack performance and stability. It is proposed that a feedforward neural network (FNN) can estimate this information using cell voltage response to an injected current. Voltage response varies with the internal chemistry, represented by charge, capacity, and impedance. These characteristics are quantified here using state of charge (SoC), state of energy (SoE), and state of power (SoP). Cell response data is collected for various states at constant temperature, resulting in 234 unique voltage responses for training and evaluating the FNN. Training is performed using 3 distinct variations on the data: (1) the full voltage response, (2) individual portions of the response, such as charging or relaxation periods, and (3) fractions of the charge and discharge periods ranging from one-half to a single open-circuit voltage measurement. Using the full response, the average mean absolute error (MAE) is 0.0057 for SoE estimation. The average MAE is below 0.0080 for SoC and SoP estimation. The results for pulse portions show that Charge-rest or Discharge-rest responses perform almost as well as the full pulse. This may inform future pulse design for further optimization. The results for pulse fractions show that error increases as the amount of input data decreases, which validates the hypothesis that pulse perturbation yields high performance in FNN. The technique can be expanded to other temperatures, with potential for estimation of other states, and even degradation mechanisms. Estimation requires 3 minutes of voltage and current data, with no charging history needed and low computational complexity. The proposed method is thus suitable for development of advanced battery management systems in electric vehicles

The Development of the Digital Twin Platform for Smart Mobility Systems With High-Resolution 3D Data

69A3551847102This project develops the main modules and algorithm models for the digital twin platform for a smart mobility testing ground currently under construction. LiDAR (Line Detection And Ranging)-sensor-based object detection and 3D infrastructure modeling modules are developed and tested in the project. The developed digital twin model is pilot tested to conduct near-miss analysis at the intersections of the DataCity Smart Mobility Testing Ground in New Brunswick, NJ

RISE-Based Integrated Motion Control of Autonomous Ground Vehicles With Asymptotic Prescribed Performance

This article investigates the integrated lane-keeping and roll control for autonomous ground vehicles (AGVs) considering the transient performance and system disturbances. The robust integral of the sign of error (RISE) control strategy is proposed to achieve the lane-keeping control purpose with rollover prevention, by guaranteeing the asymptotic stability of the closed-loop system, attenuating systematic disturbances, and maintaining the controlled states within the prescribed performance boundaries. Three contributions have been made in this article: 1) a new prescribed performance function (PPF) that does not require accurate initial errors is proposed to guarantee the tracking errors restricted within the predefined asymptotic boundaries; 2) a modified neural network (NN) estimator which requires fewer adaptively updated parameters is proposed to approximate the unknown vertical dynamics; and 3) the improved RISE control based on PPF is proposed to achieve the integrated control objective, which analytically guarantees both the controller continuity and closed-loop system asymptotic stability by integrating the signum error function. The overall system stability is proved with the Lyapunov function. The controller effectiveness and robustness are finally verified by comparative simulations using two representative driving maneuvers, based on the high-fidelity CarSim-Simulink simulation

Model based fault detection and isolation approach for actuator and sensor faults in a UAV

Thesis (MEng)--Stellenbosch University, 2021.ENGLISH ABSTRACT: This thesis presents the design and validation of model-based fault detection and isolation (FDI) approach for unmanned aerial vehicles (UAV). In safety-critical sys- tems such as chemical, nuclear plants and passenger aircraft, FDI is typically founded on hardware redundancy. In hardware redundancy, multiple actuators are spatially distributed to localise faults quickly, and sensor measurements are compared for consistency. The primary drawback with hardware redundancy is the increased installation complexity, weight, and costs. With modern computing technologies, model-based FDI offers a cost-effective, iterative and efficient FDI design process, verifiable with high fidelity computer-aided simulation (CAS). This thesis investigates the application of the Two-Stage Kalman filter (TSKF) to the problem of FDI. The TSKF solves the main deficiencies faced with the aug- mented state Kalman filter (ASKF), namely, numerical instability in ill-conditioned systems, and computational inefficiency where large parameter vectors are aug- mented. The TSKF approach utilises two parallel reduced-order KFs to estimate the system state and the parameter vectors separately. The UAV’s two rudders are not "isolable" because they produce identical moments. A novel active FDI (AFDI) method is proposed to isolate rudder actuator faults. The FDI displays high noise sensitivity under the evere Dryden turbulence model, resulting in high false detection and missed detection rates. A novel adap- tive technique is proposed to improve the robustness and sensitivity of the FDI. Unlike most methods which rely on a single scaling factor, the proposed adaptation technique employs multiple factors to weight the spread of fault parameter covari- ance matrix in the direction of flow of information, resulting in selective adaptation. Fault parameter variations are nonuniform in time and space. A static alarm threshold will induce high false alarms or missed alarms when set to low or too high, respectively. A novel adaptive threshold based on the normalised innovation squared (NIS) is proposed. A Monte Carlo campaign is carried out to validate the FDI while fault-sizes, the aircraft’s physical parameters, and disturbances are scat- tered, each with a distinct mean dispersion. The proposed strategy exhibits high robustness to noise and sensitivity to faults which indicates a reliable FDI.AFRIKAANSE OPSOMMING: Hierdie tesis beskryf die ontwerp en validering van ‘n model-gebaseerde foutop- sporing en isolasie (“fault deteciton and isolation (FDI)”) tegniek vir onbemande lugvoertuie (“unmanned aerial vehicles (UAVs)”). In veiligheidskritieke stelsels soos chemiese aanlegte, kernkragaanlegte, en passasiersvliegtuie, word FDI gewoon- lik gebaseer op hardeware-oortolligheid. Vir hardeware-oortolligheid word verskeie aktueerders ruimtelik versprei om foute vinnig op te spoor, en sensormetings word vergelyk vir ooreenstemming. Die primêre nadeel van hardeware-oortolligheid is die verhoogde installasie-kompleksiteit, gewig en koste. Met moderne rekenaarteg- nologieë bied model-gebaseerde FDI ’n koste-effektiewe, iteratiewe en doeltref-fende FDI-ontwerpproses met ‘n hoë betroubaarheid wat bevestig kan word met rekenaargesteunde simulasie. Hierdie tesis ondersoek die toepassing van die twee-stadium Kalman filter (“two- stage Kalman filter (TSKF)”) op die probleem van FDI. Die TSKF los die belangrik- ste tekortkominge van die uitgebredie-toestand Kalman-filter (“augmented state Kalman filter (ASKF)”) op, naamlik numeriese onstabiliteit in swak gekondisioneerde stelsels, en berekeningsondoeltreffendheid waar groot parametervektore bygevoeg word. Die TSKF-benadering gebruik twee parallelle Kalman filters met vermin- derde orde om die stelseltoestand en die parametervektore afsonderlik af te skat. Die UAV se twee roere (“rudders”) is egter nie “isoleerbaar” nie omdat dit hulle identiese draaimoment veroorsaak. ’n Nuwe aktiewe FDI-metode (AFDI) word voorgestel om die roeraktueerderfoute te isoleer. Die FDI vertoon hoë sensitiwiteit vir geraas vanaf erge turbulensie soos gemod- elleer deur die Dryden-turbulensie-model, wat lei tot ‘n groot aantal vals deteksies en gemiste deteksies. ’n Nuwe aanpassingstegniek word voorgestel om die robu- ustheid en sensitiwiteit van die FDI te verbeter. Anders as die meeste metodes wat op een enkele skaalfaktor staatmaak, gebruik die voorgestelde aanpassingstegniek verskeie faktore om die verspreiding van die foutparameterkovariansiematriks in die rigting van informasievloei te weeg, wat lei tot selektiewe aanpassing. Foutparametervariasies is nie eenvormig in tyd of ruimte nie. ’n Statiese alar- mdrempel sal hoë vals deteksies of gemiste deteksies veroorsaak as dit onderskei-delik óf te laag óf te hoog gestel is. ’n Nuwe aanpassingsdrempel wat gebaseer is op die genormaliseerde innovasie kwadraat (NIS) word voorgestel. ’n Monte Carlo simulasieveldtog is uitgevoer om die FDI te toets met die foutgroottes, die fisiese parameters van die vliegtuig, en die steurings lukraak gevarieer elk met ’n duide- like gemiddelde verspreiding. Die voorgestelde strategie vertoon ’n hoë robuus- theid vir geraas en sensitiwiteit vir foute, wat dui op ’n betroubare FDI