Chowdhury, Sohini Roy

Jonasson, Mats

Muppirisetty, Srikar

Ohlsson, Niklas

Zhao, Minming

English

Chalmers Research

Automated Road Friction Automated Road Friction Estimation using Car-sensor Suite: Machine Learning ApproachAssociate Professor, Tekn. Dr.Technical Expert Vehicle Dynamics and Motion ControlMats JonassonVolvo Cars, SwedenNiklas Ohlsson, Volvo Cars, SwedenProf. Sohini Roy Chowdhury, Volvo Cars, USDr. Srikar Muppirisetty, Volvo Cars, SwedenZhao Minming, Volvo Cars, USContentSweden 15 Jan 2013More than 10% of all accidents occurbecause of slippery conditionsIn the US: yearly approx 500 000 accidents of which 1800 are deadlyWhy road friction is important ?Assumed  frictionTrue frictionHigh(dry asphalt)Low(snow)Low (snow) High (dry asphalt)• False slippery warnings• Vehicle will drive unacceptably slowVehicle speed is adapted to frictionVehicle speed is adapted to friction• Vehicle speed is too high• High frequency of accidentsFriction =“grip” between tyre and road determines maximum acceleration and deceleration longitudinally and laterallyProblem formulationCan Machine Learning be an alternative approach for road friction estimation than state-of-the-art ?• Higher accuracy of friction estimate ?• Which features are useful ?• Possible to use a front-looking camera & machine learning to predict road surface condition ?State-of-the-art friction estimation methods• Contact based method• Just software• Requires sliding in tyre contact patch• Requires large accelerations Use an optical measurement device Use the tyre as a sensor with an internal model of the force-slip tyre characteristics (referred to as “baseline”)• Contact-less method• Requires extra hardware• Identifies just the surface• Sensitive to dirt contaminationWhy using Machine Learning ?An advanced driver could experience friction, but we do not fully understand the physicsUse the tyre as a sensor require valid physical tyre models -> the tyre is too complex to model with high validityWe have access to >100 000 km data, annotated by hand with “true friction”Vehicle data used to develop the algorithm• In total 86 features, most of them collected through CAN and flexray• Sensor features from:• Inertia measurement unit, Wheel speed sensors (Veh dyn sensors)• Brake system, Propulsive system and Steering system (Actuator systems)• Ambient thermometer and GPS (Environmental sensors)• Vehicle motion state estimator • Features manually noted as metadata: • tyre type• road type (asphalt, gravel,...)• surface type (wet, dry,...)• true friction [0,1]Feature rankingReduce complexity by reducing the number of features, conceptually by:1. Remove features with low correlation to true friction2. Remove redundant featuresMethods applied:1. Selecting features with more than 20% correlation2. Selecting features with more than 1% F-score 3. Minimum Redundancy Maximum Relevance (mRMR)4. Engineering experience33 features were finally selected from 86Top 5 featuresGreen= features selected by all 3 methodsYellow= features selected by 2 of 3 methodsRed=      features selected by 1 of 3  methods Correlation F-score mRMRGPSposLatitude AmbTemp RoadTypeAmbTemp SurfaceType AmbTempRoadType GPSposLatitude GPSposAltitudeSurfaceType GPSdop WheelSpeedGPSdop RoadType StabPtMaxModeFeature and correlation to frictionCorrelation to true frictionFeatures 1...86Temperature, GPS, vehicle speed, surface and road type are important features for friction estimationSurface & road type are not available in the sensor suite ->important to use a new sensorTemperatureGPSVehicle speedSurface, Road TypeNeural Network and Random ForestResult for 33 features excluding Tyre, Surface and Road type:Trainingtime (s)Prediction time (s)Testing accuracy (%)Neural Network (Multiclass, 1 hidden layer, 60 neurons) High Low 83Random Forest (Decision tree, multi class) Low Low 85Random Forest (RF) has been selected as the best method<1 s<3 minTraining has been done with an Intel i7, 2.7GHz Laptop with 8GB memoryConfusion matrix and 11 classes of friction0 1 2 3 4 5 6 7 8 9 10012 1 %3 12%4 1 %5678910Estimated frictionTrue frictionFalse Positive (false slippery warnings)False Negative (missed warnings)Accuracy=(TP+TN)/(TP+TN+FP+FN)Accuracy FP FNBaseline 58% 14% 28%Random Forest85% 5% 17%Motives to introduce a camera as a sensorRoad and surface type are both top feature and contributes to increase accuracyThe hypothesis was raised if a front-looking  can be used to predict the road conditionCamera is not today used for friction estimation at Volvo CarsThe envisioned structure adding a camera• Train Data: 40-100 frames per video (70%: random 3750 images. Interval between video frames>=1 second)• Test Data: 40 frames per video (30%: random 1550 images. Interval between video frames>=1 second)Image based road surface condition predictionDry Asphalt: Class 1 Slush: Class 3 Snow/Ice: Class 4Wet/Water: Class 2Proposed steps1. Extract GIST features from image=> Identify horizon2. Partition Image to : I. Below Horizon (Drivable Surface + Surroundings)II. Above Horizon (Surroundings + Sky)3. Extract HSV plane image4. Divide the two sub images into four by detecting the surroundings 17R1: Above Horizon: Surroundings R2: Above Horizon: SkyR3: Below Horizon: Drivable Surface R4: Below Horizon: SurroundingsHOG Features for TextureResults for road surface condition prediction3750 training images training time [s]training accuracy1550 testing images. test time [s] per imagetesting accuracy parametersCNN (3 Conv Layers) 6hr 97.63% 0.02325690 92.52% 71673984Squeezenet (AlexNet) 945.262 95.59% 0.00657387 92.26% 727428ML (NN, 60features, 60neurons) 1012.880 91.87% 0.37160000 91.23% 7200Neural Network was selected due to less parametersExamples mis-classified imagesGound truth: DryPredicted as WetGound truth: SnowPredicted as SlushGound truth: WetPredicted as IceChallengesTyres sensitivity Generalizability of machine learning algorithms to various regionsGround truthFriction is hard to measureUsing e.g. GPS will require intensive training of algorithm in all marketsMotion is generated in the tyre contact patch and tyres have individual characteristicsMachine learning is good only for scenarios it has been trained onConclusion• Temperature, GPS data, vehicle speed, and Road&Surface Type are sensitive features for friction estimation• The selected Machine Learning algorithm increases accuracy of friction estimate from 58% to 85% compared with the baseline.• The Machine Learning algorithm for road surface images achieves 92% accuracyend;

Automated Road Friction Estimation using Car-sensor Suite: MachineLearning Approach

https://research.chalmers.se/publication/516423/file/516423_Fulltext.pdf

Automated Road Friction Estimation using Car-sensor Suite: MachineLearning Approach

Abstract

Similar works

Full text

Available Versions

Chalmers Research