Time-Domain Data Fusion Using Weighted Evidence and Dempster–Shafer Combination Rule: Application in Object Classification

To apply data fusion in time-domain based on Dempster–Shafer (DS) combination rule, an 8-step algorithm with novel entropy function is proposed. The 8-step algorithm is applied to time-domain to achieve the sequential combination of time-domain data. Simulation results showed that this method is successful in capturing the changes (dynamic behavior) in time-domain object classification. This method also showed better anti-disturbing ability and transition property compared to other methods available in the literature. As an example, a convolution neural network (CNN) is trained to classify three different types of weeds. Precision and recall from confusion matrix of the CNN are used to update basic probability assignment (BPA) which captures the classification uncertainty. Real data of classified weeds from a single sensor is used test time-domain data fusion. The proposed method is successful in filtering noise (reduce sudden changes—smoother curves) and fusing conflicting information from the video feed. Performance of the algorithm can be adjusted between robustness and fast-response using a tuning parameter which is number of time-steps(ts)

Paradox Elimination in Dempster–Shafer Combination Rule with Novel Entropy Function: Application in Decision-Level Multi-Sensor Fusion

Multi-sensor data fusion technology in an important tool in building decision-making applications. Modified Dempster–Shafer (DS) evidence theory can handle conflicting sensor inputs and can be applied without any prior information. As a result, DS-based information fusion is very popular in decision-making applications, but original DS theory produces counterintuitive results when combining highly conflicting evidences from multiple sensors. An effective algorithm offering fusion of highly conflicting information in spatial domain is not widely reported in the literature. In this paper, a successful fusion algorithm is proposed which addresses these limitations of the original Dempster–Shafer (DS) framework. A novel entropy function is proposed based on Shannon entropy, which is better at capturing uncertainties compared to Shannon and Deng entropy. An 8-step algorithm has been developed which can eliminate the inherent paradoxes of classical DS theory. Multiple examples are presented to show that the proposed method is effective in handling conflicting information in spatial domain. Simulation results showed that the proposed algorithm has competitive convergence rate and accuracy compared to other methods presented in the literature

Improving the Robustness of Object Detection Through a Multi-Camera–Based Fusion Algorithm Using Fuzzy Logic

A single camera creates a bounding box (BB) for the detected object with certain accuracy through a convolutional neural network (CNN). However, a single RGB camera may not be able to capture the actual object within the BB even if the CNN detector accuracy is high for the object. In this research, we present a solution to this limitation through the usage of multiple cameras, projective transformation, and a fuzzy logic–based fusion. The proposed algorithm generates a “confidence score” for each frame to check the trustworthiness of the BB generated by the CNN detector. As a first step toward this solution, we created a two-camera setup to detect objects. Agricultural weed is used as objects to be detected. A CNN detector generates BB for each camera when weed is present. Then a projective transformation is used to project one camera’s image plane to another camera’s image plane. The intersect over union (IOU) overlap of the BB is computed when objects are detected correctly. Four different scenarios are generated based on how far the object is from the multi-camera setup, and IOU overlap is calculated for each scenario (ground truth). When objects are detected correctly and bounding boxes are at correct distance, the IOU overlap value should be close to the ground truth IOU overlap value. On the other hand, the IOU overlap value should differ if BBs are at incorrect positions. Mamdani fuzzy rules are generated using this reasoning, and three different confidence scores (“high,” “ok,” and “low”) are given to each frame based on accuracy and position of BBs. The proposed algorithm was then tested under different conditions to check its validity. The confidence score of the proposed fuzzy system for three different scenarios supports the hypothesis that the multi-camera–based fusion algorithm improved the overall robustness of the detection system

Novel Entropy Function Based Multi-Sensor Fusion in Space and Time Domain: Application in Autonomous Agricultural Robot

How can we transform an agricultural vehicle into an autonomous weeding robot? A robot that can run autonomously through a vegetable field, classify multiple types of weeds from real-time video feed and then spray specific herbicides based of previously classified weeds. In this research, we answer some of the theoretical and practical challenges regarding the transformation of an agricultural vehicle into an autonomous weeding robot. First, we propose a solution for real-time crop row detection from autonomous navigation of agricultural vehicle using domain knowledge and unsupervised machine learning based approach. We implement projective transformation to transform camera image plane to an image plane exactly at the top of the crop rows, so that parallel crop rows remain parallel. Then we use color based segmentation to differentiate crop and weed pixels from background. We implement hierarchical density-based spatial clustering of applications with noise (HDBSCAN) clustering algorithm to differentiate between the crop row clusters and weed clusters. Finally we use Random sample consensus (RANSAC) for robust line fitting through the detected crop row clusters. We test our algorithm against four different well established methods for crop row detection in-terms of processing time and accuracy. Our proposed method, Clustering Algorithm based RObust LIne Fitting (CAROLIF), shows significantly better accuracy compared to three other methods with average intersect over union (IoU) value of 73%. We also test our algorithm on a video taken from an agricultural vehicle at a corn field in Indiana. CAROLIF shows promising results under lighting variation, vibration and unusual crop-weed growth. Then we propose a robust weed classification system based on convolutional neural network (CNN) and novel decision-level evidence-based multi-sensor fusion algorithm. We create a small dataset of three different weeds (Giant ragweed, Pigweed and Cocklebur) commonly available in corn fields. We train three different CNN architectures on our dataset. Based on classification accuracy and inference time, we choose VGG16 with transfer learning architecture for real-time weed classification. To create a robust and stable weed classification pipeline, a multi-sensor fusion algorithm based on Dempster-Shafer (DS) evidence theory with a novel entropy function is proposed. The proposed novel entropy function is inspired from Shannon and Deng entropy but it shows better results at understanding uncertainties in certain scenarios, compared to Shannon and Deng entropy, under DS framework. Our proposed algorithm has two advantages compared to other sensor fusion algorithms. First, it can be applied to both space and time domain to fuse results from multiple sensors and create more robust results. Secondly, it can detect which sensor is faulty in the sensors array and compensate for the faulty sensor by giving it lower weight at real-time. Our proposed algorithm calculates the evidence distance from each sensor and determines if one sensor agrees or disagrees with another. Then it rewards the sensors which agrees with another according to their information quality which is calculated using our novel entropy function. The proposed algorithm can combine highly conflicting evidences from multiple sensors and overcomes the limitation of original DS combination rule. After testing our algorithm with real and simulation data, it shows better convergence rate, anti-disturbing ability and transition property compared to other methods available from open literature. Finally, we present a fuzzy-logic based approach to measure the confidence of the detected object’s bounding-box (BB) position from a CNN detector. The CNN detector gives us the position of BB with percentage accuracy of the object inside the BB on each image plane. But how do we know for sure that the position of the BB is correct? When we are detecting an object using multiple cameras, the position of the BB on the camera image plane may appear in different places based on the detection accuracy and the position of the cameras. But in 3D space, the object is at the exact same position for both cameras. We use this relation between the camera image planes to create a fuzzy-fusion system which will calculate the confidence value of detection. Based on the fuzzy-rules and accuracy of BB position, this system gives us confidence values at three different stages (‘Low’, ‘OK’ and ‘High’). This proposed system is successful at giving correct confidence score for scenarios where objects are correctly detected, objects are partially detected and objects are incorrectly detected

Time-Domain Data Fusion Using Weighted Evidence and Dempster–Shafer Combination Rule: Application in Object Classification

To apply data fusion in time-domain based on Dempster–Shafer (DS) combination rule, an 8-step algorithm with novel entropy function is proposed. The 8-step algorithm is applied to time-domain to achieve the sequential combination of time-domain data. Simulation results showed that this method is successful in capturing the changes (dynamic behavior) in time-domain object classification. This method also showed better anti-disturbing ability and transition property compared to other methods available in the literature. As an example, a convolution neural network (CNN) is trained to classify three different types of weeds. Precision and recall from confusion matrix of the CNN are used to update basic probability assignment (BPA) which captures the classification uncertainty. Real data of classified weeds from a single sensor is used test time-domain data fusion. The proposed method is successful in filtering noise (reduce sudden changes—smoother curves) and fusing conflicting information from the video feed. Performance of the algorithm can be adjusted between robustness and fast-response using a tuning parameter which is number of time-steps( t s )

Three-dimensional transient numerical study of hot-jet ignition of methane-hydrogen blends in a constant-volume combustor

Ignition by a jet of hot combustion product gas injected into a premixed combustible mixture from a separate pre-chamber is a complex phenomenon with jet penetration, vortex generation, flame and shock propagation and interaction. It has been considered a useful approach for lean, low-NO x combustion for automotive engines, pulsed detonation engines and wave rotor combustors. The hot-jet ignition constant-volume combustor (CVC) rig established at the Combustion and Propulsion Research Laboratory (CPRL) of the Purdue School of Engineering and Technology at Indiana University-Purdue University Indianapolis (IUPUI) is considered for numerical study. The CVC chamber contains stoichiometric methane-hydrogen blends, with pre-chamber being operated with slightly rich blends. Five operating and design parameters were investigated with respect to their effects on ignition timing. Different pre-chamber pressure (2, 4 and 6 bar), CVC chamber fuel blends (Fuel-A: 30% methane + 70% hydrogen and Fuel-B: 50% methane + 50% hydrogen by volume), active radicals in pre-chamber combusted products (H, OH, O and NO), CVC chamber temperature (298 K and 514 K) and pre-chamber traverse speed (0.983 m/s, 4.917 m/s and 13.112 m/s) are considered which span a range of fluid-dynamic mixing and chemical time scales. Ignition delay of the fuel-air mixture in the CVC chamber is investigated using a detailed mechanism with 21 species and 84 elementary reactions (DRM19). To speed up the kinetic process adaptive mesh refinement (AMR) based on velocity and temperature and multi-zone reaction technique is used. With 3D numerical simulations, the present work explains the effects of pre-chamber pressure, CVC chamber initial temperature and jet traverse speed on ignition for a specific set of fuels. An innovative post processing technique is developed to predict and understand the characteristics of ignition in 3D space and time. With the increase of pre-chamber pressure, ignition delay decreases for Fuel-A which is the relatively more reactive fuel blend. For Fuel-B which is relatively less reactive fuel blend, ignition occurs only for 2 bar pre-chamber pressure for centered stationary jet. Inclusion of active radicals in pre-chamber combusted product, decreases the ignition delay when compared with only the stable species in pre-chamber combusted product. The effects of shock-flame interaction on heat release rate is observed by studying flame surface area and vorticity changes. In general, shock-flame interaction increases heat release rate by increasing mixing (increase the amount of deposited vorticity on flame surface) and flame stretching. The heat release rate is found to be maximum just after fast-slow interaction. For Fuel-A, increasing jet traverse speed decreases the ignition delay for relatively higher pre-chamber pressures (6 and 4 bar). Only 6 bar pre-chamber pressure is considered for Fuel-B with three different pre-chamber traverse speeds. Fuel-B fails to ignite within the simulation time for all the traverse speeds. Higher initial CVC temperature (514 K) decreases the ignition delay for both fuels when compared with relatively lower initial CVC temperature (300 K). For initial temperature of 514 K, the ignition of Fuel-B is successful for all the pre-chamber pressures with lowest ignition delay observed for the intermediate 4 bar pre-chamber pressure. Fuel-A has the lowest ignition delay for 6 bar pre-chamber pressure. A specific range of pre-chamber combusted products mass fraction, CVC chamber fuel mass fraction and temperature are found at ignition point for Fuel-A which were liable for ignition initiation. The behavior of less reactive Fuel-B appears to me more complex at room temperature initial condition. No simple conclusions could be made about the range of pre-chamber and CVC chamber mass fractions at ignition point