Robust Deep Multi-Modal Sensor Fusion using Fusion Weight Regularization and Target Learning

Sensor fusion has wide applications in many domains including health care and autonomous systems. While the advent of deep learning has enabled promising multi-modal fusion of high-level features and end-to-end sensor fusion solutions, existing deep learning based sensor fusion techniques including deep gating architectures are not always resilient, leading to the issue of fusion weight inconsistency. We propose deep multi-modal sensor fusion architectures with enhanced robustness particularly under the presence of sensor failures. At the core of our gating architectures are fusion weight regularization and fusion target learning operating on auxiliary unimodal sensing networks appended to the main fusion model. The proposed regularized gating architectures outperform the existing deep learning architectures with and without gating under both clean and corrupted sensory inputs resulted from sensor failures. The demonstrated improvements are particularly pronounced when one or more multiple sensory modalities are corrupted.Comment: 8 page

Inspection System And Method For Bond Detection And Validation Of Surface Mount Devices Using Sensor Fusion And Active Perception

A hybrid surface mount component inspection system which includes both vision and infrared inspection techniques to determine the presence of surface mount components on a printed wiring board, and the quality of solder joints of surface mount components on printed wiring boards by using data level sensor fusion to combine data from two infrared sensors to obtain emissivity independent thermal signatures of solder joints, and using feature level sensor fusion with active perception to assemble and process inspection information from any number of sensors to determine characteristic feature sets of different defect classes to classify solder defects.Georgia Tech Research Corporatio

Kalman Filtering with Uncertain Process and Measurement Noise Covariances with Application to State Estimation in Sensor Networks

Distributed state estimation under uncertain process and measurement noise covariances is considered. An algorithm based on sensor fusion using Kalman filtering is investigated. It is shown that if the covariances are decomposed into a known nominal covariance plus an uncertainty term, then the uncertainty of the actual estimation error covariance for the Kalman filter grows linearly with the size of the uncertainty term. This result is extended to the sensor fusion scheme to give an upper bound on the actual error covariance for the fused state estimate. Examples are provided to illustrate how the theory can be applied in practice

Sensor Fusion and Fuzzy Logic for Stabilization System of Gimbal Camera on Hexacopter

Hexacopter has the ability to fly in the air can be used as an air monitoring system. To get the video or images that have good quality then it is used gimbal camera as a movement stabilizer and vibration damping. Stabilization System consists of two axes, x axis (roll) and y axis (pitch) and has a 2-axis camera gimbal controller which is have microcontroller ATMEGA 328 that serves to regulate the stability of the gimbal camera. The Input of this system is derived from accelerometer and gyroscope sensors that are within the sensor module MPU 6050, to determine the tilt position of hexacopter. The output of this sensor will be filtered first using complementary filters that serve to reduce noise of both sensors and complement advantages and disadvantages of each sensor. The output of this system is the movement of two brushless motors, brushless roll and pitch, that are controlled with Sugeno fuzzy logic method because it has a simple calculation so the response is faster and more suitable for real-time applications. From the case study with the data of the roll at by 35 º and pitch at by 17 º resulting PWM duty cycle value by -69.47% roll and pitch resulting PWM duty cycle value by -25.5 %, where (-) represents the direction of movement

System based on inertial sensors for behavioral monitoring of wildlife

Sensors Network is an integration of multiples sensors in a system to collect information about different environment variables. Monitoring systems allow us to determine the current state, to know its behavior and sometimes to predict what it is going to happen. This work presents a monitoring system for semi-wild animals that get their actions using an IMU (inertial measure unit) and a sensor fusion algorithm. Based on an ARM-CortexM4 microcontroller this system sends data using ZigBee technology of different sensor axis in two different operations modes: RAW (logging all information into a SD card) or RT (real-time operation). The sensor fusion algorithm improves both the precision and noise interferences.Junta de Andalucía P12-TIC-130