Real-time small-object change detection from ground vehicles using a siamese convolutional neural network

Detecting changes in an uncontrolled environment using cameras mounted on a ground vehicle is critical for the detection of roadside Improvised Explosive Devices (IEDs). Hidden IEDs are often accompanied by visible markers, whose appearances are a priori unknown. Little work has been published on detecting unknown objects using deep learning. This article shows the feasibility of applying convolutional neural networks (CNNs) to predict the location of markers in real time, compared to an earlier reference recording. The authors investigate novel encoder–decoder Siamese CNN architectures and introduce a modified double-margin contrastive loss function, to achieve pixel-level change detection results. Their dataset consists of seven pairs of challenging real-world recordings, and they investigate augmentation with artificial object data. The proposed network architecture can compare two images of 1920 × 1440 pixels in 27 ms on an RTX Titan GPU and significantly outperforms state-of-the-art networks and algorithms on our dataset in terms of F-1 score by 0.28

Development and analysis of a real-time system for automated detection of improvised explosive device indicators from ground vehicles

We propose a real-time change detection system to be used as a vehicle-mounted early-warning system for indicators of improvised explosive devices. Within the context of military route clearance, the system automatically detects suspicious changes in the environment with respect to a previous patrol. For this purpose, historical images of the live scene are retrieved from a database and registered to the live image through 2.5-D view synthesis, using the three-dimensional (3-D) scene geometry acquired from a stereo camera. Changes are then found using local-area statistics in the CIE-Lab color space. A set of spatiotemporal filters is used to reject irrelevant alarms, resulting in a limited set of confident changes to be presented to the operator through an interactive graphical user interface. Next to the algorithmic contributions, we elaborate on the real-time design, featuring graphical processing units for the most time-consuming processing tasks, a pipelined architecture to increase the system throughput, and we split the system into a live and offline processing chain. This way, real-time change detection at 3.5 fps is achieved on images of 1920 × 1440 pixels. Finally, an extensive system validation featuring realistic experiments shows promising detection capabilities and robustness to, e.g., lateral displacements of up to 6 m