Image and Information Fusion Experiments with a Software-Defined Multi-Spectral Imaging System for Aviation and Marine Sensor Networks

The availability of Internet, line-of-sight and satellite identification and surveillance information as well as low-power, low-cost embedded systems-on-a-chip and a wide range of visible to long-wave infrared cameras prompted Embry Riddle Aeronautical University to collaborate with the University of Alaska Arctic Domain Awareness Center (ADAC) in summer 2016 to prototype a camera system we call the SDMSI (Software-Defined Multi-spectral Imager). The concept for the camera system from the start has been to build a sensor node that is drop-in-place for simple roof, marine, pole-mount, or buoy-mounts. After several years of component testing, the integrated SDMSI is now being tested, first on a roof-mount at Embry Riddle Prescott. The roof-mount testing demonstrates simple installation for the high spatial, temporal and spectral resolution SDMSI. The goal is to define and develop software and systems technology to complement satellite remote sensing and human monitoring of key resources such as drones, aircraft and marine vessels in and around airports, roadways, marine ports and other critical infrastructure. The SDMSI was installed at Embry Riddle Prescott in fall 2016 and continuous recording of long-wave infrared and visible images have been assessed manually and compared to salient object detection to automatically record only frames containing objects of interest (e.g. aircraft and drones). It is imagined that ultimately users of the SDMSI can pair with it via wireless to browse salient images. Further, both ADS-B (Automatic Dependent Surveillance-Broadcast) and S-AIS (Satellite Automatic Identification System) data are envisioned to be used by the SDMSI to form expectations for observing in future tests. This paper presents the preliminary results of several experiments and compares human review with smart image processing in terms of the receiver-operator characteristic. The system design and software are open architecture, such that other researchers are encouraged to construct and participate in sharing results and networking identical or improved versions of the SDMSI for safety, security and drop-in-place scientific image sensor networking