LiDAR and Camera Detection Fusion in a Real Time Industrial Multi-Sensor Collision Avoidance System

Collision avoidance is a critical task in many applications, such as ADAS (advanced driver-assistance systems), industrial automation and robotics. In an industrial automation setting, certain areas should be off limits to an automated vehicle for protection of people and high-valued assets. These areas can be quarantined by mapping (e.g., GPS) or via beacons that delineate a no-entry area. We propose a delineation method where the industrial vehicle utilizes a LiDAR {(Light Detection and Ranging)} and a single color camera to detect passive beacons and model-predictive control to stop the vehicle from entering a restricted space. The beacons are standard orange traffic cones with a highly reflective vertical pole attached. The LiDAR can readily detect these beacons, but suffers from false positives due to other reflective surfaces such as worker safety vests. Herein, we put forth a method for reducing false positive detection from the LiDAR by projecting the beacons in the camera imagery via a deep learning method and validating the detection using a neural network-learned projection from the camera to the LiDAR space. Experimental data collected at Mississippi State University's Center for Advanced Vehicular Systems (CAVS) shows the effectiveness of the proposed system in keeping the true detection while mitigating false positives.Comment: 34 page

An Investigation Of Six Poorly Described Close Visual Double Stars Using Speckle Interferometry

Continued observation of double stars is necessary for confirmation of binarity and to provide updates to astrometric data used to compute accurate binary orbital parameters, thereby more accurately informing stellar mass estimations – the critical parameter from which stellar models are derived. In October of 2013, six double stars from the Washington Double Star (WDS) catalog exhibiting close separations, as well as significant deviations from previously published orbits, were observed and imaged using the speckle interferometric technique on the 2.1-meter telescope at Kitt Peak National Observatory (KPNO) in Arizona. The observations of the six double stars occurred as part of large, collaborative, eight-night, student-learning-centered observing run organized by principal investigator Genet of California Polytechnic Institute. The run produced in total roughly 1000 raw speckle images for each of the more than 1000 double stars and single reference stars observed, resulting in a total database of 1.4 terabytes. The speckle images for the targets, including the six targets investigated in this thesis, were taken using a relatively low-cost, portable speckle interferometry camera system developed by Genet, the heart of which is a lightweight, high speed, high signal to noise ratio (SNR) Andor electron multiplying CCD (EMCCD) camera capable of exposures on the order of tens of milliseconds. Exposures of 10-20 milliseconds are faster than atmospheric coherence timescales, and allow for the implementation of the speckle interferometry – the obtainment of diffraction-limited image information of binary stars defined by the full aperture of the telescope from the autocorrelation and Fourier analysis of randomly distributed, isoplanatically correlated speckle pairs, which represent the diffraction-limited images of the associated coherence cells above and within the atmospheric area of the primary aperture (sub-apertures). Following the Oct. 2013 observing run, reduction and analysis of the speckle images for the six target binary stars (as well as five calibration binaries) and determination of the new astrometry was completed using the general purpose astrometry software program PlateSolve3 (PS3), written and developed by Rowe & Genet (2014). Using the new astrometric data derived from the Oct. 2013 2.1-meter speckle observations, the previously published United States Naval Observatory (USNO) orbital plots for the six target doubles were updated to reflect the new, and in some cases missing measurements. Target double star orbits were reevaluated in light of the updates in order to draw conclusions about the characteristics of each proposed binary system. In all six target cases, continued trends in significant astrometric deviations from published orbits and ephemerides have been demonstrated by the new observations, indicating the need for orbital revisions of these binaries. Analysis of systems WDS22357+5413, WDS02231+7021, and WDS06256+2227 indicate rectilinear rather than Keplerian motion, and are concluded to likely be optical doubles. As a result of this work, two observations of WDS05153+4710 were shown to be erroneous and have been scheduled to be removed from this binary’s WDS observational record (Mason, private communication, 2015). Complementary to the central goal of investigating the six target close visual double stars via speckle interferometry, the entire effort demonstrated the applicability and utilization of relatively low-cost portable speckle camera systems on large telescopes, as well as the value and advantages of student participation and contribution within the realm of a large-scale observing run at a major observatory and the resulting peer reviewed scientific works that follow

Application of unmanned aerial systems for high throughput phenotyping of large wheat breeding nurseries

Citation: Haghighattalab, A., Perez, L. G., Mondal, S., Singh, D., Schinstock, D., Rutkoski, J., . . . Poland, J. (2016). Application of unmanned aerial systems for high throughput phenotyping of large wheat breeding nurseries. Plant Methods, 12, 15. https://doi.org/10.1186/s13007-016-0134-6Background: Low cost unmanned aerial systems (UAS) have great potential for rapid proximal measurements of plants in agriculture. In the context of plant breeding and genetics, current approaches for phenotyping a large number of breeding lines under field conditions require substantial investments in time, cost, and labor. For field-based high-throughput phenotyping (HTP), UAS platforms can provide high-resolution measurements for small plot research, while enabling the rapid assessment of tens-of-thousands of field plots. The objective of this study was to complete a baseline assessment of the utility of UAS in assessment field trials as commonly implemented in wheat breeding programs. We developed a semi-automated image-processing pipeline to extract plot level data from UAS imagery. The image dataset was processed using a photogrammetric pipeline based on image orientation and radiometric calibration to produce orthomosaic images. We also examined the relationships between vegetation indices (VIs) extracted from high spatial resolution multispectral imagery collected with two different UAS systems (eBee Ag carrying MultiSpec 4C camera, and IRIS+ quadcopter carrying modified NIR Canon S100) and ground truth spectral data from hand-held spectroradiometer. Results: We found good correlation between the VIs obtained from UAS platforms and ground-truth measurements and observed high broad-sense heritability for VIs. We determined radiometric calibration methods developed for satellite imagery significantly improved the precision of VIs from the UAS. We observed VIs extracted from calibrated images of Canon S100 had a significantly higher correlation to the spectroradiometer (r = 0.76) than VIs from the MultiSpec 4C camera (r = 0.64). Their correlation to spectroradiometer readings was as high as or higher than repeated measurements with the spectroradiometer per se. Conclusion: The approaches described here for UAS imaging and extraction of proximal sensing data enable collection of HTP measurements on the scale and with the precision needed for powerful selection tools in plant breeding. Low-cost UAS platforms have great potential for use as a selection tool in plant breeding programs. In the scope of tools development, the pipeline developed in this study can be effectively employed for other UAS and also other crops planted in breeding nurseries

Smart environment monitoring through micro unmanned aerial vehicles

In recent years, the improvements of small-scale Unmanned Aerial Vehicles (UAVs) in terms of flight time, automatic control, and remote transmission are promoting the development of a wide range of practical applications. In aerial video surveillance, the monitoring of broad areas still has many challenges due to the achievement of different tasks in real-time, including mosaicking, change detection, and object detection. In this thesis work, a small-scale UAV based vision system to maintain regular surveillance over target areas is proposed. The system works in two modes. The first mode allows to monitor an area of interest by performing several flights. During the first flight, it creates an incremental geo-referenced mosaic of an area of interest and classifies all the known elements (e.g., persons) found on the ground by an improved Faster R-CNN architecture previously trained. In subsequent reconnaissance flights, the system searches for any changes (e.g., disappearance of persons) that may occur in the mosaic by a histogram equalization and RGB-Local Binary Pattern (RGB-LBP) based algorithm. If present, the mosaic is updated. The second mode, allows to perform a real-time classification by using, again, our improved Faster R-CNN model, useful for time-critical operations. Thanks to different design features, the system works in real-time and performs mosaicking and change detection tasks at low-altitude, thus allowing the classification even of small objects. The proposed system was tested by using the whole set of challenging video sequences contained in the UAV Mosaicking and Change Detection (UMCD) dataset and other public datasets. The evaluation of the system by well-known performance metrics has shown remarkable results in terms of mosaic creation and updating, as well as in terms of change detection and object detection

Semi-automated geomorphological mapping applied to landslide hazard analysis

Computer-assisted three-dimensional (3D) mapping using stereo and multi-image (“softcopy”) photogrammetry is shown to enhance the visual interpretation of geomorphology in steep terrain with the direct benefit of greater locational accuracy than traditional manual mapping. This would benefit multi-parameter correlations between terrain attributes and landslide distribution in both direct and indirect forms of landslide hazard assessment. Case studies involve synthetic models of a landslide, and field studies of a rock slope and steep undeveloped hillsides with both recently formed and partly degraded, old landslide scars. Diagnostic 3D morphology was generated semi-automatically both using a terrain-following cursor under stereo-viewing and from high resolution digital elevation models created using area-based image correlation, further processed with curvature algorithms. Laboratory-based studies quantify limitations of area-based image correlation for measurement of 3D points on planar surfaces with varying camera orientations. The accuracy of point measurement is shown to be non-linear with limiting conditions created by both narrow and wide camera angles and moderate obliquity of the target plane. Analysis of the results with the planar surface highlighted problems with the controlling parameters of the area-based image correlation process when used for generating DEMs from images obtained with a low-cost digital camera. Although the specific cause of the phase-wrapped image artefacts identified was not found, the procedure would form a suitable method for testing image correlation software, as these artefacts may not be obvious in DEMs of non-planar surfaces.Modelling of synthetic landslides shows that Fast Fourier Transforms are an efficient method for removing noise, as produced by errors in measurement of individual DEM points, enabling diagnostic morphological terrain elements to be extracted. Component landforms within landslides are complex entities and conversion of the automatically-defined morphology into geomorphology was only achieved with manual interpretation; however, this interpretation was facilitated by softcopy-driven stereo viewing of the morphological entities across the hillsides.In the final case study of a large landslide within a man-made slope, landslide displacements were measured using a photogrammetric model consisting of 79 images captured with a helicopter-borne, hand-held, small format digital camera. Displacement vectors and a thematic geomorphological map were superimposed over an animated, 3D photo-textured model to aid non-stereo visualisation and communication of results