Vision technology/algorithms for space robotics applications

The thrust of automation and robotics for space applications has been proposed for increased productivity, improved reliability, increased flexibility, higher safety, and for the performance of automating time-consuming tasks, increasing productivity/performance of crew-accomplished tasks, and performing tasks beyond the capability of the crew. This paper provides a review of efforts currently in progress in the area of robotic vision. Both systems and algorithms are discussed. The evolution of future vision/sensing is projected to include the fusion of multisensors ranging from microwave to optical with multimode capability to include position, attitude, recognition, and motion parameters. The key feature of the overall system design will be small size and weight, fast signal processing, robust algorithms, and accurate parameter determination. These aspects of vision/sensing are also discussed

Body shape-based biometric person recognition from mmW images

A growing interest has arisen in the security community for the use of millimeter waves in order to detect weapons and concealed objects. Also, the use of millimetre wave images has been proposed recently for biometric person recognition to overcome certain limitations of images acquired at visible frequencies. This paper proposes a biometric person recognition system based on shape information extracted from millimetre wave images. To this aim, we report experimental results using millimeter wave images with different body shape-based feature approaches: contour coordinates, shape contexts, Fourier descriptors and row and column profiles, using Dynamic Time Warping for matching. Results suggest the potential of performing person recognition through millimetre waves using only shape information, a functionality that could be easily integrated in the security scanners deployed in airportsThis work has been partially supported by project CogniMetrics TEC2015-70627-R (MINECO/FEDER), and the SPATEK network (TEC2015-68766-REDC

Millimetre wave person recognition: hand-crafted vs learned features

Imaging using millimeter waves (mmWs) has many advantages including ability to penetrate obscurants such as clothes and polymers. Although conceal weapon detection has been the predominant mmW imaging application, in this paper, we aim to gain some insight about the potential of using mmW images for person recognition. We report experimental results using the mmW TNO database consisting of 50 individuals based on both hand-crafted and learned features from Alexnet and VGG-face pretrained CNN models. Results suggest that: i) mmW torso region is more discriminative than mmW face and the entire body, ii) CNN features produce better results compared to hand-crafted features on mmW faces and the entire body, and iii) hand-crafted features slightly outperform CNN features on mmW torsoThis work has been partially supported by project CogniMetrics TEC2015-70627-R (MINECO/FEDER), and the SPATEK network (TEC2015-68766-REDC). E. GonzalezSosa is supported by a PhD scholarship from Universidad Autonoma de Madrid. Vishal M. Patel was partially supported by US Office of Naval Research (ONR) Grant YIP N00014-16-1-3134. Authors wish to thank also TNO for providing access to the databas

An electromagnetic imaging system for metallic object detection and classification

PhD ThesisElectromagnetic imaging currently plays a vital role in various disciplines, from engineering to medical applications and is based upon the characteristics of electromagnetic fields and their interaction with the properties of materials. The detection and characterisation of metallic objects which pose a threat to safety is of great interest in relation to public and homeland security worldwide. Inspections are conducted under the prerequisite that is divested of all metallic objects. These inspection conditions are problematic in terms of the disruption of the movement of people and produce a soft target for terrorist attack. Thus, there is a need for a new generation of detection systems and information technologies which can provide an enhanced characterisation and discrimination capabilities. This thesis proposes an automatic metallic object detection and classification system. Two related topics have been addressed: to design and implement a new metallic object detection system; and to develop an appropriate signal processing algorithm to classify the targeted signatures. The new detection system uses an array of sensors in conjunction with pulsed excitation. The contributions of this research can be summarised as follows: (1) investigating the possibility of using magneto-resistance sensors for metallic object detection; (2) evaluating the proposed system by generating a database consisting of 12 real handguns with more than 20 objects used in daily life; (3) extracted features from the system outcomes using four feature categories referring to the objects’ shape, material composition, time-frequency signal analysis and transient pulse response; and (4) applying two classification methods to classify the objects into threats and non-threats, giving a successful classification rate of more than 92% using the feature combination and classification framework of the new system. The study concludes that novel magnetic field imaging system and their signal outputs can be used to detect, identify and classify metallic objects. In comparison with conventional induction-based walk-through metal detectors, the magneto-resistance sensor array-based system shows great potential for object identification and discrimination. This novel system design and signal processing achievement may be able to produce significant improvements in automatic threat object detection and classification applications.Iraqi Cultural Attaché, Londo

Millimetre wave imaging for concealed target detection

PhDConcealed weapon detection (CWD) has been a hot topic as the concern about pub- lic safety increases. A variety of approaches for the detection of concealed objects on the human body based on earth magnetic ¯eld distortion, inductive magnetic ¯eld, acoustic and ultrasonic, electromagnetic resonance, MMW (millimetre wave), THz, Infrared, x-ray technologies have been suggested and developed. Among all of them, MMW holographic imaging is considered as a promising approach due to the relatively high penetration and high resolution that it can o®er. Typical concealed target detection methods are classi¯ed into 2 categories, the ¯rst one is a resonance based target identi¯cation technique, and the second one is an imaging based system. For the former, the complex natural resonance (CNR) frequencies associated with a certain target are extracted and used for identi¯cation, but this technique has an issue of high false alarm rate. The microwave/millimetre wave imaging systems can be categorized into two types: passive systems and active sys- tems. For the active microwave/millimetre wave imaging systems, the microwave holographic imaging approach was adopted in this thesis. Such a system can oper- ate at either a single frequency or multiple frequencies (wide band). An active, coherent, single frequency operation millimetre wave imaging system based on the theory of microwave holography was developed. Based on literature surveys and ¯rst hand experimental results, this thesis aims to provide system level parame- ter determination to aid the development of a target detection imager. The goal is approached step by step in 7 chapters, with topics and issues addressed rang- ing from reviewing the past work, ¯nding out the best candidate technology, i.e. the MMW holographic imaging combined with the resonance based target recog- i nition technique, the construction of the 94 GHz MMW holographic prototype imager, experimental trade-o® investigation of system parameters, imager per- formance evaluation, low pro¯le components and image enhancement techniques, feasibility investigation of resonance based technique, to system implementation based on the parameters and results achieved. The task set forth in the beginning is completed by coming up with an entire system design in the end.

mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar

Millimeter Wave (mmWave) Radar is gaining popularity as it can work in adverse environments like smoke, rain, snow, poor lighting, etc. Prior work has explored the possibility of reconstructing 3D skeletons or meshes from the noisy and sparse mmWave Radar signals. However, it is unclear how accurately we can reconstruct the 3D body from the mmWave signals across scenes and how it performs compared with cameras, which are important aspects needed to be considered when either using mmWave radars alone or combining them with cameras. To answer these questions, an automatic 3D body annotation system is first designed and built up with multiple sensors to collect a large-scale dataset. The dataset consists of synchronized and calibrated mmWave radar point clouds and RGB(D) images in different scenes and skeleton/mesh annotations for humans in the scenes. With this dataset, we train state-of-the-art methods with inputs from different sensors and test them in various scenarios. The results demonstrate that 1) despite the noise and sparsity of the generated point clouds, the mmWave radar can achieve better reconstruction accuracy than the RGB camera but worse than the depth camera; 2) the reconstruction from the mmWave radar is affected by adverse weather conditions moderately while the RGB(D) camera is severely affected. Further, analysis of the dataset and the results shadow insights on improving the reconstruction from the mmWave radar and the combination of signals from different sensors.Comment: ACM Multimedia 2022, Project Page: https://chen3110.github.io/mmbody/index.htm

SMURF: Spatial Multi-Representation Fusion for 3D Object Detection with 4D Imaging Radar

The 4D Millimeter wave (mmWave) radar is a promising technology for vehicle sensing due to its cost-effectiveness and operability in adverse weather conditions. However, the adoption of this technology has been hindered by sparsity and noise issues in radar point cloud data. This paper introduces spatial multi-representation fusion (SMURF), a novel approach to 3D object detection using a single 4D imaging radar. SMURF leverages multiple representations of radar detection points, including pillarization and density features of a multi-dimensional Gaussian mixture distribution through kernel density estimation (KDE). KDE effectively mitigates measurement inaccuracy caused by limited angular resolution and multi-path propagation of radar signals. Additionally, KDE helps alleviate point cloud sparsity by capturing density features. Experimental evaluations on View-of-Delft (VoD) and TJ4DRadSet datasets demonstrate the effectiveness and generalization ability of SMURF, outperforming recently proposed 4D imaging radar-based single-representation models. Moreover, while using 4D imaging radar only, SMURF still achieves comparable performance to the state-of-the-art 4D imaging radar and camera fusion-based method, with an increase of 1.22% in the mean average precision on bird's-eye view of TJ4DRadSet dataset and 1.32% in the 3D mean average precision on the entire annotated area of VoD dataset. Our proposed method demonstrates impressive inference time and addresses the challenges of real-time detection, with the inference time no more than 0.05 seconds for most scans on both datasets. This research highlights the benefits of 4D mmWave radar and is a strong benchmark for subsequent works regarding 3D object detection with 4D imaging radar

Applications of Computer Vision Technologies of Automated Crack Detection and Quantification for the Inspection of Civil Infrastructure Systems

Many components of existing civil infrastructure systems, such as road pavement, bridges, and buildings, are suffered from rapid aging, which require enormous nation\u27s resources from federal and state agencies to inspect and maintain them. Crack is one of important material and structural defects, which must be inspected not only for good maintenance of civil infrastructure with a high quality of safety and serviceability, but also for the opportunity to provide early warning against failure. Conventional human visual inspection is still considered as the primary inspection method. However, it is well established that human visual inspection is subjective and often inaccurate. In order to improve current manual visual inspection for crack detection and evaluation of civil infrastructure, this study explores the application of computer vision techniques as a non-destructive evaluation and testing (NDE&T) method for automated crack detection and quantification for different civil infrastructures. In this study, computer vision-based algorithms were developed and evaluated to deal with different situations of field inspection that inspectors could face with in crack detection and quantification. The depth, the distance between camera and object, is a necessary extrinsic parameter that has to be measured to quantify crack size since other parameters, such as focal length, resolution, and camera sensor size are intrinsic, which are usually known by camera manufacturers. Thus, computer vision techniques were evaluated with different crack inspection applications with constant and variable depths. For the fixed-depth applications, computer vision techniques were applied to two field studies, including 1) automated crack detection and quantification for road pavement using the Laser Road Imaging System (LRIS), and 2) automated crack detection on bridge cables surfaces, using a cable inspection robot. For the various-depth applications, two field studies were conducted, including 3) automated crack recognition and width measurement of concrete bridges\u27 cracks using a high-magnification telescopic lens, and 4) automated crack quantification and depth estimation using wearable glasses with stereovision cameras. From the realistic field applications of computer vision techniques, a novel self-adaptive image-processing algorithm was developed using a series of morphological transformations to connect fragmented crack pixels in digital images. The crack-defragmentation algorithm was evaluated with road pavement images. The results showed that the accuracy of automated crack detection, associated with artificial neural network classifier, was significantly improved by reducing both false positive and false negative. Using up to six crack features, including area, length, orientation, texture, intensity, and wheel-path location, crack detection accuracy was evaluated to find the optimal sets of crack features. Lab and field test results of different inspection applications show that proposed compute vision-based crack detection and quantification algorithms can detect and quantify cracks from different structures\u27 surface and depth. Some guidelines of applying computer vision techniques are also suggested for each crack inspection application

A Review of Indoor Millimeter Wave Device-based Localization and Device-free Sensing Technologies and Applications

The commercial availability of low-cost millimeter wave (mmWave) communication and radar devices is starting to improve the penetration of such technologies in consumer markets, paving the way for large-scale and dense deployments in fifth-generation (5G)-and-beyond as well as 6G networks. At the same time, pervasive mmWave access will enable device localization and device-free sensing with unprecedented accuracy, especially with respect to sub-6 GHz commercial-grade devices. This paper surveys the state of the art in device-based localization and device-free sensing using mmWave communication and radar devices, with a focus on indoor deployments. We first overview key concepts about mmWave signal propagation and system design. Then, we provide a detailed account of approaches and algorithms for localization and sensing enabled by mmWaves. We consider several dimensions in our analysis, including the main objectives, techniques, and performance of each work, whether each research reached some degree of implementation, and which hardware platforms were used for this purpose. We conclude by discussing that better algorithms for consumer-grade devices, data fusion methods for dense deployments, as well as an educated application of machine learning methods are promising, relevant and timely research directions.Comment: 43 pages, 13 figures. Accepted in IEEE Communications Surveys & Tutorials (IEEE COMST