Integrating Millimeter Wave Radar with a Monocular Vision Sensor for On-Road Obstacle Detection Applications

This paper presents a systematic scheme for fusing millimeter wave (MMW) radar and a monocular vision sensor for on-road obstacle detection. As a whole, a three-level fusion strategy based on visual attention mechanism and driver’s visual consciousness is provided for MMW radar and monocular vision fusion so as to obtain better comprehensive performance. Then an experimental method for radar-vision point alignment for easy operation with no reflection intensity of radar and special tool requirements is put forward. Furthermore, a region searching approach for potential target detection is derived in order to decrease the image processing time. An adaptive thresholding algorithm based on a new understanding of shadows in the image is adopted for obstacle detection, and edge detection is used to assist in determining the boundary of obstacles. The proposed fusion approach is verified through real experimental examples of on-road vehicle/pedestrian detection. In the end, the experimental results show that the proposed method is simple and feasible

Sensors in Unmanned Robotic Vehicle

Unmanned tracked vehicles are developed for deployment in dangerous zones that are notsafe for human existence. These vehicles are to be fitted with various sensors for safe manoeuvre.Wide range of sensors for vehicle control, vision, and navigation are employed. The main purposeof the sensors is to infer the intended parameter precisely for further utilisation. Software isinseparable part of the sensors and plays major role in scaling, noise reduction, and fusion.Sensor fusion is normally adapted to enhance the decision-making. Vehicle location  andorientation can be sensed through global positioning system, accelerometer, gyroscope, andcompass. The unmanned vehicle can be navigated with the help of CCD camera, radar, lidar,ultrasonic sensor, piezoelectric sensor, microphone, etc.  Proximity sensors like capacitive andRF proximity detectors can detect obstacles in close vicinity.  This paper presents an overviewof sensors normally deployed in unmanned tracked vehicles

Toward 3D reconstruction of outdoor scenes using an MMW radar and a monocular vision sensor

International audienceIn this paper, we introduce a geometric method for 3D reconstruction of the exterior environment using a panoramic microwave radar and a camera. We rely on the complementarity of these two sensors considering the robustness to the environmental conditions and depth detection ability of the radar, on the one hand, and the high spatial resolution of a vision sensor, on the other. Firstly, geometric modeling of each sensor and of the entire system is presented. Secondly, we address the global calibration problem, which consists of finding the exact transformation between the sensors' coordinate systems. Two implementation methods are proposed and compared, based on the optimization of a non-linear criterion obtained from a set of radar-to-image target correspondences. Unlike existing methods, no special configuration of the 3D points is required for calibration. This makes the methods flexible and easy to use by a non-expert operator. Finally, we present a very simple, yet robust 3D reconstruction method based on the sensors' geometry. This method enables one to reconstruct observed features in 3D using one acquisition (static sensor), which is not always met in the state of the art for outdoor scene reconstruction.The proposed methods have been validated with synthetic and real data

Crowd-based cognitive perception of the physical world: Towards the internet of senses

This paper introduces a possible architecture and discusses the research directions for the realization of the Cognitive Perceptual Internet (CPI), which is enabled by the convergence of wired and wireless communications, traditional sensor networks, mobile crowd-sensing, and machine learning techniques. The CPI concept stems from the fact that mobile devices, such as smartphones and wearables, are becoming an outstanding mean for zero-effort world-sensing and digitalization thanks to their pervasive diffusion and the increasing number of embedded sensors. Data collected by such devices provide unprecedented insights into the physical world that can be inferred through cognitive processes, thus originating a digital sixth sense. In this paper, we describe how the Internet can behave like a sensing brain, thus evolving into the Internet of Senses, with network-based cognitive perception and action capabilities built upon mobile crowd-sensing mechanisms. The new concept of hyper-map is envisioned as an efficient geo-referenced repository of knowledge about the physical world. Such knowledge is acquired and augmented through heterogeneous sensors, multi-user cooperation and distributed learning mechanisms. Furthermore, we indicate the possibility to accommodate proactive sensors, in addition to common reactive sensors such as cameras, antennas, thermometers and inertial measurement units, by exploiting massive antenna arrays at millimeter-waves to enhance mobile terminals perception capabilities as well as the range of new applications. Finally, we distillate some insights about the challenges arising in the realization of the CPI, corroborated by preliminary results, and we depict a futuristic scenario where the proposed Internet of Senses becomes true

Application of transmitarray antennas for indoor mapping at millimeter-waves

Millimeter-waves are expected to play a key role in next 5G scenario due to the availability of a large clean unlicensed bandwidth at 60 GHz and the possibility to realize packed antenna arrays, with a consequent increase of the communication capacity and the introduction of new functionalities, such as high-definition localization and personal radar for automatic environment mapping. In this paper we propose the adoption of millimeter-wave transmitarrays for personal radar applications and we investigate the impact of the radiation pattern characteristics on the map reconstruction accuracy, by analysing how the number of array elements, of quantization bits and the focal distance affect the environment reconstruction performance

A systematic review of perception system and simulators for autonomous vehicles research

This paper presents a systematic review of the perception systems and simulators for autonomous vehicles (AV). This work has been divided into three parts. In the first part, perception systems are categorized as environment perception systems and positioning estimation systems. The paper presents the physical fundamentals, principle functioning, and electromagnetic spectrum used to operate the most common sensors used in perception systems (ultrasonic, RADAR, LiDAR, cameras, IMU, GNSS, RTK, etc.). Furthermore, their strengths and weaknesses are shown, and the quantification of their features using spider charts will allow proper selection of different sensors depending on 11 features. In the second part, the main elements to be taken into account in the simulation of a perception system of an AV are presented. For this purpose, the paper describes simulators for model-based development, the main game engines that can be used for simulation, simulators from the robotics field, and lastly simulators used specifically for AV. Finally, the current state of regulations that are being applied in different countries around the world on issues concerning the implementation of autonomous vehicles is presented.This work was partially supported by DGT (ref. SPIP2017-02286) and GenoVision (ref. BFU2017-88300-C2-2-R) Spanish Government projects, and the “Research Programme for Groups of Scientific Excellence in the Region of Murcia" of the Seneca Foundation (Agency for Science and Technology in the Region of Murcia – 19895/GERM/15)

Millimeter-wave and terahertz imaging techniques

This thesis presents the development and assessment of imaging techniques in the millimeterwave (mmW) and terahertz frequency bands. In the first part of the thesis, the development of a 94 GHz passive screener based on a total-power radiometer (TPR) with mechanical beamscanning is presented. Several images have been acquired with the TPR screener demonstrator, either in indoor and outdoor environments, serving as a testbed to acquire the know-how required to perform the research presented in the following parts of the thesis. In the second part of the thesis, a theoretical research on the performance of near-field passive screeners is described. This part stands out the tradeoff between spatial and radiometric resolutions taking into account the image distortion produced by placing the scenario in the near-field range of the radiometer array. In addition, the impact of the decorrelation effect in the image has been also studied simulating the reconstruction technique of a synthetic aperture radiometer. Guidelines to choose the proper radiometer depending on the application, the scenario, the acquisition speed and the tolerated image distortion are given in this part. In the third part of the thesis, the development of a correlation technique with optical processing applicable to millimeter-wave interferometric radiometers is described. The technique is capable of correlating wide-bandwidth signals in the optical domain with no loss of radiometric sensitivity. The theoretical development of the method as well as measurements validating the suitability to correlate radiometric signals are presented in this part. In the final part of the thesis, the frequency band of the imaging problem is increased to frequencies beyond 100 GHz, covering the THz band. In this case the research is centered in tomographic techniques that include spectral information of the samples in the reconstructed images. The tomographic algorithm can provide detection and identification of chemical compounds that present a certain spectral footprint in the THz frequency band.Postprint (published version

Millimetre wave imaging for concealed target detection

PhDConcealed weapon detection (CWD) has been a hot topic as the concern about pub- lic safety increases. A variety of approaches for the detection of concealed objects on the human body based on earth magnetic ¯eld distortion, inductive magnetic ¯eld, acoustic and ultrasonic, electromagnetic resonance, MMW (millimetre wave), THz, Infrared, x-ray technologies have been suggested and developed. Among all of them, MMW holographic imaging is considered as a promising approach due to the relatively high penetration and high resolution that it can o®er. Typical concealed target detection methods are classi¯ed into 2 categories, the ¯rst one is a resonance based target identi¯cation technique, and the second one is an imaging based system. For the former, the complex natural resonance (CNR) frequencies associated with a certain target are extracted and used for identi¯cation, but this technique has an issue of high false alarm rate. The microwave/millimetre wave imaging systems can be categorized into two types: passive systems and active sys- tems. For the active microwave/millimetre wave imaging systems, the microwave holographic imaging approach was adopted in this thesis. Such a system can oper- ate at either a single frequency or multiple frequencies (wide band). An active, coherent, single frequency operation millimetre wave imaging system based on the theory of microwave holography was developed. Based on literature surveys and ¯rst hand experimental results, this thesis aims to provide system level parame- ter determination to aid the development of a target detection imager. The goal is approached step by step in 7 chapters, with topics and issues addressed rang- ing from reviewing the past work, ¯nding out the best candidate technology, i.e. the MMW holographic imaging combined with the resonance based target recog- i nition technique, the construction of the 94 GHz MMW holographic prototype imager, experimental trade-o® investigation of system parameters, imager per- formance evaluation, low pro¯le components and image enhancement techniques, feasibility investigation of resonance based technique, to system implementation based on the parameters and results achieved. The task set forth in the beginning is completed by coming up with an entire system design in the end.

Vision-Based Monocular SLAM in Micro Aerial Vehicle

Micro Aerial Vehicles (MAVs) are popular for their efficiency, agility, and lightweights. They can navigate in dynamic environments that cannot be accessed by humans or traditional aircraft. These MAVs rely on GPS and it will be difficult for GPS-denied areas where it is obstructed by buildings and other obstacles.  Simultaneous Localization and Mapping (SLAM) in an unknown environment can solve the aforementioned problems faced by flying robots.  A rotation and scale invariant visual-based solution, oriented fast and rotated brief (ORB-SLAM) is one of the best solutions for localization and mapping using monocular vision.  In this paper, an ORB-SLAM3 has been used to carry out the research on localizing micro-aerial vehicle Tello and mapping an unknown environment.  The effectiveness of ORB-SLAM3 was tested in a variety of indoor environments.   An integrated adaptive controller was used for an autonomous flight that used the 3D map, produced by ORB-SLAM3 and our proposed novel technique for robust initialization of the SLAM system during flight.  The results show that ORB-SLAM3 can provide accurate localization and mapping for flying robots, even in challenging scenarios with fast motion, large camera movements, and dynamic environments.  Furthermore, our results show that the proposed system is capable of navigating and mapping challenging indoor situations