GNSS-based passive radar techniques for maritime surveillance

The improvement of maritime traffic safety and security is a subject of growing interest, since the traffic is constantly increasing. In fact, a large number of human activities take place in maritime domain, varying from cruise and trading ships up to vessels involved in nefarious activities such as piracy, human smuggling or terrorist actions. The systems based on Automatic Identification System (AIS) transponder cannot cope with non-cooperative or non-equipped vessels that instead can be detected, tracked and identified by means of radar system. In particular, passive bistatic radar (PBR) systems can perform these tasks without a dedicated transmitter, since they exploit illuminators of opportunity as transmitters. The lack of a dedicated transmitter makes such systems low cost and suitable to be employed in areas where active sensors cannot be placed such as, for example, marine protected areas. Innovative solutions based on terrestrial transmitters have been considered in order to increase maritime safety and security, but these kinds of sources cannot guarantee a global coverage, such as in open sea. To overcome this problem, the exploitation of global navigation satellites system (GNSS) as transmitters of opportunity is a prospective solution. The global, reliable and persistent nature of these sources makes them potentially able to guarantee the permanent monitoring of both coastal and open sea areas. To this aim, this thesis addresses the exploitation of Global Navigation Satellite Systems (GNSS) as transmitters of opportunity in passive bistatic radar (PBR) systems for maritime surveillance. The main limitation of this technology is the restricted power budget provided by navigation satellites, which makes it necessary to define innovative moving target detection techniques specifically tailored for the system under consideration. For this reason, this thesis puts forward long integration time techniques able to collect the signal energy over long time intervals (tens of seconds), allowing the retrieval of suitable levels of signal-to-disturbance ratios for detection purposes. The feasibility of this novel application is firstly investigated in a bistatic system configuration. A long integration time moving target detection technique working in bistatic range&Doppler plane is proposed and its effectiveness is proved against synthetic and experimental datasets. Subsequently the exploitation of multiple transmitters for the joint detection and localization of vessels at sea is also investigated. A single-stage approach to jointly detect and localize the ship targets by making use of long integration times (tens of seconds) and properly exploiting the spatial diversity offered by such a configuration is proposed. Furthermore, the potential of the system to extract information concerning the detected target characteristics for further target classification is assessed

Proof-of-concept of a single-point Time-of-Flight LiDAR system and guidelines towards integrated high-accuracy timing, advanced polarization sensing and scanning with a MEMS micromirror

Dissertação de mestrado integrado em Engenharia Física (área de especialização em Dispositivos, Microssistemas e Nanotecnologias)The core focus of the work reported herein is the fulfillment of a functional Light Detection and Ranging (LiDAR) sensor to validate the direct Time-of-Flight (ToF) ranging concept and the acquisition of critical knowledge regarding pivotal aspects jeopardizing the sensor’s performance, for forthcoming improvements aiming a realistic sensor targeted towards automotive applications. Hereupon, the ToF LiDAR system is implemented through an architecture encompassing both optical and electronical functions and is subsequently characterized under a sequence of test procedures usually applied in benchmarking of LiDAR sensors. The design employs a hybrid edge-emitting laser diode (pulsed at 6kHz, 46ns temporal FWHM, 7ns rise-time; 919nm wavelength with 5nm FWHM), a PIN photodiode to detect the back-reflected radiation, a transamplification stage and two Time-to-Digital Converters (TDCs), with leading-edge discrimination electronics to mark the transit time between emission and detection events. Furthermore, a flexible modular design is adopted using two separate Printed Circuit Boards (PCBs), comprising the transmitter (TX) and the receiver (RX), i.e. detection and signal processing. The overall output beam divergence is 0.4º×1º and an optical peak power of 60W (87% overall throughput) is realized. The sensor is tested indoors from 0.56 to 4.42 meters, and the distance is directly estimated from the pulses transit time. The precision within these working distances ranges from 4cm to 7cm, reflected in a Signal-to-Noise Ratio (SNR) between 12dB and 18dB. The design requires a calibration procedure to correct systematic errors in the range measurements, induced by two sources: the timing offset due to architecture-inherent differences in the optoelectronic paths and a supplementary bias resulting from the design, which renders an intensity dependence and is denoted time-walk. The calibrated system achieves a mean accuracy of 1cm. Two distinct target materials are used for characterization and performance evaluation: a metallic automotive paint and a diffuse material. This selection is representative of two extremes of actual LiDAR applications. The optical and electronic characterization is thoroughly detailed, including the recognition of a good agreement between empirical observations and simulations in ZEMAX, for optical design, and in a SPICE software, for the electrical subsystem. The foremost meaningful limitation of the implemented design is identified as an outcome of the leading-edge discrimination. A proposal for a Constant Fraction Discriminator addressing sub-millimetric accuracy is provided to replace the previous signal processing element. This modification is mandatory to virtually eliminate the aforementioned systematic bias in range sensing due to the intensity dependency. A further crucial addition is a scanning mechanism to supply the required Field-of-View (FOV) for automotive usage. The opto-electromechanical guidelines to interface a MEMS micromirror scanner, achieving a 46º×17º FOV, with the LiDAR sensor are furnished. Ultimately, a proof-of-principle to the use of polarization in material classification for advanced processing is carried out, aiming to complement the ToF measurements. The original design is modified to include a variable wave retarder, allowing the simultaneous detection of orthogonal linear polarization states using a single detector. The material classification with polarization sensing is tested with the previously referred materials culminating in an 87% and 11% degree of linear polarization retention from the metallic paint and the diffuse material, respectively, computed by Stokes parameters calculus. The procedure was independently validated under the same conditions with a micro-polarizer camera (92% and 13% polarization retention).O intuito primordial do trabalho reportado no presente documento é o desenvolvimento de um sensor LiDAR funcional, que permita validar o conceito de medição direta do tempo de voo de pulsos óticos para a estimativa de distância, e a aquisição de conhecimento crítico respeitante a aspetos fundamentais que prejudicam a performance do sensor, ambicionando melhorias futuras para um sensor endereçado para aplicações automóveis. Destarte, o sistema LiDAR é implementado através de uma arquitetura que engloba tanto funções óticas como eletrónicas, sendo posteriormente caracterizado através de uma sequência de testes experimentais comumente aplicáveis em benchmarking de sensores LiDAR. O design tira partido de um díodo de laser híbrido (pulsado a 6kHz, largura temporal de 46ns; comprimento de onda de pico de 919nm e largura espetral de 5nm), um fotodíodo PIN para detetar a radiação refletida, um andar de transamplificação e dois conversores tempo-digital, com discriminação temporal com threshold constante para marcar o tempo de trânsito entre emissão e receção. Ademais, um design modular flexível é adotado através de duas PCBs independentes, compondo o transmissor e o recetor (deteção e processamento de sinal). A divergência global do feixe emitido para o ambiente circundante é 0.4º×1º, apresentando uma potência ótica de pico de 60W (eficiência de 87% na transmissão). O sensor é testado em ambiente fechado, entre 0.56 e 4.42 metros. A precisão dentro das distâncias de trabalho varia entre 4cm e 7cm, o que se reflete numa razão sinal-ruído entre 12dB e 18dB. O design requer calibração para corrigir erros sistemáticos nas distâncias adquiridas devido a duas fontes: o desvio no ToF devido a diferenças nos percursos optoeletrónicos, inerentes à arquitetura, e uma dependência adicional da intensidade do sinal refletido, induzida pela técnica de discriminação implementada e denotada time-walk. A exatidão do sistema pós-calibração perfaz um valor médio de 1cm. Dois alvos distintos são utilizados durante a fase de caraterização e avaliação performativa: uma tinta metálica aplicada em revestimentos de automóveis e um material difusor. Esta seleção é representativa de dois cenários extremos em aplicações reais do LiDAR. A caraterização dos subsistemas ótico e eletrónico é minuciosamente detalhada, incluindo a constatação de uma boa concordância entre observações empíricas e simulações óticas em ZEMAX e elétricas num software SPICE. O principal elemento limitante do design implementado é identificado como sendo a técnica de discriminação adotada. Por conseguinte, é proposta a substituição do anterior bloco por uma técnica de discriminação a uma fração constante do pulso de retorno, com exatidões da ordem sub-milimétrica. Esta modificação é imperativa para eliminar o offset sistemático nas medidas de distância, decorrente da dependência da intensidade do sinal. Uma outra inclusão de extrema relevância é um mecanismo de varrimento que assegura o cumprimento dos requisitos de campo de visão para aplicações automóveis. As diretrizes para a integração de um micro-espelho no sensor concebido são providenciadas, permitindo atingir um campo de visão de 46º×17º. Conclusivamente, é feita uma prova de princípio para a utilização da polarização como complemento das medições do tempo de voo, de modo a suportar a classificação de materiais em processamento avançado. A arquitetura original é modificada para incluir uma lâmina de atraso variável, permitindo a deteção de estados de polarização ortogonais com um único fotodetetor. A classificação de materiais através da aferição do estado de polarização da luz refletida é testada para os materiais supramencionados, culminando numa retenção de polarização de 87% (tinta metálica) e 11% (difusor), calculados através dos parâmetros de Stokes. O procedimento é independentemente validado com uma câmara polarimétrica nas mesmas condições (retenção de 92% e 13%)

Reconstruction and recognition of confusable models using three-dimensional perception

Perception is one of the key topics in robotics research. It is about the processing of external sensor data and its interpretation. The necessity of fully autonomous robots makes it crucial to help them to perform tasks more reliably, flexibly, and efficiently. As these platforms obtain more refined manipulation capabilities, they also require expressive and comprehensive environment models: for manipulation and affordance purposes, their models have to involve each one of the objects present in the world, coincidentally with their location, pose, shape and other aspects. The aim of this dissertation is to provide a solution to several of these challenges that arise when meeting the object grasping problem, with the aim of improving the autonomy of the mobile manipulator robot MANFRED-2. By the analysis and interpretation of 3D perception, this thesis covers in the first place the localization of supporting planes in the scenario. As the environment will contain many other things apart from the planar surface, the problem within cluttered scenarios has been solved by means of Differential Evolution, which is a particlebased evolutionary algorithm that evolves in time to the solution that yields the cost function lowest value. Since the final purpose of this thesis is to provide with valuable information for grasping applications, a complete model reconstructor has been developed. The proposed method holdsmany features such as robustness against abrupt rotations, multi-dimensional optimization, feature extensibility, compatible with other scan matching techniques, management of uncertain information and an initialization process to reduce convergence timings. It has been designed using a evolutionarybased scan matching optimizer that takes into account surface features of the object, global form and also texture and color information. The last tackled challenge regards the recognition problem. In order to procure with worthy information about the environment to the robot, a meta classifier that discerns efficiently the observed objects has been implemented. It is capable of distinguishing between confusable objects, such as mugs or dishes with similar shapes but different size or color. The contributions presented in this thesis have been fully implemented and empirically evaluated in the platform. A continuous grasping pipeline covering from perception to grasp planning including visual object recognition for confusable objects has been developed. For that purpose, an indoor environment with several objects on a table is presented in the nearby of the robot. Items are recognized from a database and, if one is chosen, the robot will calculate how to grasp it taking into account the kinematic restrictions associated to the anthropomorphic hand and the 3D model for this particular object. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------La percepción es uno de los temas más relevantes en el mundo de la investigaci ón en robótica. Su objetivo es procesar e interpretar los datos recibidos por un sensor externo. La gran necesidad de desarrollar robots autónomos hace imprescindible proporcionar soluciones que les permita realizar tareas más precisas, flexibles y eficientes. Dado que estas plataformas cada día adquieren mejores capacidades para manipular objetos, también necesitarán modelos expresivos y comprensivos: para realizar tareas de manipulación y prensión, sus modelos han de tener en cuenta cada uno de los objetos presentes en su entorno, junto con su localizaci ón, orientación, forma y otros aspectos. El objeto de la presente tesis doctoral es proponer soluciones a varios de los retos que surgen al enfrentarse al problema del agarre, con el propósito final de aumentar la capacidad de autonomía del robot manipulador MANFRED-2. Mediante el análisis e interpretación de la percepción tridimensional, esta tesis cubre en primer lugar la localización de planos de soporte en sus alrededores. Dado que el entorno contendrá muchos otros elementos aparte de la superficie de apoyo buscada, el problema en entornos abarrotados ha sido solucionado mediante Evolución Diferencial, que es un algoritmo evolutivo basado en partículas que evoluciona temporalmente a la solución que contempla el menor resultado en la función de coste. Puesto que el propósito final de este trabajo de investigación es proveer de información valiosa a las aplicaciones de prensión, se ha desarrollado un reconstructor de modelos completos. El método propuesto posee diferentes características como robustez a giros abruptos, optimización multidimensional, extensión a otras características, compatibilidad con otras técnicas de reconstrucción, manejo de incertidumbres y un proceso de inicialización para reducir el tiempo de convergencia. Ha sido diseñado usando un registro optimizado mediante técnicas evolutivas que tienen en cuenta las particularidades de la superficie del objeto, su forma global y la información relativa a la textura. El último problema abordado está relacionado con el reconocimiento de objetos. Con la intención de abastecer al robot con la mayor información posible sobre el entorno, se ha implementado un meta clasificador que diferencia de manera eficaz los objetos observados. Ha sido capacitado para distinguir objetos confundibles como tazas o platos con formas similares pero con diferentes colores o tamaños. Las contribuciones presentes en esta tesis han sido completamente implementadas y probadas de manera empírica en la plataforma. Se ha desarrollado un sistema que cubre el problema de agarre desde la percepción al cálculo de la trayectoria incluyendo el sistema de reconocimiento de objetos confundibles. Para ello, se ha presentado una mesa con objetos en un entorno cerrado cercano al robot. Los elementos son comparados con una base de datos y si se desea agarrar uno de ellos, el robot estimará cómo cogerlo teniendo en cuenta las restricciones cinemáticas asociadas a una mano antropomórfica y el modelo tridimensional generado del objeto en cuestión

Proceedings of the 2011 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

This book is a collection of 15 reviewed technical reports summarizing the presentations at the 2011 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory. The covered topics include image processing, optical signal processing, visual inspection, pattern recognition and classification, human-machine interaction, world and situation modeling, autonomous system localization and mapping, information fusion, and trust propagation in sensor networks

Experimental and Analysis of Electromagnetic Characterization of Biological and Non-Biological Materials in Microwave, Millimeter-wave, and Terahertz Frequency Bands

The goal of this research is to characterize the electromagnetic properties of biological and non-biological materials at terahertz (THz), millimeter-wave, and microwave frequency bands. The biological specimens are measured using the THz imaging and spectroscopy system, whereas the non-biological materials are measured using the microwave and millimeter-wave free-space system. These facilities are located in the Engineering Research Center at the University of Arkansas. The THz imaging system (TPS 3000) uses a Ti-Sapphire laser directed on the photoconductive antennas to generate a THz time domain pulse. Upon using the Fourier Transform, the spectrum of the pulsed THz signal includes frequencies from 0.1 THz to 4 THz. On the other hand, the free space system uses a PNA network analyzer to produce a signal at frequencies ranging from 10 MHz to 110 GHz. For the biological specimens, the research focused on imaging the freshly excised breast tumors to detect the cancer on the margins using THz radiation. The tumor margin assessment depends on the THz contrast between cancer, collagen, and fat tissues in the tumor. Three models of breast tumors are investigated in this research: humans, mice (xenograft and transgenic), and Sprague Dawley rats. The results showed good differentiation between the cancerous and non-cancerous tissues in all three models. In addition, an excellent distinction was observed between cancer, collagen, and fat in the formalin-fixed paraffin-embedded (FFPE) block tissue with ~ 90-95% correlation with the pathology images. Furthermore, the FFPE ductal carcinoma in situ (DCIS) tumors are investigated, also using the THz imaging. The THz images of the DCIS samples are compared with those of the FFPE invasive ductal carcinoma (IDC) specimens. The results demonstrated that THz electric field reflection from the IDC were higher than that from the collagen, DCIS, and then the fat tissue region. Furthermore, a pilot study is conducted to investigate the effect of optical clearance (e.g., glycerol solution) on THz images of freshly excised tumors. The results showed that the glycerol reduced the absorption coefficients of pre-treated tissues compared with those of untreated tissues. For the non-biological materials, the research focuses on characterizing highly conductive non-magnetic radar absorbing materials (RAM) for the automotive industry. The ingredients of material components in the RAM samples are unrevealed under a non-disclosure agreement (NDA). The material characterization involves the extraction of the complex relative permittivity utilizing the transmission measurement data obtained at the K-band (18 GHz to 26.5 GHz) and the W-band (75 GHz to 110 GHz). The measurements are obtained using the free-space conical horn antenna system. A transmission line based extraction model is implemented, and the results are validated with the experimental measurements of the S-parameters. The maximum error reported between the measured and the calculated S-parameters was less than 1 dB. In conclusion, the THz imaging of breast cancer tumors presents a potential margin assessment of other solid tumors, and the microwave, millimeter-wave, and THz spectroscopy of materials demonstrate a potential application in the fifth and sixth generations of wireless communications

Fruit Detection and Tree Segmentation for Yield Mapping in Orchards

Accurate information gathering and processing is critical for precision horticulture, as growers aim to optimise their farm management practices. An accurate inventory of the crop that details its spatial distribution along with health and maturity, can help farmers efficiently target processes such as chemical and fertiliser spraying, crop thinning, harvest management, labour planning and marketing. Growers have traditionally obtained this information by using manual sampling techniques, which tend to be labour intensive, spatially sparse, expensive, inaccurate and prone to subjective biases. Recent advances in sensing and automation for field robotics allow for key measurements to be made for individual plants throughout an orchard in a timely and accurate manner. Farmer operated machines or unmanned robotic platforms can be equipped with a range of sensors to capture a detailed representation over large areas. Robust and accurate data processing techniques are therefore required to extract high level information needed by the grower to support precision farming. This thesis focuses on yield mapping in orchards using image and light detection and ranging (LiDAR) data captured using an unmanned ground vehicle (UGV). The contribution is the framework and algorithmic components for orchard mapping and yield estimation that is applicable to different fruit types and orchard configurations. The framework includes detection of fruits in individual images and tracking them over subsequent frames. The fruit counts are then associated to individual trees, which are segmented from image and LiDAR data, resulting in a structured spatial representation of yield. The first contribution of this thesis is the development of a generic and robust fruit detection algorithm. Images captured in the outdoor environment are susceptible to highly variable external factors that lead to significant appearance variations. Specifically in orchards, variability is caused by changes in illumination, target pose, tree types, etc. The proposed techniques address these issues by using state-of-the-art feature learning approaches for image classification, while investigating the utility of orchard domain knowledge for fruit detection. Detection is performed using both pixel-wise classification of images followed instance segmentation, and bounding-box regression approaches. The experimental results illustrate the versatility of complex deep learning approaches over a multitude of fruit types. The second contribution of this thesis is a tree segmentation approach to detect the individual trees that serve as a standard unit for structured orchard information systems. The work focuses on trellised trees, which present unique challenges for segmentation algorithms due to their intertwined nature. LiDAR data are used to segment the trellis face, and to generate proposals for individual trees trunks. Additional trunk proposals are provided using pixel-wise classification of the image data. The multi-modal observations are fine-tuned by modelling trunk locations using a hidden semi-Markov model (HSMM), within which prior knowledge of tree spacing is incorporated. The final component of this thesis addresses the visual occlusion of fruit within geometrically complex canopies by using a multi-view detection and tracking approach. Single image fruit detections are tracked over a sequence of images, and associated to individual trees or farm rows, with the spatial distribution of the fruit counting forming a yield map over the farm. The results show the advantage of using multi-view imagery (instead of single view analysis) for fruit counting and yield mapping. This thesis includes extensive experimentation in almond, apple and mango orchards, with data captured by a UGV spanning a total of 5 hectares of farm area, over 30 km of vehicle traversal and more than 7,000 trees. The validation of the different processes is performed using manual annotations, which includes fruit and tree locations in image and LiDAR data respectively. Additional evaluation of yield mapping is performed by comparison against fruit counts on trees at the farm and counts made by the growers post-harvest. The framework developed in this thesis is demonstrated to be accurate compared to ground truth at all scales of the pipeline, including fruit detection and tree mapping, leading to accurate yield estimation, per tree and per row, for the different crops. Through the multitude of field experiments conducted over multiple seasons and years, the thesis presents key practical insights necessary for commercial development of an information gathering system in orchards