Sistemas automáticos de informação e segurança para apoio na condução de veículos

Doutoramento em Engenharia MecânicaO objeto principal desta tese é o estudo de algoritmos de processamento e representação automáticos de dados, em particular de informação obtida por sensores montados a bordo de veículos (2D e 3D), com aplicação em contexto de sistemas de apoio à condução. O trabalho foca alguns dos problemas que, quer os sistemas de condução automática (AD), quer os sistemas avançados de apoio à condução (ADAS), enfrentam hoje em dia. O documento é composto por duas partes. A primeira descreve o projeto, construção e desenvolvimento de três protótipos robóticos, incluindo pormenores associados aos sensores montados a bordo dos robôs, algoritmos e arquitecturas de software. Estes robôs foram utilizados como plataformas de ensaios para testar e validar as técnicas propostas. Para além disso, participaram em várias competições de condução autónoma tendo obtido muito bons resultados. A segunda parte deste documento apresenta vários algoritmos empregues na geração de representações intermédias de dados sensoriais. Estes podem ser utilizados para melhorar técnicas já existentes de reconhecimento de padrões, deteção ou navegação, e por este meio contribuir para futuras aplicações no âmbito dos AD ou ADAS. Dado que os veículos autónomos contêm uma grande quantidade de sensores de diferentes naturezas, representações intermédias são particularmente adequadas, pois podem lidar com problemas relacionados com as diversas naturezas dos dados (2D, 3D, fotométrica, etc.), com o carácter assíncrono dos dados (multiplos sensores a enviar dados a diferentes frequências), ou com o alinhamento dos dados (problemas de calibração, diferentes sensores a disponibilizar diferentes medições para um mesmo objeto). Neste âmbito, são propostas novas técnicas para a computação de uma representação multi-câmara multi-modal de transformação de perspectiva inversa, para a execução de correcção de côr entre imagens de forma a obter mosaicos de qualidade, ou para a geração de uma representação de cena baseada em primitivas poligonais, capaz de lidar com grandes quantidades de dados 3D e 2D, tendo inclusivamente a capacidade de refinar a representação à medida que novos dados sensoriais são recebidos.The main object of this thesis is the study of algorithms for automatic information processing and representation, in particular information provided by onboard sensors (2D and 3D), to be used in the context of driving assistance. The work focuses on some of the problems facing todays Autonomous Driving (AD) systems and Advanced Drivers Assistance Systems (ADAS). The document is composed of two parts. The first part describes the design, construction and development of three robotic prototypes, including remarks about onboard sensors, algorithms and software architectures. These robots were used as test beds for testing and validating the developed techniques; additionally, they have participated in several autonomous driving competitions with very good results. The second part of this document presents several algorithms for generating intermediate representations of the raw sensor data. They can be used to enhance existing pattern recognition, detection or navigation techniques, and may thus benefit future AD or ADAS applications. Since vehicles often contain a large amount of sensors of different natures, intermediate representations are particularly advantageous; they can be used for tackling problems related with the diverse nature of the data (2D, 3D, photometric, etc.), with the asynchrony of the data (multiple sensors streaming data at different frequencies), or with the alignment of the data (calibration issues, different sensors providing different measurements of the same object). Within this scope, novel techniques are proposed for computing a multicamera multi-modal inverse perspective mapping representation, executing color correction between images for obtaining quality mosaics, or to produce a scene representation based on polygonal primitives that can cope with very large amounts of 3D and 2D data, including the ability of refining the representation as new information is continuously received

Intelligent Navigation Service Robot Working in a Flexible and Dynamic Environment

Numerous sensor fusion techniques have been reported in the literature for a number of robotics applications. These techniques involved the use of different sensors in different configurations. However, in the case of food driving, the possibility of the implementation has been overlooked. In restaurants and food delivery spots, enhancing the food transfer to the correct table is neatly required, without running into other robots or diners or toppling over. In this project, a particular algorithm module has been proposed and implemented to enhance the robot driving methodology and maximize robot functionality, accuracy, and the food transfer experience. The emphasis has been on enhancing movement accuracy to reach the targeted table from the start to the end. Four major elements have been designed to complete this project, including mechanical, electrical, electronics, and programming. Since the floor condition greatly affecting the wheels and turning angle selection, the movement accuracy was improved during the project. The robot was successfully able to receive the command from the restaurant and go to deliver the food to the customers\u27 tables, considering any obstacles on the way to avoid. The robot has equipped with two trays to mount the food with well-configured voices to welcome and greet the customer. The performance has been evaluated and undertaken using a routine robot movement tests. As part of this study, the designed service wheeled robot required to be with a high-performance real-time processor. As long as the processor was adequate, the experimental results showed a highly effective search robot methodology. Having concluded from the study that a minimum number of sensors are needed if they are placed appropriately and used effectively on a robot\u27s body, as navigation could be performed by using a small set of sensors. The Arduino Due has been used to provide a real-time operating system. It has provided a very successful data processing and transfer throughout any regular operation. Furthermore, an easy-to-use application has been developed to improve the user experience, so that the operator can interact directly with the robot via a special setting screen. It is possible, using this feature, to modify advanced settings such as voice commands or IP address without having to return back to the code

Implementation and improvement of an unmanned aircraft system for precision farming purposes

Precision farming (PF) is an agricultural concept that accounts for within-field variability by gathering spatial and temporal information with modern sensing technology and performs variable and targeted treatments on a smaller scale than field scale. PF research quickly recognized the possible benefits unmanned aerial vehicles (UAVs) can add to the site-specific management of farms. As UAVs are flexible carrier platforms, they can be equipped with a range of different sensing devices and used in a variety of close-range remote sensing scenarios. Most frequently, UAVs are utilized to gather actual in-season canopy information with imaging sensors that are sensitive to reflected electro-magnetic radiation in the visual (VIS) and near-infrared (NIR) spectrum. They are generally used to infer the crops biophysical and biochemical parameters to support farm management decisions. A current disadvantage of UAVs is that they are not designed to interact with their attached sensor payload. This leads to the need of intensive data post-processing and prohibits the possibility of real-time scenarios, in which UAVs can directly transfer information to field machinery or robots. In consequence, this thesis focused on the development of a smart unmanned aircraft system (UAS), which in the thesis context was regarded as a combination of a UAV carrier platform, an on-board central processing unit for sensor control and data processing, and a remotely connected ground control station. The ground control station was supposed to feature the possibility of flight mission control and the standardized distribution of sensor data with a sensor data infrastructure, serving as a data basis for a farm management information system (FMIS). The UAS was intended to be operated as a flexible monitoring tool for in-season above-ground biomass and nitrogen content estimation as well as crop yield prediction. Therefore, the selection, development, and validation of appropriate imaging sensors and processing routines were key parts to prove the UAS usability in PF scenarios. The individual objectives were (i) to implement an advanced UAV for PF research, providing the possibilities of remotely-controlled and automatic flight mission execution, (ii) to improve the developed UAV to a UAS by implementing sensor control, data processing and communication functionalities, (iii) to select and develop appropriate sensor systems for yield prediction and nitrogen fertilization strategies, (iv) to integrate the sensor systems into the UAS and to test the performance in example use cases, and (v) to embed the UAS into a standardized sensor data infrastructure for data storage and usage in PF applications. This work demonstrated the successful development of a custom rotary-wing UAV carrier platform with an embedded central processing unit. A modular software framework was developed with the ability to control any kind of sensor payload in real-time. The sensors can be triggered and their measurements are retrieved, fused together with the carriers navigation information, logged and broadcasted to a ground control station. The setup was used as basis for further research, focusing on information generation by sophisticated data processing. For a first application of predicting the grain yield of corn (Zea mays L.), a simple RGB camera was selected to acquire a set of aerial imagery of early- and mid-season corn crops. Orthoimages were processed with different ground resolutions and were computed to simple vegetation indices (VI) for a crop/non-crop classification. In addition to that, crop surface models (CSMs) were generated to estimate the crop heights. Linear regressions were performed with the corn grain yield as dependent variable and crop height and crop coverage as independent variable. The analysis showed the best prediction results of a relative root mean square error (RMSE) of 8.8 % at mid-season growth stages and ground resolutions of 4 cm px −1 . Moreover, the results indicate that with on-going canopy closure and homogeneity accounting for high ground resolutions and crop/non-crop classification becomes less and less important. For the estimation of above-ground biomass and nitrogen content in winter wheat (Triticum aestivum L.) a programmable multispectral camera was developed. It is based on an industrial multi-sensor camera, which was equipped with bandpass filters to measure four narrow wavelength bands in the so-called red-edge region. This region is the transition zone in between the VIS and NIR spectrum and known to be sensitive to leaf chlorophyll content and the structural state of the plant. It is often used to estimate biomass and nitrogen content with the help of the normalized difference vegetation index (NDVI) and the red-edge inflection point (REIP). The camera system was designed to measure ambient light conditions during the flight mission to set appropriate image acquisition times, which guarantee images with high contrast. It is fully programmable and can be further developed to a real-time image processing system. The analysis relies on semi-automatic orthoimage processing. The NDVI orthoimages were analyzed for the correlation with biomass by means of simple linear regression. These models proved to estimate biomass for all measurements with RMSEs of 12.3 % to 17.6 %. The REIP was used to infer nitrogen content and showed good results with RMSEs of 7.6 % to 11.7 %. Both NDVI and REIP were also tested for the in-season grain yield prediction ability (RMSE = 9.012.1 %), whereas grain protein content could be modeled with the REIP, except for low-fertilized wheat plots. The last part of the thesis comprised the development of a standardized sensor data infrastructure as a first step to a holistic farm management. The UAS was integrated into a real-time sensor data acquisition network with standardized data base storage capabilities. The infrastructure was based on open source software and the geo-data standards of the Open Geospatial Consortium (OGC). A prototype implementation was tested for four exemplary sensor systems and proved to be able to acquire, log, visualize and store the sensor data in a standardized data base via a sensor observation service on-the-fly. The setup is scalable to scenarios, where a multitude of sensors, data bases, and web services interact with each other to exchange and process data. This thesis demonstrates the successful prototype implementation of a smart UAS and a sensor data infrastructure, which offers real-time data processing functionality. The UAS is equipped with appropriate sensor systems for agricultural crop monitoring and has the potential to be used in real-world scenarios.Precision farming (PF) ist ein landwirtschaftliches Konzept, das die Variabilität innerhalb eines Feldes berücksichtigt, indem es mit Hilfe moderner Sensortechnologien räumliche und zeitliche Bestandsinformationen sammelt. Dadurch ist PF in der Lage, gezielte teilflächenspezifische Anwendungen innerhalb eines Feldes durchzuführen. Die Forschung im Bereich von PF hat früh die potenziellen Vorzüge von kleinen Luftfahrzeugen, sogenannten unmanned aerial vehicles (UAVs), für die teilflächenspezifische Bewirtschaftung erkannt. Da UAVs flexible Lastenträger darstellen, können sie mit den verschiedensten Sensoren ausgestattet und in einer Vielzahl von fernerkundlichen Anwendungsfällen in der Landwirtschaft genutzt werden. Dabei werden sie am häufigsten mit bildgebenden Sensoren eingesetzt, um aktuelle Informationen über den Pflanzenbestand in der Vegetationsperiode zu liefern. Die eingesetzten Sensoren sind dabei meist zur Messung elektromagnetischer Strahlung im sichtbaren (VIS) und nahen infraroten (NIR) Bereich ausgelegt. Im Allgemeinen werden sie dazu benutzt auf biophysikalische und biochemische Eigenschaften der Nutzpflanzen zu schließen und damit die Entscheidungsprozesse in der Bestandsführung zu unterstützen. Ein aktueller Nachteil der UAVs ist, dass sie nicht dafür gebaut werden um mit ihrer Nutzlast zu interagieren. Das führt zu einem Bedarf an erheblicher Datennachverarbeitung und verhindert Echtzeitszenarios, in denen UAVs Informationen direkt an Feldmaschinen und Roboter senden können. Aus diesem Grund konzentrierte sich diese Dissertation auf die Entwicklung eines intelligenten fliegenden Systems, eines sogenannten unmanned aircraft system (UAS), welches im Kontext dieser Dissertation als eine Kombination aus UAV Trägerplattform, zentralem Computer zur Sensorsteuerung und Datenverarbeitung, sowie einer entfernt verbundenen Bodenstation betrachtet wurde. Die Bodenstation war zur Flugüberwachung und zur standardisierten Verteilung der Sensordaten über eine Sensordateninfrastruktur bestimmt. Die Sensordateninfrastruktur diente als Basis eines sogenannten farm management information system (FMIS), das die Verwaltung und Bewirtschaftung eines landwirtschaftlichen Betriebs mit Methoden der Informatik unterstützt. Das UAS sollte als flexibles Aufklärungswerkzeug eingesetzt werden, um Schätzungen von Biomasse, Stickstoffgehalt und erwartetem Ertrag während der Vegetationsperiode zu liefern. Daher war die Auswahl, Entwicklung und Validierung geeigneter bildgebender Sensoren und zugehöriger Verarbeitungsmethoden ein zentraler Bestandteil, um die Nutzbarkeit von UAS im PF zu belegen. Die einzelnen Ziele waren (i) der Aufbau eines UAVs für das PF, das sich fernsteuern und automatisch nach Wegpunkten fliegen lässt, (ii) die Erweiterung des UAVs zum UAS, durch die Entwicklung einer zentralen Sensorsteuerung, Datenverarbeitung und Kommunikationsfähigkeit, (iii) die Auswahl und Entwicklung geeigneter Sensorsysteme zur Ertragsschätzung und Stickstoffdüngung, (iv) der Einbau der Sensorsysteme in das UAS und deren Validierung in Beispielanwendungen und (v) die Integration des UAS in eine standardisierte Sensordateninfrastruktur um die Daten für PF-Anwendungen abzuspeichern und verfügbar zu machen. Diese Dissertation präsentiert eine erfolgreiche Entwicklung eines Drehflügler-UAVs mit zentraler Steuereinheit. Dazu passend wurde eine modulare Software entwickelt, die jegliche Sensorik in Echtzeit steuern kann. Messungen können ausgelöst, empfangen, mit den Navigationsdaten des UAVs fusioniert, gespeichert und an eine Bodenstation gesendet werden. Das UAV diente als Basis weiterer Forschung, die die Verarbeitung von Sensordaten zur Erzeugung pflanzenbaulicher Information zum Ziel hatte. Eine erste Anwendung war die Ertragsschätzung von Körnermais (Zea mays L.). Eine einfache RGB Kamera wurde dazu benutzt Luftbilder von Maispflanzen in frühen und mittleren Wachstumsstadien aufzunehmen. Daraus wurden Orthophotos mit unterschiedlichen Bodenauflösungen erzeugt und zu einfachen Vegetationsindizes (VIs) zur Klassifizierung der Pixel als Pflanze oder nicht Pflanze weiterverarbeitet. Zusätzlich wurden Oberflächenmodelle des Pflanzenbestands, sogenannte crop surface models (CSMs), erzeugt, um die Pflanzenhöhen abzuschätzen. Mit dem Ertrag als abhängige Variable, sowie Pflanzenhöhe und Bedeckungsgrad als unabhängige Variablen, wurden lineare Regressionen durchgeführt. Die Analyse ergab beste Vorhersagen mit geringsten Standardabweichungen (SD) von 8.8 % für die Messungen in mittleren Wachstumsstadien mit einer Bodenauflösung von 4 cm px −1 . Darüber hinaus zeigten die Ergebnisse, dass hohe Bodenauflösungen und Klassifizierung mit fortschreitendem Reihenschluss und sich angleichendem Pflanzenbestand immer unwichtiger werden. Zur Schätzung von Biomasse und Stickstoffgehalt von Winterweizen (Triticum aestivum L.) wurde eine programmierbare multispektrale Kamera entwickelt. Sie basiert auf einer Industriekamera mit mehreren Sensorköpfen, von denen jeder mit einem Bandpassfilter bestückt wurde. Die Kamera misst vier schmalbandige Wellenlängen im Übergangsbereich vom VIS- zum NIR-Spektrum, der sogenannten roten Kante red-edge. Dieser Bereich ist dafür bekannt Rückschlüsse auf den Chlorophyllgehalt der Blätter und die Pflanzenstruktur zuzulassen. Mit Hilfe der Formeln zur Berechnung des normalized difference vegetation index (NDVI) und des red-edge inflection point (REIP) wird dieser Bereich oft zur Schätzung von Biomasse und Stickstoffgehalt genutzt. Das Kamerasystem wurde darüber hinaus entworfen, die Lichtverhältnisse während des Fluges zu messen und geeignete Belichtungszeiten festzulegen, um Bilder mit hohem Kontrast zu erzeugen. Die Kamera ist komplett programmierbar und kann zur Echtzeitbildverarbeitung weiterentwickelt werden. Die Untersuchung basiert auf der teilautomatisierten Erzeugung von Orthophotos. Die NDVI Orthophotos wurden mit Hilfe einer einfachen linearen Regression auf ihre Korrelation mit Biomasse getestet. Sie zeigten über alle Messzeitpunkte, dass sie Biomasse mit Standardabweichungen von 12.3 % bis 17.6 % schätzen konnten. Der REIP wurde zur Stickstoffgehaltschätzung heran gezogen und zeigte gute Ergebnisse mit Standardabweichungen von 7.6 % bis 11.7 %. Beide, NDVI und REIP, wurden auch auf ihre Vorhersagefähigkeit des Kornertrags getestet (SD = 9.012.1 %). Überdies konnte, außer in gering gedüngten Parzellen, der Proteingehalt im Korn mit dem REIP abgeschätzt werden. Der letzte Teil der Dissertation beinhaltete die Entwicklung einer standardisierten Sensordateninfrastruktur als Schritt hin zu einem umfassenden Bewirtschaftungskonzept, das möglichst viele Faktoren berücksichtigt. Das UAS wurde in ein echtzeitbasiertes Sensordatennetzwerk integriert, das Sensordaten erfassen und standardisiert in Datenbanken ablegen kann. Die Infrastruktur basiert auf quellcodeoffener open source software und den Geodatenstandards des Open Geospatial Consortiums (OGC). Eine erste Umsetzung einer solchen Infrastruktur wurde mit vier Beispielsensoren getestet und zeigte, dass Sensordaten in Echtzeit erfasst, lokal gespeichert, visualisiert und mittels eines Sensordatendienstes (sensor observation service) standardisiert in einer Datenbank gespeichert werden konnten. Die Umsetzung ist auf eine beliebige Anzahl von Sensoren und Diensten erweiterbar und ermöglicht ihnen den Austausch und die Verarbeitung von Daten. Diese Dissertation zeigt eine erfolgreiche Umsetzung eines intelligenten UAS und einer Sensordateninfrastruktur, die Sensordatenverarbeitung in Echtzeit anbietet. Das UAS ist mit Sensoren ausgestattet, die zur landwirtschaftlichen Beurteilung von Pflanzenbeständen geeignet sind und zeigt Potential auch unter realistischen Bedingungen eingesetzt werden zu können

Automated Visual Database Creation For A Ground Vehicle Simulator

This research focuses on extracting road models from stereo video sequences taken from a moving vehicle. The proposed method combines color histogram based segmentation, active contours (snakes) and morphological processing to extract road boundary coordinates for conversion into Matlab or Multigen OpenFlight compatible polygonal representations. Color segmentation uses an initial truth frame to develop a color probability density function (PDF) of the road versus the terrain. Subsequent frames are segmented using a Maximum Apostiori Probability (MAP) criteria and the resulting templates are used to update the PDFs. Color segmentation worked well where there was minimal shadowing and occlusion by other cars. A snake algorithm was used to find the road edges which were converted to 3D coordinates using stereo disparity and vehicle position information. The resulting 3D road models were accurate to within 1 meter

2016 International Land Model Benchmarking (ILAMB) Workshop Report

As earth system models (ESMs) become increasingly complex, there is a growing need for comprehensive and multi-faceted evaluation of model projections. To advance understanding of terrestrial biogeochemical processes and their interactions with hydrology and climate under conditions of increasing atmospheric carbon dioxide, new analysis methods are required that use observations to constrain model predictions, inform model development, and identify needed measurements and field experiments. Better representations of biogeochemistryclimate feedbacks and ecosystem processes in these models are essential for reducing the acknowledged substantial uncertainties in 21st century climate change projections

Laser Scanner Technology

Laser scanning technology plays an important role in the science and engineering arena. The aim of the scanning is usually to create a digital version of the object surface. Multiple scanning is sometimes performed via multiple cameras to obtain all slides of the scene under study. Usually, optical tests are used to elucidate the power of laser scanning technology in the modern industry and in the research laboratories. This book describes the recent contributions reported by laser scanning technology in different areas around the world. The main topics of laser scanning described in this volume include full body scanning, traffic management, 3D survey process, bridge monitoring, tracking of scanning, human sensing, three-dimensional modelling, glacier monitoring and digitizing heritage monuments

FEUPCAR 2.0 : Condução autónoma no Festival Nacional de Robótica

Tese de mestrado integrado. Engenharia Electrotécnica e de Computadores (Major Automação). Faculdade de Engenharia. Universidade do Porto. 201