On Semantic Segmentation and Path Planning for Autonomous Vehicles within Off-Road Environments

There are many challenges involved in creating a fully autonomous vehicle capable of safely navigating through off-road environments. In this work we focus on two of the most prominent such challenges, namely scene understanding and path planning. Scene understanding is a challenging computer vision task with recent advances in convolutional neural networks (CNN) achieving results that notably surpass prior traditional feature driven approaches. Here, we build on recent work in urban road-scene understanding, training a state of the art CNN architecture towards the task of classifying off-road scenes. We analyse the effects of transfer learning and training data set size on CNN performance, evaluating multiple configurations of the network at multiple points during the training cycle, investigating in depth how the training process is affected. We compare this CNN to a more traditional feature-driven approach with Support Vector Machine (SVM) classifier and demonstrate state-of-the-art results in this particularly challenging problem of off-road scene understanding. We then expand on this with the addition of multi-channel RGBD data, which we encode in multiple configurations for CNN input. We evaluate each of these configuration over our own off-road RGBD data set and compare performance to that of the network model trained using RGB data. Next, we investigate end-to-end navigation, whereby a machine learning algorithm optimises to predict the vehicle control inputs of a human driver. After evaluating such a technique in an off-road environment and identifying several limitations, we propose a new approach in which a CNN learns to predict vehicle path visually, combining a novel approach to automatic training data creation with state of the art CNN architecture to map a predicted route directly onto image pixels. We then evaluate this approach using our off-road data set, and demonstrate effectiveness surpassing existing end-to-end methods

Development of bent-up triangular tab shear transfer (BTTST) enhancement in cold-formed steel (CFS)-concrete composite beams

Cold-formed steel (CFS) sections, have been recognised as an important contributor to environmentally responsible and sustainable structures in developed countries, and CFS framing is considered as a sustainable 'green' construction material for low rise residential and commercial buildings. However, there is still lacking of data and information on the behaviour and performance of CFS beam in composite construction. The use of CFS has been limited to structural roof trusses and a host of nonstructural applications. One of the limiting features of CFS is the thinness of its section (usually between 1.2 and 3.2 mm thick) that makes it susceptible to torsional, distortional, lateral-torsional, lateral-distortional and local buckling. Hence, a reasonable solution is resorting to a composite construction of structural CFS section and reinforced concrete deck slab, which minimises the distance from the neutral-axis to the top of the deck and reduces the compressive bending stress in the CFS sections. Also, by arranging two CFS channel sections back-to-back restores symmetricity and suppresses lateraltorsional and to a lesser extent, lateral-distortional buckling. The two-fold advantages promised by the system, promote the use of CFS sections in a wider range of structural applications. An efficient and innovative floor system of built-up CFS sections acting compositely with a concrete deck slab was developed to provide an alternative composite system for floors and roofs in buildings. The system, called Precast Cold-Formed SteelConcrete Composite System, is designed to rely on composite actions between the CFS sections and a reinforced concrete deck where shear forces between them are effectively transmitted via another innovative shear transfer enhancement mechanism called a bentup triangular tab shear transfer (BTTST). The study mainly comprises two major components, i.e. experimental and theoretical work. Experimental work involved smallscale and large-scale testing of laboratory tests. Sixty eight push-out test specimens and fifteen large-scale CFS-concrete composite beams specimens were tested in this program. In the small-scale test, a push-out test was carried out to determine the strength and behaviour of the shear transfer enhancement between the CFS and concrete. Four major parameters were studied, which include compressive strength of concrete, CFS strength, dimensions (size and angle) of BTTST and CFS thickness. The results from push-out test were used to develop an expression in order to predict the shear capacity of innovative shear transfer enhancement mechanism, BTTST in CFS-concrete composite beams. The value of shear capacity was used to calculate the theoretical moment capacity of CFSconcrete composite beams. The theoretical moment capacities were used to validate the large-scale test results. The large-scale test specimens were tested by using four-point load bending test. The results in push-out tests show that specimens employed with BTTST achieved higher shear capacities compared to those that rely only on a natural bond between cold-formed steel and concrete and specimens with Lakkavalli and Liu bent-up tab (LYLB). Load capacities for push-out test specimens with BTTST are ii relatively higher as compared to the equivalent control specimen, i.e. by 91% to 135%. When compared to LYLB specimens the increment is 12% to 16%. In addition, shear capacities of BTTST also increase with the increase in dimensions (size and angle) of BTTST, thickness of CFS and concrete compressive strength. An equation was developed to determine the shear capacity of BTTST and the value is in good agreement with the observed test values. The average absolute difference between the test values and predicted values was found to be 8.07%. The average arithmetic mean of the test/predicted ratio (n) of this equation is 0.9954. The standard deviation (a) and the coefficient of variation (CV) for the proposed equation were 0.09682 and 9.7%, respectively. The proposed equation is recommended for the design of BTTST in CFSconcrete composite beams. In large-scale testing, specimens employed with BTTST increased the strength capacities and reduced the deflection of the specimens. The moment capacities, MU ) e X p for all specimens are above Mu>theory and show good agreement with the calculated ratio (>1.00). It is also found that, strength capacities of CFS-concrete composite beams also increase with the increase in dimensions (size and angle) of BTTST, thickness of CFS and concrete compressive strength and a CFS-concrete composite beam are practically designed with partial shear connection for equal moment capacity by reducing number of BTTST. It is concluded that the proposed BTTST shear transfer enhancement in CFS-concrete composite beams has sufficient strength and is also feasible. Finally, a standard table of characteristic resistance, P t a b of BTTST in normal weight concrete, was also developed to simplify the design calculation of CFSconcrete composite beams

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Lidar-based Obstacle Detection and Recognition for Autonomous Agricultural Vehicles

Today, agricultural vehicles are available that can drive autonomously and follow exact route plans more precisely than human operators. Combined with advancements in precision agriculture, autonomous agricultural robots can reduce manual labor, improve workflow, and optimize yield. However, as of today, human operators are still required for monitoring the environment and acting upon potential obstacles in front of the vehicle. To eliminate this need, safety must be ensured by accurate and reliable obstacle detection and avoidance systems.In this thesis, lidar-based obstacle detection and recognition in agricultural environments has been investigated. A rotating multi-beam lidar generating 3D point clouds was used for point-wise classification of agricultural scenes, while multi-modal fusion with cameras and radar was used to increase performance and robustness. Two research perception platforms were presented and used for data acquisition. The proposed methods were all evaluated on recorded datasets that represented a wide range of realistic agricultural environments and included both static and dynamic obstacles.For 3D point cloud classification, two methods were proposed for handling density variations during feature extraction. One method outperformed a frequently used generic 3D feature descriptor, whereas the other method showed promising preliminary results using deep learning on 2D range images. For multi-modal fusion, four methods were proposed for combining lidar with color camera, thermal camera, and radar. Gradual improvements in classification accuracy were seen, as spatial, temporal, and multi-modal relationships were introduced in the models. Finally, occupancy grid mapping was used to fuse and map detections globally, and runtime obstacle detection was applied on mapped detections along the vehicle path, thus simulating an actual traversal.The proposed methods serve as a first step towards full autonomy for agricultural vehicles. The study has thus shown that recent advancements in autonomous driving can be transferred to the agricultural domain, when accurate distinctions are made between obstacles and processable vegetation. Future research in the domain has further been facilitated with the release of the multi-modal obstacle dataset, FieldSAFE

Visual attention and swarm cognition for off-road robots

Tese de doutoramento, Informática (Engenharia Informática), Universidade de Lisboa, Faculdade de Ciências, 2011Esta tese aborda o problema da modelação de atenção visual no contexto de robôs autónomos todo-o-terreno. O objectivo de utilizar mecanismos de atenção visual é o de focar a percepção nos aspectos do ambiente mais relevantes à tarefa do robô. Esta tese mostra que, na detecção de obstáculos e de trilhos, esta capacidade promove robustez e parcimónia computacional. Estas são características chave para a rapidez e eficiência dos robôs todo-o-terreno. Um dos maiores desafios na modelação de atenção visual advém da necessidade de gerir o compromisso velocidade-precisão na presença de variações de contexto ou de tarefa. Esta tese mostra que este compromisso é resolvido se o processo de atenção visual for modelado como um processo auto-organizado, cuja operação é modulada pelo módulo de selecção de acção, responsável pelo controlo do robô. Ao fechar a malha entre o processo de selecção de acção e o de percepção, o último é capaz de operar apenas onde é necessário, antecipando as acções do robô. Para fornecer atenção visual com propriedades auto-organizadas, este trabalho obtém inspiração da Natureza. Concretamente, os mecanismos responsáveis pela capacidade que as formigas guerreiras têm de procurar alimento de forma auto-organizada, são usados como metáfora na resolução da tarefa de procurar, também de forma auto-organizada, obstáculos e trilhos no campo visual do robô. A solução proposta nesta tese é a de colocar vários focos de atenção encoberta a operar como um enxame, através de interacções baseadas em feromona. Este trabalho representa a primeira realização corporizada de cognição de enxame. Este é um novo campo de investigação que procura descobrir os princípios básicos da cognição, inspeccionando as propriedades auto-organizadas da inteligência colectiva exibida pelos insectos sociais. Logo, esta tese contribui para a robótica como disciplina de engenharia e para a robótica como disciplina de modelação, capaz de suportar o estudo do comportamento adaptável.Esta tese aborda o problema da modelação de atenção visual no contexto de robôs autónomos todo-o-terreno. O objectivo de utilizar mecanismos de atenção visual é o de focar a percepção nos aspectos do ambiente mais relevantes à tarefa do robô. Esta tese mostra que, na detecção de obstáculos e de trilhos, esta capacidade promove robustez e parcimónia computacional. Estas são características chave para a rapidez e eficiência dos robôs todo-o-terreno. Um dos maiores desafios na modelação de atenção visual advém da necessidade de gerir o compromisso velocidade-precisão na presença de variações de contexto ou de tarefa. Esta tese mostra que este compromisso é resolvido se o processo de atenção visual for modelado como um processo auto-organizado, cuja operação é modulada pelo módulo de selecção de acção, responsável pelo controlo do robô. Ao fechar a malha entre o processo de selecção de acção e o de percepção, o último é capaz de operar apenas onde é necessário, antecipando as acções do robô. Para fornecer atenção visual com propriedades auto-organizadas, este trabalho obtém inspi- ração da Natureza. Concretamente, os mecanismos responsáveis pela capacidade que as formi- gas guerreiras têm de procurar alimento de forma auto-organizada, são usados como metáfora na resolução da tarefa de procurar, também de forma auto-organizada, obstáculos e trilhos no campo visual do robô. A solução proposta nesta tese é a de colocar vários focos de atenção encoberta a operar como um enxame, através de interacções baseadas em feromona. Este trabalho representa a primeira realização corporizada de cognição de enxame. Este é um novo campo de investigação que procura descobrir os princípios básicos da cognição, ins- peccionando as propriedades auto-organizadas da inteligência colectiva exibida pelos insectos sociais. Logo, esta tese contribui para a robótica como disciplina de engenharia e para a robótica como disciplina de modelação, capaz de suportar o estudo do comportamento adaptável.Fundação para a Ciência e a Tecnologia (FCT,SFRH/BD/27305/2006); Laboratory of Agent Modelling (LabMag