Autonomous UAS-Based Agriculture Applications: General Overview and Relevant European Case Studies

Emerging precision agriculture techniques rely on the frequent collection of high-quality data which can be acquired efficiently by unmanned aerial systems (UAS). The main obstacle for wider adoption of this technology is related to UAS operational costs. The path forward requires a high degree of autonomy and integration of the UAS and other cyber physical systems on the farm into a common Farm Management System (FMS) to facilitate the use of big data and artificial intelligence (AI) techniques for decision support. Such a solution has been implemented in the EU project AFarCloud (Aggregated Farming in the Cloud). The regulation of UAS operations is another important factor that impacts the adoption rate of agricultural UAS. An analysis of the new European UAS regulations relevant for autonomous operation is included. Autonomous UAS operation through the AFarCloud FMS solution has been demonstrated at several test farms in multiple European countries. Novel applications have been developed, such as the retrieval of data from remote field sensors using UAS and in situ measurements using dedicated UAS payloads designed for physical contact with the environment. The main findings include that (1) autonomous UAS operation in the agricultural sector is feasible once the regulations allow this; (2) the UAS should be integrated with the FMS and include autonomous data processing and charging functionality to offer a practical solution; and (3) several applications beyond just asset monitoring are relevant for the UAS and will help to justify the cost of this equipment.publishedVersio

Soft-tissue evidence for homeothermy and crypsis in a Jurassic ichthyosaur

Ichthyosaurs are extinct marine reptiles that display a notable external similarity to modern toothed whales. Here we show that this resemblance is more than skin deep. We apply a multidisciplinary experimental approach to characterize the cellular and molecular composition of integumental tissues in an exceptionally preserved specimen of the Early Jurassic ichthyosaur Stenopterygius. Our analyses recovered still-flexible remnants of the original scaleless skin, which comprises morphologically distinct epidermal and dermal layers. These are underlain by insulating blubber that would have augmented streamlining, buoyancy and homeothermy. Additionally, we identify endogenous proteinaceous and lipid constituents, together with keratinocytes and branched melanophores that contain eumelanin pigment. Distributional variation of melanophores across the body suggests countershading, possibly enhanced by physiological adjustments of colour to enable photoprotection, concealment and/or thermoregulation. Convergence of ichthyosaurs with extant marine amniotes thus extends to the ultrastructural and molecular levels, reflecting the omnipresent constraints of their shared adaptation to pelagic life

Embedded high-resolution stereo-vision of high frame-rate and low latency through FPGA-acceleration

Autonomous agents rely on information from the surrounding environment to act upon. In the array of sensors available, the image sensor is perhaps the most versatile, allowing for detection of colour, size, shape, and depth. For the latter, in a dynamic environment, assuming no a priori knowledge, stereo vision is a commonly adopted technique. How to interpret images, and extract relevant information, is referred to as computer vision. Computer vision, and specifically stereo-vision algorithms, are complex and computationally expensive, already considering a single stereo pair, with results that are, in terms of accuracy, qualitatively difficult to compare. Adding to the challenge is a continuous stream of images, of a high frame rate, and the race of ever increasing image resolutions. In the context of autonomous agents, considerations regarding real-time requirements, embedded/resource limited processing platforms, power consumption, and physical size, further add up to an unarguably challenging problem. This thesis aims to achieve embedded high-resolution stereo-vision of high frame-rate and low latency, by approaching the problem from two different angles, hardware and algorithmic development, in a symbiotic relationship. The first contributions of the thesis are the GIMME and GIMME2 embedded vision platforms, which offer hardware accelerated processing through FGPAs, specifically targeting stereo vision, contrary to available COTS systems at the time. The second contribution, toward stereo vision algorithms, is twofold. Firstly, the problem of scalability and the associated disparity range is addressed by proposing a segment-based stereo algorithm. In segment space, matching is independent of image scale, and similarly, disparity range is measured in terms of segments, indicating relatively few hypotheses to cover the entire range of the scene. Secondly, more in line with the conventional stereo correspondence for FPGAs, the Census Transform (CT) has been identified as a recurring cost metric. This thesis proposes an optimisation of the CT through a Genetic Algorithm (GA) - the Genetic Algorithm Census Transform (GACT). The GACT shows promising results for benchmark datasets, compared to established CT methods, while being resource efficient.Autonoma agenter är beroende av information från den omgivande miljön för att agera. I en mängd av tillgängliga sensorer är troligtvis bildsensorn den mest mångsidiga, då den möjliggör särskillnad av färg, storlek, form och djup. För det sistnämnda är, i en dynamisk miljö utan krav på förkunskaper, stereovision en vanligt tillämpad teknik. Tolkning av bildinnehåll och extrahering av relevant information går under benämningen datorseende. Datorseende, och specifikt stereoalgoritmer, är redan för ett enskilt bildpar komplexa och beräkningsmässigt kostsamma, och ger resultat som, i termer av noggrannhet, är kvalitativt svåra att jämföra. Problematiken utökas vidare av kontinuerlig ström av bilder, med allt högre bildfrekvens och upplösning. För autonoma agenter krävs dessutom överväganden vad gäller realtidskrav, inbyggda system/resursbegränsade beräkningsplattformar, strömförbrukning och fysisk storlek, vilket summerar till ett otvetydigt utmanande problem. Den här avhandlingen syftar till att åstadkomma högupplöst stereovision med hög uppdateringsfrekvens och låg latens på inbyggda system. Genom att närma sig problemet från två olika vinklar, hårdvaru- och algoritmmässigt, kan ett symbiotiskt förhållande däremellan säkerställas.Avhandlingens första bidrag är GIMME och GIMME2 inbyggda visionsplattformar, som erbjuder FPGA-baserad hårdvaruaccelerering, med särskilt fokus på stereoseende, i kontrast till för tidpunkten kommersiellt tillgängliga system.Det andra bidraget, härrörande stereoalgoritmer, är tudelat.Först hanteras skalbarhetproblemet, sammankopplat med disparitetsomfånget, genom att föreslå en segmentbaserad stereoalgoritm.I segmentrymden är matchningen oberoende av bildupplösningen, samt att disparitetsomfånget definieras i termer av segment, vilket antyder att relativt få hypoteser behövs för att omfatta hela scenen.I det andra bidraget på algoritmnivå, mer i linje med konventionella stereoalgoritmer för FPGAer, har Censustransformen (CT) identifierats som ett återkommande kostnadsmått för likhet. Här föreslås en optimering av CT genom att tillämpa genetisk algoritm (GA) - Genetisk Algoritm Census Transform (GACT). GACT visar lovande resultat för referensdataset jämfört med etablerade CT-metoder, men är samtidigt resurseffektiv

Optisk detektion och höjdestimering av kraftledningar relativt till en UAV

Inspection of power lines is a crucial activity to locate the source of power outages or to preemptively stop them. Visual inspection methods using optical cameras attached to Unmanned Aerial Vehicles (UAV) have become increasingly popular due to cheap operating costs, large areal coverage and the wide availability of these platforms. The efficiency of power line inspection can further be improved using an automated UAV system, though the UAV must maintain a safe distance from the power lines to reduce the risk of collision. This thesis proposes a method of detecting visually straight power line cables and estimating the position of a UAV with respect to the detections in real time. Two power line detection strategies are investigated; a Canny and a Laplacian edge detection based approach. The algorithms are evaluated on power lines images in a range of lighting conditions, backgrounds and transmission cables. The Canny approach is chosen as the line detection method due to a superior performance when evaluated with a recall and precision line detection metric. Using this approach, UAV translational information with respect to the power lines is calculated by using the distance between the outermost power line cables and UAV odometry measurements. A complete evaluation of the algorithm is performed in both a simulated environment and on recordings of real power lines. The UAV translational error relative to the power lines was found to be below 0.1 m in general when using a simulated environment. Using real data, where only the altitude could be evaluated, the proposed method achieved an error of 1.6 m. The greater error was caused by inaccurate power line detections in conditions with poor cable visibility and uncertain ground truth values due to terrain approximations. Overall, the proposed algorithm provides a UAV translation estimate with respect to transmission lines in real time when individual cables are easily distinguishable from the background, thus improving the capabilities of automated power line inspection. Kraftledningsinspektioner är ett nödvändigt arbete för att hitta källan till strömavbrott eller för att kunna motverka dem i förebyggande syfte. Inspektionsmetoder som använder optiska kameror monterade på obemannade luftfarkoster (UAV) har blivit allt mer vanliga på grund av utbudet och billiga operationskostnader. Inspektionen kan ytterligare förbättras genom autonoma UAV- system, dock krävs det att en UAV håller ett säkert avstånd till kraftledningarna för att minska kollisionsrisken. Det här examensarbetet föreslår en metod avsett för detektion av raka kraftledningar och en positionsuppskattning av en UAV relativt detektionerna i realtid. Två metoder för kraftledningsdetektion undersöks: en Canny och en Laplaciansk metod för kantdetektion. Algoritmerna utvärderas på bilder innehållande kraftledningar i olika belysningar, bakgrunder och typer av kraftledningar. Metoden baserat på Canny- algoritmen väljs för kraftledningsdektetion då metoden har överlägsen prestanda när precision och sensitivitet används som prestandamätvärden. Med denna metod beräknades UAV-positionen med anseende till kraftledningar genom användningen av avståndet mellan de yttersta kraftledningarna och UAV- odometrin. En omfattande utvärdering av hela algoritmen utförs i både en simulerad miljö och på inspelningar av verkliga kraftledningar. Felet för positionsuppskattningen i den simulerade miljön är generellt under 0.1 m, men ökade vid utvärderingen på verklig data. Specifikt för den uppskattade höjden är felet 1.6 m på verkliga kraftledningar. Det större felet orsakas av felaktiga kraftledningsdetektioner i förhållanden med undermålig belysning och osäkra referensvärden. Överlag kan den föreslagna algoritmen skapa en UAV- positionsestimering med avseende på kraftledningar i realtid när enskilda kraftledningar är urskiljbara från bakgrunden. Därav har förmågan att utföra autonoma kraftledningsinspektioner under dessa förhållanden utökats

Optisk detektion och höjdestimering av kraftledningar relativt till en UAV

Inspection of power lines is a crucial activity to locate the source of power outages or to preemptively stop them. Visual inspection methods using optical cameras attached to Unmanned Aerial Vehicles (UAV) have become increasingly popular due to cheap operating costs, large areal coverage and the wide availability of these platforms. The efficiency of power line inspection can further be improved using an automated UAV system, though the UAV must maintain a safe distance from the power lines to reduce the risk of collision. This thesis proposes a method of detecting visually straight power line cables and estimating the position of a UAV with respect to the detections in real time. Two power line detection strategies are investigated; a Canny and a Laplacian edge detection based approach. The algorithms are evaluated on power lines images in a range of lighting conditions, backgrounds and transmission cables. The Canny approach is chosen as the line detection method due to a superior performance when evaluated with a recall and precision line detection metric. Using this approach, UAV translational information with respect to the power lines is calculated by using the distance between the outermost power line cables and UAV odometry measurements. A complete evaluation of the algorithm is performed in both a simulated environment and on recordings of real power lines. The UAV translational error relative to the power lines was found to be below 0.1 m in general when using a simulated environment. Using real data, where only the altitude could be evaluated, the proposed method achieved an error of 1.6 m. The greater error was caused by inaccurate power line detections in conditions with poor cable visibility and uncertain ground truth values due to terrain approximations. Overall, the proposed algorithm provides a UAV translation estimate with respect to transmission lines in real time when individual cables are easily distinguishable from the background, thus improving the capabilities of automated power line inspection. Kraftledningsinspektioner är ett nödvändigt arbete för att hitta källan till strömavbrott eller för att kunna motverka dem i förebyggande syfte. Inspektionsmetoder som använder optiska kameror monterade på obemannade luftfarkoster (UAV) har blivit allt mer vanliga på grund av utbudet och billiga operationskostnader. Inspektionen kan ytterligare förbättras genom autonoma UAV- system, dock krävs det att en UAV håller ett säkert avstånd till kraftledningarna för att minska kollisionsrisken. Det här examensarbetet föreslår en metod avsett för detektion av raka kraftledningar och en positionsuppskattning av en UAV relativt detektionerna i realtid. Två metoder för kraftledningsdetektion undersöks: en Canny och en Laplaciansk metod för kantdetektion. Algoritmerna utvärderas på bilder innehållande kraftledningar i olika belysningar, bakgrunder och typer av kraftledningar. Metoden baserat på Canny- algoritmen väljs för kraftledningsdektetion då metoden har överlägsen prestanda när precision och sensitivitet används som prestandamätvärden. Med denna metod beräknades UAV-positionen med anseende till kraftledningar genom användningen av avståndet mellan de yttersta kraftledningarna och UAV- odometrin. En omfattande utvärdering av hela algoritmen utförs i både en simulerad miljö och på inspelningar av verkliga kraftledningar. Felet för positionsuppskattningen i den simulerade miljön är generellt under 0.1 m, men ökade vid utvärderingen på verklig data. Specifikt för den uppskattade höjden är felet 1.6 m på verkliga kraftledningar. Det större felet orsakas av felaktiga kraftledningsdetektioner i förhållanden med undermålig belysning och osäkra referensvärden. Överlag kan den föreslagna algoritmen skapa en UAV- positionsestimering med avseende på kraftledningar i realtid när enskilda kraftledningar är urskiljbara från bakgrunden. Därav har förmågan att utföra autonoma kraftledningsinspektioner under dessa förhållanden utökats

Modeling Far Ultraviolet Auroral Ovals at Ganymede

Ganymede, one of Jupiters moons, differs from other moons in the Solar System as it has its own magnetic field. This rare property shapes the morphology on the existing far ultraviolet oxygen auroral ovals on the celestial body in the northern and southern hemisphere created by high energy electrons colliding into the atmosphere.With the help of the Hubble Space Telescope (HST) this phenomenon has been captured and analyzed multiple times during the past 20 years using the on-board Space Telescope Imaging Spectrograph (STIS). The ultimate goal of this project is recreating the far ultraviolet oxygen auroral emissions on Ganymede as a 3D computer model in MATLAB by using the data recovered from HST.The method used to reach this goal was to implement a model with main characteristics of the auroral ovals, project it onto a plane and then use a Cauchy distribution to filter the model. To compare the model with images from HST, a χ2-value was calculated for every pixel in each image. To further improvethe model the Nelder-Mead Simplex optimization method was applied.The project succeeded in such a way that the final model created views of the locations and the appearance of the bright spots that represent the auroral ovals around Ganymede with an accurate result in relation to the given data

Modeling Far Ultraviolet Auroral Ovals at Ganymede

Ganymede, one of Jupiters moons, differs from other moons in the Solar System as it has its own magnetic field. This rare property shapes the morphology on the existing far ultraviolet oxygen auroral ovals on the celestial body in the northern and southern hemisphere created by high energy electrons colliding into the atmosphere.With the help of the Hubble Space Telescope (HST) this phenomenon has been captured and analyzed multiple times during the past 20 years using the on-board Space Telescope Imaging Spectrograph (STIS). The ultimate goal of this project is recreating the far ultraviolet oxygen auroral emissions on Ganymede as a 3D computer model in MATLAB by using the data recovered from HST.The method used to reach this goal was to implement a model with main characteristics of the auroral ovals, project it onto a plane and then use a Cauchy distribution to filter the model. To compare the model with images from HST, a χ2-value was calculated for every pixel in each image. To further improvethe model the Nelder-Mead Simplex optimization method was applied.The project succeeded in such a way that the final model created views of the locations and the appearance of the bright spots that represent the auroral ovals around Ganymede with an accurate result in relation to the given data

Modeling Far Ultraviolet Auroral Ovals at Ganymede

Ganymede, one of Jupiters moons, differs from other moons in the Solar System as it has its own magnetic field. This rare property shapes the morphology on the existing far ultraviolet oxygen auroral ovals on the celestial body in the northern and southern hemisphere created by high energy electrons colliding into the atmosphere.With the help of the Hubble Space Telescope (HST) this phenomenon has been captured and analyzed multiple times during the past 20 years using the on-board Space Telescope Imaging Spectrograph (STIS). The ultimate goal of this project is recreating the far ultraviolet oxygen auroral emissions on Ganymede as a 3D computer model in MATLAB by using the data recovered from HST.The method used to reach this goal was to implement a model with main characteristics of the auroral ovals, project it onto a plane and then use a Cauchy distribution to filter the model. To compare the model with images from HST, a χ2-value was calculated for every pixel in each image. To further improvethe model the Nelder-Mead Simplex optimization method was applied.The project succeeded in such a way that the final model created views of the locations and the appearance of the bright spots that represent the auroral ovals around Ganymede with an accurate result in relation to the given data