Automated Ortho-Rectification of UAV-Based Hyperspectral Data over an Agricultural Field Using Frame RGB Imagery

Low-cost Unmanned Airborne Vehicles (UAVs) equipped with consumer-grade imaging systems have emerged as a potential remote sensing platform that could satisfy the needs of a wide range of civilian applications. Among these applications, UAV-based agricultural mapping and monitoring have attracted significant attention from both the research and professional communities. The interest in UAV-based remote sensing for agricultural management is motivated by the need to maximize crop yield. Remote sensing-based crop yield prediction and estimation are primarily based on imaging systems with different spectral coverage and resolution (e.g., RGB and hyperspectral imaging systems). Due to the data volume, RGB imaging is based on frame cameras, while hyperspectral sensors are primarily push-broom scanners. To cope with the limited endurance and payload constraints of low-cost UAVs, the agricultural research and professional communities have to rely on consumer-grade and light-weight sensors. However, the geometric fidelity of derived information from push-broom hyperspectral scanners is quite sensitive to the available position and orientation established through a direct geo-referencing unit onboard the imaging platform (i.e., an integrated Global Navigation Satellite System (GNSS) and Inertial Navigation System (INS). This paper presents an automated framework for the integration of frame RGB images, push-broom hyperspectral scanner data and consumer-grade GNSS/INS navigation data for accurate geometric rectification of the hyperspectral scenes. The approach relies on utilizing the navigation data, together with a modified Speeded-Up Robust Feature (SURF) detector and descriptor, for automating the identification of conjugate features in the RGB and hyperspectral imagery. The SURF modification takes into consideration the available direct geo-referencing information to improve the reliability of the matching procedure in the presence of repetitive texture within a mechanized agricultural field. Identified features are then used to improve the geometric fidelity of the previously ortho-rectified hyperspectral data. Experimental results from two real datasets show that the geometric rectification of the hyperspectral data was improved by almost one order of magnitude

High resolution thermal and multispectral UAV imagery for precision assessment of apple tree response to water stress

UMR AGAP - équipe AFEF - Architecture et fonctionnement des espèces fruitières(Edited by Pablo Gonzalez-de-Santos and Angela Ribeiro)This manuscript presents a comprehensive methodology to obtain Thermal, Visible and Near Infrared ortho-mosaics, as a previous step for the further image-based assessment of response to water stress of an experimental apple tree orchard. Using this methodology, multi-temporal ortho-mosaics of the field plot were created and accuracy of ortho-rectification and geo-location computed. Unmanned aerial vehicle (UAV) flights were performed on an irrigated apple tree orchard located in Southern France. The 6400 m² plot was composed of 520 apple trees which were disposed in 10 rows. In this field set-up, five well irrigated rows alternated with five rows submitted to progressive summer water constraints. For remote image acquisition, on 4th July, 19th July, 1st August and 6th September UAV flights with three cameras onboard (thermal, visible and near infrared) were performed at solar noon. On 1st August, five successive UAV flights were carried out at 8, 10, 12, 14 and 16 h (solar time). By using selfdeveloped software, frames were automatically extracted from the recorded thermal video and turned in the right image format. The temperature of four different targets (hot, cold, wet and dry bare soil) was continuously measured by the IR120 thermoradiometers during each flight, for radiometric calibration purpose. Based each on thirty images, all ortho-mosaics were successfully obtained. As high spatial resolution imagery requires high precision geo-location, and the root mean squared error (RMSE) of each ortho-mosaic positioning was calculated in order to assess its spatial accuracy. RMSE values were less than twice the pixel size in every case, which allowed a precise overlapping of the mosaics created. Canopy temperature data extracted from thermal images for showed significantly higher temperatures in water stressed trees compared to well irrigated, difference being related to severity of water stress. Thanks to the ultrahigh resolution of remote images obtained (<0.1m spatial resolution for thermal infrared images), and beyond its capacity to delineate efficiently each individual tree, the methodology presented here will also make it possible the analysis of intra-canopy variations and the accurate calculation of vegetation and water stress indices

Modeling the Response of Black Walnut -dominant Mixed Hardwoods to Seasonal Climate Effects with UAV-based Hyperspectral Sensor and Aerial Photogrammetry

The development of compact sensors in recent years has inspired the use of UAS-based hyperspectral and aerial imaging techniques for small-scale remote sensing applications. With increasing concerns about climate change, spectrally-derived vegetation indices (VIs) have proven useful for quantifying stress-induced vegetation response. The goal of this study was to develop predictive models and assess methodology for modeling the biological response of a black walnut -dominant mixed hardwood stand to seasonal climate events using UAV-based hyperspectral remote-sensing. The derived VIs were evaluated against the means of four seasonal measures of climate calculated for a two-week period prior to the flight date. A best subsets regression was used to create best fitting linear regression models according to Bayesian Information Criterion (BIC). The highest-ranked model for total precipitation had an AdjR² of 0.0839 and RMSE of 0.0827 inches. The highest-ranked model for maximum air temperature had an AdjR² of 0.9922 and RMSE of 0.5485 °F. The highest-ranked model for average air temperature had an AdjR² of 0.9987 and RMSE of 0.2256 °F. The highest-ranked model for total solar radiation had an AdjR² of 0.9961 and RMSE of 0.06405 MJ/M². The results indicate that select VIs measured at the canopy level may be useful in estimating the response to at least some measures seasonal climate. The proposed regression models could help local researchers and landowners in making short-term management decisions, as well as further our understanding of climate-induced tree stress for maintaining sustainable forests in Missouri

UAV-Multispectral Sensed Data Band Co-Registration Framework

Precision farming has greatly benefited from new technologies over the years. The use of multispectral and hyperspectral sensors coupled to Unmanned Aerial Vehicles (UAV) has enabled farms to monitor crops, improve the use of resources and reduce costs. Despite being widely used, multispectral images present a natural misalignment among the various spectra due to the use of different sensors. The variation of the analyzed spectrum also leads to a loss of characteristics among the bands which hinders the feature detection process among the bands, which makes the alignment process complex. In this work, we propose a new framework for the band co-registration process based on two premises: i) the natural misalignment is an attribute of the camera, so it does not change during the acquisition process; ii) the speed of displacement of the UAV when compared to the speed between the acquisition of the first to the last band, is not sufficient to create significant distortions. We compared our results with the ground-truth generated by a specialist and with other methods present in the literature. The proposed framework had an average back-projection (BP) error of 0.425 pixels, this result being 335% better than the evaluated frameworks.CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível SuperiorDissertação (Mestrado)A agricultura de precisão se beneficiou muito das novas tecnologias ao longo dos anos. O uso de sensores multiespectrais e hiperespectrais acoplados aos Veículos Aéreos Não Tripulados (VANT) permitiu que as fazendas monitorassem as lavouras, melhorassem o uso de recursos e reduzissem os custos. Apesar de amplamente utilizadas, as imagens multiespectrais apresentam um desalinhamento natural entre os vários espectros devido ao uso de diferentes sensores. A variação do espectro analisado também leva à perda de características entre as bandas, o que dificulta o processo de detecção de atributos entre as bandas, o que torna complexo o processo de alinhamento. Neste trabalho, propomos um novo framework para o processo de alinhamento entre as bandas com base em duas premissas: i) o desalinhamento natural é um atributo da câmera, e por esse motivo ele não é alterado durante o processo de aquisição; ii) a velocidade de deslocamento do VANT, quando comparada à velocidade entre a aquisição da primeira e a última banda, não é suficiente para criar distorções significativas. Os resultados obtidos foram comparados com o padrão ouro gerado por um especialista e com outros métodos presentes na literatura. O framework proposto teve um back-projection error (BP) de 0, 425 pixels, sendo este resultado 335% melhor aos frameworks avaliados

Implications of Sensor Inconsistencies and Remote Sensing Error in the Use of Small Unmanned Aerial Systems for Generation of Information Products for Agricultural Management

Small, unmanned aerial systems (sUAS) for remote sensing represent a relatively new and growing technology to support decisions for agricultural operations. The size and power limitations of these systems present challenges for the weight, size, and capability of the sensors that can be carried, as well as the geographical coverage that is possible. These factors, together with a lack of standards for sensor technology, its deployment, and data analysis, lead to uncertainties in data quality that can be difficult to detect or characterize. These, in turn, limit comparability between data from different sources and, more importantly, imply limits on the analyses that can be accomplished with the data that are acquired with sUAS. This paper offers a simple statistical examination of the implications toward information products of an array of sensor data uncertainty issues. The analysis relies upon high-resolution data collected in 2016 over a commercial vineyard, located near Lodi, California, for the USDA Agricultural Research Service Grape Remote sensing Atmospheric Profile and Evapotranspiration eXperiment (GRAPEX) Program. A Monte Carlo analysis is offered of how uncertainty in sensor spectral response and/or orthorectification accuracy can affect the estimation of information products of potential interest to growers, as illustrated in the form of common vegetation indices

Hyperspectral Imaging for Fine to Medium Scale Applications in Environmental Sciences

The aim of the Special Issue “Hyperspectral Imaging for Fine to Medium Scale Applications in Environmental Sciences” was to present a selection of innovative studies using hyperspectral imaging (HSI) in different thematic fields. This intention reflects the technical developments in the last three decades, which have brought the capacity of HSI to provide spectrally, spatially and temporally detailed data, favoured by e.g., hyperspectral snapshot technologies, miniaturized hyperspectral sensors and hyperspectral microscopy imaging. The present book comprises a suite of papers in various fields of environmental sciences—geology/mineral exploration, digital soil mapping, mapping and characterization of vegetation, and sensing of water bodies (including under-ice and underwater applications). In addition, there are two rather methodically/technically-oriented contributions dealing with the optimized processing of UAV data and on the design and test of a multi-channel optical receiver for ground-based applications. All in all, this compilation documents that HSI is a multi-faceted research topic and will remain so in the future

An under-ice hyperspectral and RGB imaging system to capture fine-scale biophysical properties of sea ice

Sea-ice biophysical properties are characterized by high spatio-temporal variability ranging from the meso- to the millimeter scale. Ice coring is a common yet coarse point sampling technique that struggles to capture such variability in a non-invasive manner. This hinders quantification and understanding of ice algae biomass patchiness and its complex interaction with some of its sea ice physical drivers. In response to these limitations, a novel under-ice sled system was designed to capture proxies of biomass together with 3D models of bottom topography of land-fast sea-ice. This system couples a pushbroom hyperspectral imaging (HI) sensor with a standard digital RGB camera and was trialed at Cape Evans, Antarctica. HI aims to quantify per-pixel chlorophyll-a content and other ice algae biological properties at the ice-water interface based on light transmitted through the ice. RGB imagery processed with digital photogrammetry aims to capture under-ice structure and topography. Results from a 20 m transect capturing a 0.61 m wide swath at sub-mm spatial resolution are presented. We outline the technical and logistical approach taken and provide recommendations for future deployments and developments of similar systems. A preliminary transect subsample was processed using both established and novel under-ice bio-optical indices (e.g., normalized difference indexes and the area normalized by the maximal band depth) and explorative analyses (e.g., principal component analyses) to establish proxies of algal biomass. This first deployment of HI and digital photogrammetry under-ice provides a proof-of-concept of a novel methodology capable of delivering non-invasive and highly resolved estimates of ice algal biomass in-situ, together with some of its environmental drivers. Nonetheless, various challenges and limitations remain before our method can be adopted across a range of sea-ice conditions. Our work concludes with suggested solutions to these challenges and proposes further method and system developments for future research

드론을 활용한 위성 지표반사도 산출물 공간 패턴 분석

학위논문(석사) -- 서울대학교대학원 : 농업생명과학대학 생태조경·지역시스템공학부(생태조경학), 2021.8. 조대솔.High-resolution satellites are assigned to monitor land surface in detail. The reliable surface reflectance (SR) is the fundamental in terrestrial ecosystem modeling so the temporal and spatial validation is essential. Usually based on multiple ground control points (GCPs), field spectroscopy guarantees the temporal continuity. Due to limited sampling, however, it hardly illustrates the spatial pattern. As a map, the pixelwise spatial variability of SR products is not well-documented. In this study, we introduced drone-based hyperspectral image (HSI) as a reference and compared the map with Sentinel 2 and Landsat 8 SR products on a heterogeneous rice paddy landscape. First, HSI was validated by field spectroscopy and swath overlapping, which assured qualitative radiometric accuracy within the viewing geometry. Second, HSI was matched to the satellite SRs. It involves spectral and spatial aggregation, co-registration and nadir bidirectional reflectance distribution function (BRDF)-adjusted reflectance (NBAR) conversion. Then, we 1) quantified the spatial variability of the satellite SRs and the vegetation indices (VIs) including NDVI and NIRv by APU matrix, 2) qualified them pixelwise by theoretical error budget and 3) examined the improvement by BRDF normalization. Sentinel 2 SR exhibits overall good agreement with drone HSI while the two NIRs are biased up to 10%. Despite the bias in NIR, the NDVI shows a good match on vegetated areas and the NIRv only displays the discrepancy on built-in areas. Landsat 8 SR was biased over the VIS bands (-9 ~ -7.6%). BRDF normalization just contributed to a minor improvement. Our results demonstrate the potential of drone HSI to replace in-situ observation and evaluate SR or atmospheric correction algorithms over the flat terrain. Future researches should replicate the results over the complex terrain and canopy structure (i.e. forest).원격탐사에서 지표 반사도(SR)는 지표정보를 비파괴적이고 즉각적인 방법으로 전달해주는 매개체 역할을 한다. 신뢰할 수 있는 SR은 육상 생태계 모델링의 기본이고, 이에 따라 SR의 시공간적 검증이 요구된다. 일반적으로 SR은 여러 지상 기준점(GCP)을 기반으로 하는 현장 분광법을 통해서 시간적 연속성이 보장된다. 그러나 현장 분광법은 제한적인 샘플링으로 공간 패턴을 거의 보여주지 않아, 위성 SR의 픽셀 별 공간 변동성은 잘 분석되지 않았다. 본 연구에서는 드론 기반의 초분광 영상(HSI)을 참고자료로 도입하여, 이를 이질적인 논 경관에서 Sentinel 2 및 Landsat 8 SR과 비교하였다. 우선, 드론 HSI는 현장 분광법 및 경로 중첩을 통해서 관측각도 범위 내에서 정성적인 방사 측정을 보장한다고 검증되었다. 이후, 드론 HSI는 위성 SR의 분광반응특성, 공간해상도 및 좌표계를 기준으로 맞춰졌고, 관측 기하를 통일하기 위해서 드론 HIS와 위성 SR은 각각 양방향반사율분포함수 (BRDF) 정규화 반사도 (NBAR)로 변환되었다. 마지막으로, 1) APU 행렬으로 위성 SR과 NDVI, NIRv를 포함하는 식생지수(VI)의 공간변동성을 정량화 했고, 2) 대기보정의 이론적 오차를 기준으로 SR과 VI를 픽셀별로 평가했고, 3) BRDF 정규화를 통한 개선 사항을 검토했다. Sentinel 2 SR은 드론 HSI와 전반적으로 좋은 일치를 보이나, 두 NIR 채널은 최대 10% 편향되었다. NIR의 편향은 식생지수에서 토지 피복에 따라 다른 영향을 미쳤다. NDVI는 식생에서는 낮은 편향을 보여줬고, NIRv는 도시시설물 영역에서만 높은 편향을 보였다. Landsat 8 SR은 VIS 채널에 대해 편향되었다 (-9 ~ -7.6%). BRDF 정규화는 위성 SR의 품질을 개선했지만, 그 영향은 부수적이었다. 본 연구에서는 평탄한 지형에서 드론 HSI가 현장 관측을 대체할 수 있고, 따라서 위성 SR이나 대기보정 알고리즘을 평가하는데 활용될 수 있다는 것을 보였다. 향후 연구에서는 산림으로 대상지를 확대하여, 지형과 캐노피 구조가 드론 HSI 및 위성 SR에 미치는 영향을 분석할 필요가 있다.Chapter 1. Introduction 1 1.1 Background 1 Chapter 2. Method 3 2.1 Study Site 3 2.2 Drone campaign 4 2.3 Data processing 4 2.3.1 Sensor calibration 5 2.3.2 Bidirectional reflectance factor (BRF) calculation 7 2.3.3 BRDF correction 7 2.3.4 Orthorectification 8 2.3.5 Spatial Aggregation 9 2.3.6 Co-registration 10 2.4 Satellite dataset 10 2.4.2 Landsat 8 12 Chapter 3. Result and Discussion 12 3.1 Drone BRF map quality assessment 12 3.1.1 Radiometric accuracy 12 3.1.2 BRDF effect 15 3.2 Spatial variability in satellite surface reflectance product 16 3.2.1 Sentinel 2B (10m) 17 3.2.2 Sentinel 2B (20m) 22 3.2.3 Landsat 8 26 Chapter 4. Conclusion 28 Supplemental Materials 30 Bibliography 34 Abstract in Korean 43석