Classification of 3D Point Clouds Using Color Vegetation Indices for Precision Viticulture and Digitizing Applications

Remote sensing applied in the digital transformation of agriculture and, more particularly, in precision viticulture offers methods to map field spatial variability to support site-specific management strategies; these can be based on crop canopy characteristics such as the row height or vegetation cover fraction, requiring accurate three-dimensional (3D) information. To derive canopy information, a set of dense 3D point clouds was generated using photogrammetric techniques on images acquired by an RGB sensor onboard an unmanned aerial vehicle (UAV) in two testing vineyards on two different dates. In addition to the geometry, each point also stores information from the RGB color model, which was used to discriminate between vegetation and bare soil. To the best of our knowledge, the new methodology herein presented consisting of linking point clouds with their spectral information had not previously been applied to automatically estimate vine height. Therefore, the novelty of this work is based on the application of color vegetation indices in point clouds for the automatic detection and classification of points representing vegetation and the later ability to determine the height of vines using as a reference the heights of the points classified as soil. Results from on-ground measurements of the heights of individual grapevines were compared with the estimated heights from the UAV point cloud, showing high determination coefficients (R² > 0.87) and low root-mean-square error (0.070 m). This methodology offers new capabilities for the use of RGB sensors onboard UAV platforms as a tool for precision viticulture and digitizing applications

Forestry and Arboriculture Applications Using High-Resolution Imagery from Unmanned Aerial Vehicles (UAV)

Forests cover over one-third of the planet and provide unmeasurable benefits to the ecosystem. Forest managers have collected and processed countless amounts of data for use in studying, planning, and management of these forests. Data collection has evolved from completely manual operations to the incorporation of technology that has increased the efficiency of data collection and decreased overall costs. Many technological advances have been made that can be incorporated into natural resources disciplines. Laser measuring devices, handheld data collectors and more recently, unmanned aerial vehicles, are just a few items that are playing a major role in the way data is managed and collected. Field hardware has also been aided with new and improved mobile and computer software. Over the course of this study, field technology along with computer advancements have been utilized to aid in forestry and arboricultural applications. Three-dimensional point cloud data that represent tree shape and height were extracted and examined for accuracy. Traditional fieldwork collection (tree height, tree diameter and canopy metrics) was derived from remotely sensed data by using new modeling techniques which will result in time and cost savings. Using high resolution aerial photography, individual tree species are classified to support tree inventory development. Point clouds were used to create digital elevation models (DEM) which can further be used in hydrology analysis, slope, aspect, and hillshades. Digital terrain models (DTM) are in geographic information system (GIS), and along with DEMs, used to create canopy height models (CHM). The results of this study can enhance how the data are utilized and prompt further research and new initiatives that will improve and garner new insight for the use of remotely sensed data in forest management

Bayesian Methods for Radiometric Calibration in Motion Picture Encoding Workflows

A method for estimating the Camera Response Function (CRF) of an electronic motion picture camera is presented in this work. The accurate estimation of the CRF allows for proper encoding of camera exposures into motion picture post-production workflows, like the Academy Color Encoding Specification (ACES), this being a necessary step to correctly combine images from different capture sources into one cohesive final production and minimize non-creative manual adjustments. Although there are well known standard CRFs implemented in typical video camera workflows, motion picture workflows and newer High Dynamic Range (HDR) imaging workflows have introduced new standard CRFs as well as custom and proprietary CRFs that need to be known for proper post-production encoding of the camera footage. Current methods to estimate this function rely on the use of measurement charts, using multiple static images taken under different exposures or lighting conditions, or assume a simplistic model of the function’s shape. All these methods become problematic and tough to fit into motion picture production and post-production workflows where the use of test charts and varying camera or scene setups becomes impractical and where a method based solely on camera footage, comprised of a single image or a series of images, would be advantageous. This work presents a methodology initially based on the work of Lin, Gu, Yamazaki and Shum that takes into account edge color mixtures in an image or image sequence, that are affected by the non-linearity introduced by a CRF. In addition, a novel feature based on image noise is introduced to overcome some of the limitations of edge color mixtures. These features provide information that is included in the likelihood probability distribution in a Bayesian framework to estimate the CRF as the expected value of a posterior probability distribution, which is itself approximated by a Markov Chain Monte Carlo (MCMC) sampling algorithm. This allows for a more complete description of the CRF over methods like Maximum Likelihood (ML) and Maximum A Posteriori (MAP). The CRF function is modeled by Principal Component Analysis (PCA) of the Database of Response Functions (DoRF) compiled by Grossberg and Nayar, and the prior probability distribution is modeled by a Gaussian Mixture Model (GMM) of the PCA coefficients for the responses in the DoRF. CRF estimation results are presented for an ARRI electronic motion picture camera, showing the improved estimation accuracy and practicality of this method over previous methods for motion picture post-production workflows

A metrological approach for multispectral photogrammetry

This paper presents the design and development of a three-dimensional reference object for the metrological quality assessment of photogrammetry-based techniques, for application in the cultural heritage field. The reference object was 3D printed, with nominal manufacturing uncertainty of the order of 0.01 mm. The object has been realized as a dodecahedron, and in each face, a different pictorial preparation has been inserted. The preparations include several pigments, binders, and varnishes, to be representative of the materials and techniques used historically by artists. Since the reference object’s shape, size and uncertainty are known, it is possible to use this object as a reference to evaluate the quality of a 3D model from the metric point of view. In particular, verification of dimensional precision and accuracy are performed using the standard deviation on measurements acquired on the reference object and the final 3D model. In addition, the object can be used as a reference for UV-induced Visible Luminescence (UVL) acquisition, being the materials employed UV-fluorescent. Results obtained with visible-reflected and UVL images are presented and discussed

Laser Scanner Technology

Laser scanning technology plays an important role in the science and engineering arena. The aim of the scanning is usually to create a digital version of the object surface. Multiple scanning is sometimes performed via multiple cameras to obtain all slides of the scene under study. Usually, optical tests are used to elucidate the power of laser scanning technology in the modern industry and in the research laboratories. This book describes the recent contributions reported by laser scanning technology in different areas around the world. The main topics of laser scanning described in this volume include full body scanning, traffic management, 3D survey process, bridge monitoring, tracking of scanning, human sensing, three-dimensional modelling, glacier monitoring and digitizing heritage monuments

Detail Enhancing Denoising of Digitized 3D Models from a Mobile Scanning System

The acquisition process of digitizing a large-scale environment produces an enormous amount of raw geometry data. This data is corrupted by system noise, which leads to 3D surfaces that are not smooth and details that are distorted. Any scanning system has noise associate with the scanning hardware, both digital quantization errors and measurement inaccuracies, but a mobile scanning system has additional system noise introduced by the pose estimation of the hardware during data acquisition. The combined system noise generates data that is not handled well by existing noise reduction and smoothing techniques. This research is focused on enhancing the 3D models acquired by mobile scanning systems used to digitize large-scale environments. These digitization systems combine a variety of sensors – including laser range scanners, video cameras, and pose estimation hardware – on a mobile platform for the quick acquisition of 3D models of real world environments. The data acquired by such systems are extremely noisy, often with significant details being on the same order of magnitude as the system noise. By utilizing a unique 3D signal analysis tool, a denoising algorithm was developed that identifies regions of detail and enhances their geometry, while removing the effects of noise on the overall model. The developed algorithm can be useful for a variety of digitized 3D models, not just those involving mobile scanning systems. The challenges faced in this study were the automatic processing needs of the enhancement algorithm, and the need to fill a hole in the area of 3D model analysis in order to reduce the effect of system noise on the 3D models. In this context, our main contributions are the automation and integration of a data enhancement method not well known to the computer vision community, and the development of a novel 3D signal decomposition and analysis tool. The new technologies featured in this document are intuitive extensions of existing methods to new dimensionality and applications. The totality of the research has been applied towards detail enhancing denoising of scanned data from a mobile range scanning system, and results from both synthetic and real models are presented

CNN기반의 FusionNet 신경망과 농지 경계추출 알고리즘을 이용한 토지피복분류모델 개발

학위논문 (석사) -- 서울대학교 대학원 : 농업생명과학대학 생태조경.지역시스템공학부(지역시스템공학전공), 2021. 2. 송인홍.토지이용이 빠르게 변화함에 따라, 토지 피복에 대한 공간정보를 담고 있는 토지 피복 지도의 신속한 최신화는 필수적이다. 하지만, 현 토지 피복 지도는 많은 시간과 노동력을 요구하는 manual digitizing 방법으로 제작됨에 따라, 토지 피복 지도의 업데이트 및 배포에 긴 시간 간격이 발생하는 실정이다. 본 연구에서는 convolutional neural network (CNN) 기반의 인공신경망을 이용하여 high-resolution remote sensing (HRRS) 영상으로부터 토지 피복을 분류하는 모델을 개발하고, 특히 농지 경계추출 알고리즘을 적용하여 농업지역에서 분류 정확도를 개선하고자 하였다. 개발된 토지 피복 분류모델은 전처리(pre-processing) 모듈, 토지 피복 분류(land cover classification) 모듈, 그리고 후처리(post-processing) 모듈의 세 모듈로 구성된다. 전처리 모듈은 입력된 HRRS 영상을 75%씩 중첩 분할하여 관점을 다양화하는 모듈로, 한 관점에서 토지 피복을 분류할 때 발생할 수 있는 오분류를 줄이고자 하였다. 토지 피복 분류 모듈은 FusionNet model 구조를 바탕으로 개발되었고, 이는 분할된 HRRS 이미지의 픽셀별로 최적 토지 피복을 부여하도록 설계되었다. 후처리 모듈은 픽셀별 최종 토지 피복을 결정하는 모듈로, 분할된 HRRS 이미지의 분류결과를 취합하여 최빈값을 최종 토지 피복으로 결정한다. 추가로 농지에서는 농지경계를 추출하고, 필지별 분류된 토지 피복을 집계하여 한 필지에 같은 토지 피복을 부여하였다. 개발된 토지 피복 분류모델은 전라남도 지역(면적: 547 km2)의 2018년 정사영상과 토지 피복 지도를 이용하여 학습되었다. 토지 피복 분류모델 검증은 학습지역과 시간, 공간적으로 구분된, 2018년 전라남도 수북면과 2016년 충청북도 대소면의 두 검증지역에서 수행되었다. 각 검증지역에서 overall accuracy는 0.81, 0.71로 집계되었고, kappa coefficients는 0.75, 0.64로 산정되어 substantial 수준의 토지 피복 분류 정확도를 확인하였다. 특히, 개발된 모델은 필지 경계를 고려한 농업지역에서 overall accuracy 0.89, kappa coefficient 0.81로 almost perfect 수준의 우수한 분류 정확도를 보였다. 이에 개발된 토지 피복 분류모델은 특히 농업지역에서 현 토지 피복 분류 방법을 지원하여 토지 피복 지도의 빠르고 정확한 최신화에 기여할 수 있을 것으로 생각된다.The rapid update of land cover maps is necessary because spatial information of land cover is widely used in various areas. However, these maps have been released or updated in the interval of several years primarily owing to the manual digitizing method, which is time-consuming and labor-intensive. This study was aimed to develop a land cover classification model using the concept of a convolutional neural network (CNN) that classifies land cover labels from high-resolution remote sensing (HRRS) images and to increase the classification accuracy in agricultural areas using the parcel boundary extraction algorithm. The developed model comprises three modules, namely the pre-processing, land cover classification, and post-processing modules. The pre-processing module diversifies the perspective of the HRRS images by separating images with 75% overlaps to reduce the misclassification that can occur in a single image. The land cover classification module was designed based on the FusionNet model structure, and the optimal land cover type was assigned for each pixel of the separated HRRS images. The post-processing module determines the ultimate land cover types for each pixel unit by summing up the several-perspective classification results and aggregating the pixel-classification result for the parcel-boundary unit in agricultural areas. The developed model was trained with land cover maps and orthographic images (area: 547 km2) from the Jeonnam province in Korea. Model validation was conducted with two spatially and temporally different sites including Subuk-myeon of Jeonnam province in 2018 and Daseo-myeon of Chungbuk province in 2016. In the respective validation sites, the models overall accuracies were 0.81 and 0.71, and kappa coefficients were 0.75 and 0.64, implying substantial model performance. The model performance was particularly better when considering parcel boundaries in agricultural areas, exhibiting an overall accuracy of 0.89 and kappa coefficient 0.81 (almost perfect). It was concluded that the developed model may help perform rapid and accurate land cover updates especially for agricultural areas.Chapter 1. Introduction 1 1.1. Study background 1 1.2. Objective of thesis 4 Chapter 2. Literature review 6 2.1. Development of remote sensing technique 6 2.2. Land cover segmentation 9 2.3. Land boundary extraction 13 Chapter 3. Development of the land cover classification model 15 3.1. Conceptual structure of the land cover classification model 15 3.2. Pre-processing module 16 3.3. CNN based land cover classification module 17 3.4. Post processing module 22 3.4.1 Determination of land cover in a pixel unit 22 3.4.2 Aggregation of land cover to parcel boundary 24 Chapter 4. Verification of the land cover classification model 30 4.1. Study area and data acquisition 31 4.1.1. Training area 31 4.1.2. Verification area 32 4.1.3. Data acquisition 33 4.2. Training the land cover classification model 36 4.3. Verification method 37 4.3.1. The performance measurement methods of land cover classification model 37 4.3.2. Accuracy estimation methods of agricultural parcel boundary 39 4.3.3. Comparison of boundary based classification result with ERDAS Imagine 41 4.4. Verification of land cover classification model 42 4.4.1. Performance of land cover classification at the child subcategory 42 4.4.2. Classification accuracy of the aggregated land cover to main category 46 4.4.3. Classification accuracy of boundary based aggregation in agricultural area 57 Chapter 5. Conclusions 71 Reference 73 국 문 초 록 83Maste