Deriving Vegetation Indices For Phenology Analysis Using Genetic Programming

Plant phenology studies recurrent plant life cycle events and is a key component for understanding the impact of climate change. To increase accuracy of observations, new technologies have been applied for phenological observation, and one of the most successful strategies relies on the use of digital cameras, which are used as multi-channel imaging sensors to estimate color changes that are related to phenological events. We monitor leaf-changing patterns of a cerrado-savanna vegetation by taking daily digital images. We extract individual plant color information and correlate with leaf phenological changes. For that, several vegetation indices associated with plant species are exploited for both pattern analysis and knowledge extraction. In this paper, we present a novel approach for deriving appropriate vegetation indices from vegetation digital images. The proposed method is based on learning phenological patterns from plant species through a genetic programming framework. A comparative analysis of different vegetation indices is conducted and discussed. Experimental results show that our approach presents higher accuracy on characterizing plant species phenology.26P36169Ahrends, H., Etzold, S., Kutsch, W., Stoeckli, R., Bruegger, R., Jeanneret, F., Wanner, H., Eugster, W., Tree phenology and carbon dioxide fluxes: use of digital photography for process-based interpretation at the ecosystem scale (2009) Clim. Res., 39, pp. 261-274Alberton, B., Almeida, J., Henneken, R., da S.Torres, R., Menzel, A., Morellato, L.P.C., Using phenological cameras to track the green up in a cerrado savanna and its on-the-ground validation (2014) Ecol. Inform., 19, pp. 62-70Almeida, J., dos Santos, J.A., Alberton, B., da S.Torres, R., Morellato, L.P.C., Remote phenology: applying machine learning to detect phenological patterns in a cerrado savanna (2012) IEEE International Conference on eScience (eScience'12), pp. 1-8Almeida, J., dos Santos, J.A., Alberton, B., Morellato, L.P.C., da S.Torres, R., Plant species identification with phenological visual rhythms (2013) IEEE International Conference on eScience (eScience'13), pp. 148-154Almeida, J., dos Santos, J.A., Alberton, B., Morellato, L.P.C., da S.Torres, R., Visual rhythm-based time series analysis for phenology studies (2013) IEEE International Conference on Image Processing (ICIP'13), pp. 4412-4416Almeida, J., dos Santos, J.A., Alberton, B., da S.Torres, R., Morellato, L.P.C., Applying machine learning based on multiscale classifiers to detect remote phenology patterns in cerrado savanna trees (2014) Ecol. Inform., 23, pp. 49-61Andrade, F.S.P., Almeida, J., Pedrini, H., da S.Torres, R., Fusion of local and global descriptors for content-based image and video retrieval (2012) Iberoamerican Congress on, Pattern Recognition (CIARP'12), pp. 845-853Baeza-Yates, R.A., Ribeiro-Neto, B.A., (1999) Modern Information Retrieval, , Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USABimbo, A., (1999) Visual Information Retrieval, , Morgan Kaufmann Publishers Inc., San Francisco, CA, USACalumby, R.T., da S.Torres, R., Gonçalves, M.A., Multimodal retrieval with relevance feedback based on genetic programming (2014) Multimed. Tools Appl., 69 (3), pp. 991-1019Conti, J.C., Faria, F.A., Almeida, J., Alberton, B., Morellato, L.P.C., Camolesi, L., da S.Torres, R., Evaluation of time series distance functions in the task of detecting remote phenology patterns (2014) IEEE International Conference on Pattern Recognition (ICPR'14), pp. 1-6da S.Torres, R., Falcão, A.X., Gonçalves, M.A., Papa, J.P., Zhang, B., Fan, W., Fox, E.A., A genetic programming framework for content-based image retrieval (2009) Pattern Recogn., 42 (2), pp. 283-292Davis, J., Goadrich, M., The relationship between precision-recall and roc curves (2006) ACM International Conference on, Machine Learning (ICML'06), pp. 233-240Fan, W., Fox, E.A., Pathak, P., Wu, H., The effects of fitness functions on genetic programming-based ranking discovery forweb search (2004) J. Am. Soc. Inf. Sci. Technol., 55 (7), pp. 628-636Faria, F.A., Veloso, A., Almeida, H.M., Valle, E., da S.Torres, R., Gonçalves, M.A., Meira, W., Learning to rank for content-based image retrieval (2010) ACM International Conference on Multimedia Information Retrieval (MIR'10), pp. 285-294Ferreira, C.D., dos Santos, J.A., da S.Torres, R., Gonçalves, M.A., Rezende, R.C., Fan, W., Relevance feedback based on genetic programming for image retrieval (2011) Pattern Recogn. Lett., 32 (1), pp. 27-37Fishburn, P.C., (1988) Nonlinear Preference and Utility Theory, , Johns Hopkins University PressGillespie, A.R., Kahle, A.B., Walker, R.E., Color enhancement of highly correlated images. II. Channel ratio and "chromaticity" transformation techniques (1987) Remote Sens. Environ., 22 (3), pp. 343-365Gonzalez, R.C., Woods, R.E., (2007) Digital Image Processing, , Prentice-Hall Inc., Upper Saddle River, NJ, USAGuigues, L., Cocquerez, J., Le Men, H., Scale-sets image analysis (2006) Int. J. Comput. Vis., 68, pp. 289-317Henneken, R., Dose, V., Schleip, C., Menzel, A., Detecting plant seasonality from webcams using bayesian multiple change point analysis (2013) Agric. For. Meteorol., 168, pp. 177-185Ide, R., Oguma, H., Use of digital cameras for phenological observations (2010) Ecol. Inform., 5 (5), pp. 339-347Ide, R., Oguma, H., A cost-effective monitoring method using digital time-lapse cameras for detecting temporal and spatial variations of snowmelt and vegetation phenology in alpine ecosystems (2013) Ecol. Inform., 16, pp. 25-34Koza, J.R., (1992) Genetic Programming: on the Programming of Computers by Means of Natural Selection, , MIT PressKurc, S., Benton, L., Digital image-derived greenness links deep soil moisture to carbon uptake in a creosotebush-dominated shrubland (2010) J. Arid Environ., 74, pp. 585-594Migliavacca, M., Galvagno, M., Cremonese, E., Rossini, M., Meroni, M., Sonnentag, O., Cogliati, S., Richardson, A.D., Using digital repeat photography and eddy covariance data to model grassland phenology and photosynthetic co2 uptake (2011) Agric. For. Meteorol., 151 (10), pp. 1325-1337Morellato, L.P.C., Rodrigues, R.R., Leitão Filho, H.F., Joly, C.A., Estudo comparativo da fenologia de espécies arbóreas de floresta de altitude e floresta mesófila semidecídua na Serra do Iapí, Jundiaí, São Paulo (1989) Braz. J. Bot., 12, pp. 85-98Morellato, L.P.C., Camargo, M.G.G., Gressler, E., A review of plant phenology in south and central america (2013) Phenology: An Integrative Environmental Science, pp. 91-113. , Springer, (chapter 6), M.D. Schwartz (Ed.)Morisette, J.T., Richardson, A.D., Knapp, A.K., Fisher, J.I., Graham, E.A., Abatzoglou, J., Wilson, B.E., Liang, L., Tracking the rhythm of the seasons in the face of global change: phenological research in the 21st century (2009) Front. Ecol. Environ., 7 (5), pp. 253-260Muttil, N., Lee, J.H.W., Genetic programming for analysis and real-time prediction of coastal algal blooms (2005) Ecol. Model., 189 (3-4), pp. 363-376Nagai, S., Maeda, T., Gamo, M., Muraoka, H., Suzuki, R., Nasahara, K.N., Using digital camera images to detect canopy condition of deciduous broad-leaved trees (2011) Plant Ecol. Divers., 4, pp. 79-89Richardson, A.D., Jenkins, J.P., Braswell, B.H., Hollinger, D.Y., Ollinger, S.V., Smith, M.L., Use of digital webcam images to track spring greep-up in a deciduous broadleaf forest (2007) Oecologia, 152, pp. 323-334Richardson, A.D., Braswell, B.H., Hollinger, D.Y., Jenkins, J.P., Ollinger, S.V., Near-surface remote sensing of spatial and temporal variation in canopy phenology (2009) Ecol. Appl., 19, pp. 1417-1428Rocha, A., Almeida, J., Nascimento, M.A., da, S., Torres, R., Goldenstein, S., Efficient and flexible cluster-and-search approach for CBIR (2008) Advanced Concepts for Intelligent Vision Systems (ACIVS'08), pp. 77-88Schwartz, M.D., (2013) Phenology: An Integrative Environmental Science, , SpringerSonnentag, O., Hufkens, K., Teshera-Sterne, C., Young, A.M., Friedl, M., Braswell, B.H., Milliman, T.O., Richardson, A.D., Digital repeat photography for phenological research in forest ecosystems (2012) Agric. For. Meteorol., 152, pp. 159-177Woebbecke, D.M., Meyer, G.E., Von-Bargen, K., Mortensen, A.D., Color indices for weed identification under various soil, residue, and lighting conditions (1995) Trans. ASAE, 38 (1), pp. 259-269Zhao, J., Zhang, Y., Tan, Z., Song, Q., Liang, N., Yu, L., Zhao, J., Using digital cameras for comparative phenological monitoring in an evergreen broad-leaved forest and a seasonal rain forest (2012) Ecol. Inform., 10, pp. 65-72Zhou, L., He, H., Sun, X., Zhang, L., Yu, G., Ren, X.-L., Wang, J.-Y., Zhao, F.-H., Modeling winter wheat phenology and carbon dioxide fluxes at the ecosystem scale based on digital photography and eddy covariance data (2013) Ecol. Inform., 18, pp. 69-7

Image-based time series representations for satellite images classification

Orientador: Ricardo da Silva TorresTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: A classificação de imagens de sensoriamento remoto por pixel com base no perfil temporal desempenha um papel importante em várias aplicações, tais como: reconhecimento de culturas, estudos fenológicos e monitoramento de mudanças na cobertura do solo. Avanços sensores captura de imagem aumentaram a necessidade de criação de metodologias para analisar o perfil temporal das informações coletadas. Nós investigamos dados coletados em dois tipos de sensores: (i) sensores em plataformas orbitais, esse tipo de imagem sofre interferências de nuvens e fatores atmosféricos; e (ii) sensores fixados em campo, mais especificamente, uma câmera digital no alto de uma torre, cujas imagens capturadas podem conter dezenas de espécies, dificultando a identificação de padrões de interesse. Devido às particularidades dos dados detectados remotamente, torna-se custoso enviar a imagem capturada pelo sensor diretamente para métodos de aprendizado de máquina sem realizar um pré-processamento. Para algumas aplicações de sensoriamento remoto, comumente não se utiliza as imagens brutas oriundas dos sensores, mas os índices de vegetação extraídos das regiões de interesse ao longo do tempo. Assim, o perfil temporal pode ser caracterizado como uma série de observações dos índices de vegetação dos pixels de interesse. Métodos baseados em aprendizado profundo obtiveram bons resultados em aplicações de sensoriamento remoto relacionadas à classificação de imagens. Contudo, em consequência da natureza dos dados, nem sempre é possível realizar o treinamento adequado das redes de aprendizado profundo pela limitação causada por dados faltantes. Entretanto, podemos nos beneficiar de redes previamente treinadas para detecção de objetos para extrair características e padrões de imagens. O problema alvo deste trabalho é classificar séries temporais extraídas de imagens de sensoriamento remoto representando as características temporais como imagens 2D. Este trabalho investiga abordagens que codificam séries temporais como representação de imagem para propor metodologias de classificação binária e multiclasse no contexto de sensoriamento remoto, se beneficiando de redes extratoras de características profundas. Os experimentos conduzidos para classificação binária foram realizados em dados de satélite para identificar plantações de eucalipto. Os resultados superaram métodos baseline propostos recentemente. Os experimentos realizados para classificação multiclasse focaram em imagens capturadas com câmera digital para detectar o padrão fenológico de regiões de interesse. Os resultados mostram que a acurácia aumenta se consideramos conjuntos de pixelsAbstract: Pixelwise remote sensing image classification based on temporal profile plays an important role in several applications, such as crop recognition, phenological studies, and land cover change monitoring. Advances in image capture sensors have increased the need for methodologies to analyze the temporal profile of collected information. We investigate data collected by two types of sensors: (i) sensors on orbital platforms, this type of image suffers from cloud interference and atmospheric factors; and (ii) field-mounted sensors, in particular, a digital camera on top of a tower, where captured images may contain dozens of species, making it difficult to identify patterns of interest. Due to the particularities of remotely detected data, it is prohibitive to send sensor captured images directly to machine learning methods without preprocessing. In some remote sensing applications, it is not commonly used the raw images from the sensors, but the vegetation indices extracted from regions of interest over time. Thus, the temporal profile can be characterized as a series of observations of vegetative indices of pixels of interest. Deep learning methods have been successfully in remote sensing applications related to image classification. However, due to the nature of the data, it is not always possible to properly train deep learning networks because of the lack of enough labeled data. However, we can benefit from previously trained 2D object detection networks to extract features and patterns from images. The target problem of this work is to classify remote sensing images, based on pixel time series represented as 2D representations. This work investigates approaches that encode time series into image representations to propose binary and multiclass classification methodologies in the context of remote sensing, taking advantage of data-driven feature extractor approaches. The experiments conducted for binary classification were performed on satellite data to identify eucalyptus plantations. The results surpassed the ones of recently proposed baseline methods. The experiments performed for multiclass classification focused on detecting regions of interest within images captured by a digital camera. The results show that the accuracy increases if we consider a set of pixelsDoutoradoCiência da ComputaçãoDoutora em Ciência da ComputaçãoCAPE