Deep learning for agricultural land use classification from Sentinel-2

[ES] En el campo de la teledetección se ha producido recientemente un incremento del uso de técnicas de aprendizaje profundo (deep learning). Estos algoritmos se utilizan con éxito principalmente en la estimación de parámetros y en la clasificación de imágenes. Sin embargo, se han realizado pocos esfuerzos encaminados a su comprensión, lo que lleva a ejecutarlos como si fueran “cajas negras”. Este trabajo pretende evaluar el rendimiento y acercarnos al entendimiento de un algoritmo de aprendizaje profundo, basado en una red recurrente bidireccional de memoria corta a largo plazo (2-BiLSTM), a través de un ejemplo de clasificación de usos de suelo agrícola de la Comunidad Valenciana dentro del marco de trabajo de la política agraria común (PAC) a partir de series temporales de imágenes Sentinel-2. En concreto, se ha comparado con otros algoritmos como los árboles de decisión (DT), los k-vecinos más cercanos (k-NN), redes neuronales (NN), máquinas de soporte vectorial (SVM) y bosques aleatorios (RF) para evaluar su precisión. Se comprueba que su precisión (98,6% de acierto global) es superior a la del resto en todos los casos. Por otra parte, se ha indagado cómo actúa el clasificador en función del tiempo y de los predictores utilizados. Este análisis pone de manifiesto que, sobre el área de estudio, la información espectral y espacial derivada de las bandas del rojo e infrarrojo cercano, y las imágenes correspondientes a las fechas del período de verano, son la fuente de información más relevante utilizada por la red en la clasificación. Estos resultados abren la puerta a nuevos estudios en el ámbito de la explicabilidad de los resultados proporcionados por los algoritmos de aprendizaje profundo en aplicaciones de teledetección.[EN] The use of deep learning techniques for remote sensing applications has recently increased. These algorithms have proven to be successful in estimation of parameters and classification of images. However, little effort has been made to make them understandable, leading to their implementation as “black boxes”. This work aims to evaluate the performance and clarify the operation of a deep learning algorithm, based on a bi-directional recurrent network of long short-term memory (2-BiLSTM). The land use classification in the Valencian Community based on Sentinel-2 image time series in the framework of the common agricultural policy (CAP) is used as an example. It is verified that the accuracy of the deep learning techniques is superior (98.6 % overall success) to that other algorithms such as decision trees (DT), k-nearest neighbors (k-NN), neural networks (NN), support vector machines (SVM) and random forests (RF). The performance of the classifier has been studied as a function of time and of the predictors used. It is concluded that, in the study area, the most relevant information used by the network in the classification are the images corresponding to summer and the spectral and spatial information derived from the red and near infrared bands. These results open the door to new studies in the field of the explainable deep learning in remote sensing applications.Este trabajo ha sido subvencionado gracias al Convenio 2019 y 2020 de colaboración entre la Generalitat Valenciana, a través de la Conselleria d’Agricultura, Medi Ambient, Canvi Climàtic i Desenvolupament Rural, y la Universitat de València – Estudi General.Campos-Taberner, M.; García-Haro, F.; Martínez, B.; Gilabert, M. (2020). Deep learning para la clasificación de usos de suelo agrícola con Sentinel-2. Revista de Teledetección. 0(56):35-48. https://doi.org/10.4995/raet.2020.13337OJS3548056Baraldi, A., Parmiggiani, F. 1995. An investigation of the textural characteristics associated with gray level cooccurrence matrix statistical parameters. IEEE Transactions on Geoscience and Remote Sensing, 33(2), 293-304. https://doi.org/10.1109/36.377929Bengio, Y., Simard, P., Frasconi, P. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2), 157-166. https://doi.org/10.1109/72.279181Breiman, L., Friedman, J., Olshen, R.A., Stone, C.J. 1984. Classification and regression trees. Taylor & Francis: London, UK.Breiman, L. 2001. Random forests. Machine learning, 45(1), 5-32. https://doi.org/10.1023/A:1010933404324Campos-Taberner, M., Romero-Soriano, A., Gatta, C., Camps-Valls, G., Lagrange, A., Le Saux, B., Beaupère, A., Boulch, A., Chan-Hon-Tong, A., Herbin, S., Randrianarivo, H., Ferecatu, M., Shimoni, M., Moser, G., Tuia, D. 2016. Processing of extremely high-resolution Lidar and RGB data: outcome of the 2015 IEEE GRSS data fusion contest-part a: 2-D contest. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(12), 5547-5559. https://doi.org/10.1109/JSTARS.2016.2569162Campos-Taberner, M., García-Haro, F.J., Martínez, B., Sánchez-Ruíz, S., Gilabert, M.A. 2019a. A Copernicus Sentinel-1 and Sentinel-2 Classification Framework for the 2020+ European Common Agricultural Policy: A Case Study in València (Spain). Agronomy, 9(9), 556. https://doi.org/10.3390/agronomy9090556Campos-Taberner, M., García-Haro, F.J., Martínez, B., Sánchez-Ruiz, S., Gilabert, M.A. 2019b. Evaluación del potencial de Sentinel-2 para actualizar el SIGPAC de la Comunitat Valenciana. En: XVIII Congreso de la Asociación Española de Teledetección. Valladolid, España, 24-27, septiembre. pp 11-14.Camps-Valls, G., Tuia, D., Bruzzone, L., Benediktsson, J.A. 2013. Advances in hyperspectral image classification: Earth monitoring with statistical learning methods. IEEE Signal Processing Magazine, 31(1), 45-54. https://doi.org/10.1109/MSP.2013.2279179Chuvieco, E. 2008. Teledetección Ambiental. La observación de la Tierra desde el espacio. Madrid: Ariel.Cover, T., Hart, P. 1967. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21-27. https://doi.org/10.1109/TIT.1967.1053964Estrada, J., Sánchez, H., Hernanz, L., Checa, M.J., Roman, D. 2017. Enabling the Use of Sentinel-2 and LiDAR Data for Common Agriculture Policy Funds Assignment. ISPRS International Journal of Geo-Information, 6(8), 255. https://doi.org/10.3390/ijgi6080255Friedl, M.A., McIver, D.K., Hodges, J.C., Zhang, X.Y., Muchoney, D., Strahler, A.H., Baccini, A. 2002. Global land cover mapping from MODIS: algorithms and early results. Remote sensing of Environment, 83(1-2), 287-302. https://doi.org/10.1016/S0034-4257(02)00078-0Graves, A., Mohamed, A.R., Hinton, G. 2013. Speech recognition with deep recurrent neural networks. En 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6645-6649. https://doi.org/10.1109/ICASSP.2013.6638947Gilabert, M.A., González-Piqueras, J., García-Haro, J., 1997. Acerca de los índices de vegetación. Revista de teledetección, 8, 1-10. Disponible en http://www.aet.org.es/?q=revista8-4González-Guerrero, O., y Pons, X., 2020. The 2017 Land Use/Land Cover Map of Catalonia based on Sentinel-2 images and auxiliary data. Revista de Teledetección, 55, 81-92. https://doi.org/10.4995/raet.2020.13112Gregrio, A., Jansen, J. 2000. Land cover classification system (LCCS); Classification concepts and user manual for software version 2. Roma: FAO.Griffiths, P., Nendel, C., Hostert, P. 2019. Intra-annual reflectance composites from Sentinel-2 and Landsat for national-scale crop and land cover mapping. Remote Sensing of Environment, 220, 135-151. https://doi.org/10.1016/j.rse.2018.10.031Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., Yang, G.Z. 2019. XAI-Explainable artificial intelligence. Science Robotics, 4(37). https://doi.org/10.1126/scirobotics.aay7120Haralick, R.M., Shanmugam, K., Dinstein, I.H. 1973. Textural features for image classification. IEEE Transactions on systems, man, and cybernetics, 3(6), 610-621. https://doi.org/10.1109/TSMC.1973.4309314Haykin, S. 1994. Neural networks: a comprehensive foundation. River: Prentice Hall.Immitzer, M., Vuolo, F., Atzberger, C. 2016. First experience with Sentinel-2 data for crop and tree species classifications in central Europe. Remote Sensing, 8(3), 166. https://doi.org/10.3390/rs8030166Hochreiter, S., Schmidhuber, J. 1997. Long short-term memory. Neural computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735Kussul, N., Lavreniuk, M., Skakun, S., Shelestov, A. 2017. Deep learning classification of land cover and crop types using remote sensing data. IEEE Geoscience and Remote Sensing Letters, 14(5), 778-782. https://doi.org/10.1109/LGRS.2017.2681128Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., Johnson, B.A. 2019. Deep learning in remote sensing applications: A meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, 152, 166-177. https://doi.org/10.1016/j.isprsjprs.2019.04.015Maggiori, E., Tarabalka, Y., Charpiat, G., Alliez, P. 2016. Convolutional neural networks for large-scale remotesensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(2), 645-657. https://doi.org/10.1109/TGRS.2016.2612821Mikolov, T., Kombrink, S., Burget, L., Černocký, J., Khudanpur, S. 2011. Extensions of recurrent neural network language model. En 2011 IEEE International Conference on acoustics, speech and signal processing (ICASSP). Praga, República Checa, 22-27 Mayo. pp. 5528-5531. https://doi.org/10.1109/ICASSP.2011.5947611Montavon, G., Samek, W., Müller, K.R. 2018. Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73, 1-15. https://doi.org/10.1016/j.dsp.2017.10.011Mou, L., Ghamisi, P., Zhu, X.X. 2017. Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7), 3639-3655. https://doi.org/10.1109/TGRS.2016.2636241Mountrakis, G., Im, J., Ogole, C. 2011. Support vector machines in remote sensing: A review. ISPRS Journal of Photogrammetry and Remote Sensing, 66(3), 247- 259. https://doi.org/10.1016/j.isprsjprs.2010.11.001Ndikumana, E., Ho Tong Minh, D., Baghdadi, N., Courault, D., Hossard, L. 2018. Deep recurrent neural network for agricultural classification using multitemporal SAR Sentinel-1 for Camargue, France. Remote Sensing, 10(8), 1217. https://doi.org/10.3390/rs10081217Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N. 2019. Deep learning and process understanding for data-driven Earth system science. Nature, 566(7743), 195-204. https://doi.org/10.1038/s41586-019-0912-1Ruiz, L.A., Almonacid-Caballer, J., Crespo-Peremarch, P., Recio, J.A., Pardo-Pascual, J.E., SánchezGarcía, E. 2020. Automated classification of crop types and condition in a mediterranean area using a fine-tuned convolutional neural network. En The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences. Niza, Francia, 31 Agosto - 2 Septiembre (en línea). pp. 1061-1068. https://doi.org/10.5194/isprs-archivesXLIII-B3-2020-1061-2020Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., Muller, K.R. (Eds.). 2020. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Cham: Springer Nature. https://doi.org/10.1007/978-3-030-28954-6Schmedtmann, J., Campagnolo, M.L. 2015. Reliable crop identification with satellite imagery in the context of common agriculture policy subsidy control. Remote Sensing, 7(7), 9325-9346. https://doi.org/10.3390/rs70709325Schuster, M., Paliwal, K.K. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673-2681. https://doi.org/10.1109/78.650093Sitokonstantinou, V., Papoutsis, I., Kontoes, C., Lafarga Arnal, A., Armesto Andrés, A.P., Garraza Zurbano, J.A. 2018. Scalable parcel-based crop identification scheme using sentinel-2 data time-series for the monitoring of the common agricultural policy. Remote Sensing, 10(6), 911. https://doi.org/10.3390/rs10060911Story, M., Congalton, R.G. 1986. Accuracy assessment: a user's perspective. Photogrammetric Engineering and Remote Sensing, 52(3), 397-399.Tapsall, B., Milenov, P., Tasdemir, K. 2010. Analysis of RapidEye imagery for annual landcover mapping as an aid to European Union (EU) common agricultural polic. En ISPRS Technical Commission VII Symposium - 100 Years ISPRS. Viena, Austria, 5-7 Julio. pp. 568-573.Van Den Oord, A., Kalchbrenner, N., Kavukcuoglu, K. 2016. Pixel recurrent neural networks. En Proceedings of the 33rd International Conference on International Conference on Machine Learning. New York, EEUU., 20-22 Junio. pp. 1747-1756.Vuolo, F., Neuwirth, M., Immitzer, M., Atzberger, C., Ng, W.T. 2018. How much does multi-temporal Sentinel-2 data improve crop type classification? International Journal of Applied Earth Observation and Geoinformation, 72, 122-130. https://doi.org/10.1016/j.jag.2018.06.007Wardlow, B.D., Egbert, S.L., Kastens, J.H. 2007. Analysis of time-series MODIS 250 m vegetation index data for crop classification in the US Central Great Plains. Remote Sensing of Environment, 108(3), 290-310. https://doi.org/10.1016/j.rse.2006.11.021Watson, R.T., Noble, I.R., Bolin, B., Ravindranath, N.H., Verardo, D.J., Dokken, D.J. 2000. Land use, land-use change and forestry: a special report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press.Zhan, X., Defries, R., Townshend, J.R.G., Dimiceli, C., Hansen, M., Huang, C., Sohlberg, R. 2000. The 250 m global land cover change product from the Moderate Resolution Imaging Spectroradiometer of NASA's Earth Observing System. International Journal of Remote Sensing, 21(6-7), 1433-1460. https://doi.org/10.1080/014311600210254Zhang, L., Zhang, L., Du, B. 2016. Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2), 22-40. https://doi.org/10.1109/MGRS.2016.2540798Zhong, L., Hu, L., Zhou, H. 2019. Deep learning based multi-temporal crop classification. Remote sensing of environment, 221, 430-443. https://doi.org/10.1016/j.rse.2018.11.032Zhu, Z., Woodcock, C.E. 2014. Continuous change detection and classification of land cover using all available Landsat data. Remote Sensing of Environment, 144, 152-171. https://doi.org/10.1016/j.rse.2014.01.011Zhu, X.X., Tuia, D., Mou, L., Xia, G.S., Zhang, L., Xu, F., Fraundorfer, F. 2017. Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4), 8-36. https://doi.org/10.1109/MGRS.2017.276230

Análise do potencial de dados Sentinel-2 na classificação da ocupação do solo no controlo de subsídios agrícolas

Mestrado em Engenharia Agronómica - Instituto Superior de AgronomiaOs agricultores submetem anualmente as suas candidaturas aos subsídios agrícolas no âmbito da Política Agrícola Comum (PAC). Cabe às administrações dos Estados-Membros estabelecer um sistema de controlo fiável e eficaz para garantir a validade das declarações dos agricultores e o cumprimento dos critérios de eligibilidade. O controlo por deteção remota (CwRS) tem demonstrado ser um método importante no apoio aos controlos de superfície (OTSC) através da fotointerpretação assistida por computador (CAPI) para a identificação de culturas. O principal objetivo deste estudo foi analisar o potencial das imagens Sentinel-2A (S2A) para a identificação das culturas, quanto ao tipo e à estação, enquanto combinadas com dados Landsat-7 (L7) e Landsat-8 (L8) aplicando algoritmo de classificação Random Forest (RF). Para acompanhar a fenologia das culturas em diferentes fases, as imagens L7, L8 e S2A foram adquiridas entre outubro de 2015 e agosto de 2016 sobre Beja (Portugal). Foram testados dois cenários diferentes pelo algoritmo RF para cada caso de identificação: usar apenas bandas espectrais ou usar as bandas mais os índices de vegetação (IV) - normalized difference vegetation index (NDVI), green normalized difference vegetation index (GNDVI), enhanced vegetation index (EVI), soil adjusted vegetation index (SAVI) e modified soil adjusted vegetation index (MSAVI). O algoritmo RF foi calibrado e validado utilizando dados de controlo validados e fornecidos pelo Instituto de Financiamento da Agricultura e Pescas (IFAP). As classificações foram executadas usando uma abordagem assistida baseada em pixels e em objetos e as precisões foram obtidas a partir da validação cruzada 10-fold. A avaliação da precisão baseou-se na comparação entre dados da classificação e de referência. Os resultados mostraram que a melhor precisão global de 99.7% foi obtida na distinção entre estações quando se utiliza uma abordagem ao nível do pixel e os IV. Para a classificação de culturas a maior precisão global de 98.22% foi obtida quando se usa a mesma abordagem e a combinação entre bandas espectrais e IV. Na classificação das parcelas, obteve-se uma precisão global de 92.8%, sendo a precisão das culturas sempre maior ou igual a 79.2%, sendo as culturas de cereais que apresentam a maior confusão na classificaçãoN/

Enabling the Use of Sentinel-2 and LiDAR Data for Common Agriculture Policy Funds Assignment

A comprehensive strategy combining remote sensing and field data can be helpful for more effective agriculture management. Satellite data are suitable for monitoring large areas over time, while LiDAR provides specific and accurate data on height and relief. Both types of data can be used for calibration and validation purposes, avoiding field visits and saving useful resources. In this paper, we propose a process for objective and automated identification of agricultural parcel features based on processing and combining Sentinel-2 data (to sense different types of irrigation patterns) and LiDAR data (to detect landscape elements). The proposed process was validated in several use cases in Spain, yielding high accuracy rates in the identification of irrigated areas and landscape elements. An important application example of the work reported in this paper is the European Union (EU) Common Agriculture Policy (CAP) funds assignment service, which would significantly benefit from a more objective and automated process for the identification of irrigated areas and landscape elements, thereby enabling the possibility for the EU to save significant amounts of money yearly.publishedVersio