Attention spectrale et explicabilité pour la classification de séries temporelles satellitaires par réseaux de neurones profonds

International audienceDeep Neural Networks (DNNs) are getting increasing attention to deal with Land Cover Classification (LCC) relying on Satellite Image Time Series (SITS). Though high performances can be achieved, the rationale of a prediction yielded by a DNN often remains unclear. An architecture expressing predictions with respect to input channels is thus proposed in this paper. It relies on convolutional layers and an attention mechanism weighting the importance of each channel in the final classification decision. The correlation between channels is taken into account to set up shared kernels and lower model complexity. Experiments based on a Sentinel-2 SITS show promising results.Les réseaux profonds de neurones sont de plus en plus plébiscités, à fortiori, en classification d'occupation des sols, à partir de séries temporelles d'images satellites. Malgré leurs performances, ces réseaux souffrent de manque d'explicabilité. Dans ce cadre, nous étudions différentes architectures de réseaux profonds convolutionnels qui extraient les caractéristiques spectro-temporelles des séries pour ensuite classifier chaque pixel. Afin d'expliquer ses décisions, nous y intégrons, de deux manières différentes, un module d'attention donnant la contribution de chaque caractéristique spectrale. Des expériences ont été menées sur des données Sentinel-2 de la Réunion. Les corrélations spectrales, inhérentes à ce type de séries, sont prises en compte en factorisant les convolutions sur des bandes fortement corrélées afin d'obtenir le meilleur compromis complexité/performance du modèle. Les résultats semblant prometteurs en termes d'explicabilité et de performances, des développements sont à l'étude pour étendre les attentions au niveau spectral et temporel (cartes de chaleur) et pour comprendre plus finement le fonctionnement des réseaux "multi-sorties" qui permettent d'entraîner le module d'attention

Explaining a deep spatiotemporal land cover classifier with attention and redescription mining

International audienceDeep learning-based land cover classifiers learnt from Satellite Image Time Series (SITS) are known to reach high performances. In order to explain, at least partly, the rationale leading to each one of their decisions, attention-based architectures have been proposed to automatically weight the importance of predefined data components in the classification process. Though generated for each decision separately, the informational content conveyed by such explanations can remain insufficient to end-users because of the complex nature of SITS. Moreover, getting a general perspective about the way a classifier works requires merging all explanations for each class and relating them to its mode of operation, which is not always straightforward. A preliminary and complementary approach for automatically identifying the data features detected by a pixel-wise deep spatiotemporal land cover classifier and explaining its behavior at the class level is therefore proposed in this paper. Classified pixels are first described using interpretable features coming under the form of data mining patterns. A redescription mining technique is then employed to automatically select, for each class, the features matching the different activation level configurations of the layer that is assumed to capture the aforementioned patterns. Experiments based on a Sentinel-2 time series and a deep spatiotemporal neural network implementing a channel-separated processing as well as a channel-based attention mechanism show the interest of such a combined approach

Unsupervised Spatiotemporal Mining of Satellite Image Time Series Using Grouped Frequent Sequential Patterns

International audienceAn important aspect of satellite image time series is the simultaneous access to spatial and temporal information. Various tools allow end users to interpret these data without having to browse the whole data set. In this paper, we intend to extract, in an unsupervised way, temporal evolutions at the pixel level and select those covering at least a minimum surface and having a high connectivity measure. To manage the huge amount of data and the large number of potential temporal evolutions, a new approach based on data-mining techniques is presented. We have developed a frequent sequential pattern extraction method adapted to that spatiotemporal context. A successful application to crop monitoring involving optical data is described. Another application to crustal deformation monitoring using synthetic aperture radar images gives an indication about the generic nature of the proposed approach

Unsupervised Spatiotemporal Mining of Satellite Image Time Series Using Grouped Frequent Sequential Patterns

International audienceAn important aspect of satellite image time series is the simultaneous access to spatial and temporal information. Various tools allow end users to interpret these data without having to browse the whole data set. In this paper, we intend to extract, in an unsupervised way, temporal evolutions at the pixel level and select those covering at least a minimum surface and having a high connectivity measure. To manage the huge amount of data and the large number of potential temporal evolutions, a new approach based on data-mining techniques is presented. We have developed a frequent sequential pattern extraction method adapted to that spatiotemporal context. A successful application to crop monitoring involving optical data is described. Another application to crustal deformation monitoring using synthetic aperture radar images gives an indication about the generic nature of the proposed approach