Temporally Transferable Crop Mapping with Temporal Encoding and Deep Learning Augmentations

Abstract

Detailed maps on the spatial and temporal distribution of crops are key for a better understanding of agricultural practices and for food security management. Multi-temporal remote sensing data and deep learning (DL) have been extensively studied for deriving accurate crop maps. However, strategies to solve the problem of transferring crop classification models over time, e.g., training the model with recent year and mapping back to the past, have not been fully explored. This is due to the lack of a generalized method for aggregating optical data with regards the irregularity in annual clear sky observations and the scarcity of multi-annual crop reference data to support a more generalized DL model. In this study, we tackled these challenges by introducing a method namely Temporal Encoding (TE) to capture the irregular phenological information. Subsequently, we adapted and integrated two methods, i.e., Random Observations Selection (ROS) and Random Day Shifting (RDS) to simulate the variability of temporal sparsity as well as the shifts of crop phenology over different years. Combined with 1-dimensional Convolutional Neural Network (1D-CNN) classifier, our results showed that the model trained with crop reference data from 2018 with temporally dense data from Landsat 7/8 and Sentinel-2 A/B can be transferred with little decreases in accuracy to map 12 consecutive years from 2010 to 2021. Furthermore, the proposed models could achieve similar performances in the same years with and without fully available satellite information. Our approach appears well suited for improving temporal transferability to support long term historic crop mapping.

Keywords: Annual crop mapping, transferability, temporal encoding, data augmentations, 1D-CNN