Semantic Segmentation of Mining Areas in Satellite Images: Towards a Global Approach

Abstract

The land footprint of global mineral extraction is estimated to exceed 100,000 square kilometers, causing significant environmental and societal impacts in affected areas. To date, assessing the extent of mining areas relies on visual interpretation of satellite imagery, a costly and inefficient process. Recent advancements in pre-trained foundation models provide a promising pathway to enable global-scale applications to monitor mining sites. This study evaluates the potential of machine learning models for semantic segmentation of mining areas in satellite images, benchmarking their performance to task-specific models. Three approaches are examined: (a) combining the Visual Foundation Models (VFMs) Grounding DINO and SAM, (b) fine-tuning the Earth Observation Foundation Model (EOFM) Clay, and (c) training custom U-Net models. Results indicate that custom-trained U-Net models, achieving an Intersection over Union (IoU) of 0.53 on the test set, outperform both VFMs and EOFMs, reaching IoU scores of 0.34 and 0.35, respectively. Transfer learning, power loss functions, and the near-infrared channel improve segmentation accuracy for U-Net models. Performance varies regionally and depends on mine characteristics, e.g., large-scale industrial and small-scale artisanal mines are segmented well. In contrast, small-scale mines near urban areas are more challenging due to the similar spectral response of mining infrastructure and urbanization. These insights contribute toward the development of fully automated approaches to monitor the expansion of mining globally and provide timely spatial information to assess the impacts of mining.

Keywords: Computer Vision, semantic segmentation, mining area monitoring, Land Use, foundation models, sentinel-2