Simultaneous Estimation of Soil Moisture and Soil Organic Matter from in Situ Dielectric - Part 2: Application of Optimal Estimation and Machine Learning Approaches

Abstract

This study examines two complementary strategies for deriving soil organic carbon (SOC) and soil moisture (SM) from existing in-situ microwave dielectric data, addressing the increasing demand for accurate real-time monitoring of both SM and SOC. One approach utilizes an optimal estimation (OE) algorithm based on a multiphase dielectric mixing model. This model effectively captures the nonlinear variations in dielectric properties across different moisture levels as influenced by soil organic matter. Using the Nelder–Mead simplex-based search, the OE method simultaneously refines SM and SOC estimates through a discontinuous model by adjusting the priors within the cost function. The second approach utilizes machine learning (ML) techniques, including eXtreme Gradient Boosting and Artificial Neural Networks. These models are trained using easily obtainable field measurements and publicly available soil maps without the need for difficult-to-obtain gravimetric soil moisture and laboratory-based soil organic carbon data for model training. While ML based methods outperform OE methods for both SM and SOC in internal validation with organic matter (OM) maps, external validation with direct field observations from SMAPVEX12 in Manitoba, Canada demonstrates the greater flexibility and performance of OE. By using these insights existing SM in-situ networks could deliver near‐real‐time measurements of both SM and OM without additional cost, improving resource management, enhancing sustainability, and fostering more precise soil carbon accounting in agricultural landscapes.

Keywords: carbon sequestration, carbon farming, smartfarm, soil moisture, soil organic matter, machine learning