Enhancing Soil Organic Matter Estimation Using Advanced Machine Learning Algorithms: A Comparative Study of PolSAR and IoT Sensor Data

Abstract

Monitoring soil organic matter (SOM) is a cornerstone of sustainable agriculture and food authenticity verification, yet traditional laboratory-based assessments are too resource-intensive for regional-scale deployment. This study presents a rigorous comparative evaluation of two sensing modalities: a terrestrial IoT sensor network and polarimetric synthetic aperture radar (PolSAR). To address the lack of laboratory-certified labels, we engineered the Soil Fertility Index (SFI) and Component-based Soil Quality Index (CSQI) as continuous biophysical proxies.

Our methodology employs a dual-phase machine learning strategy: first, unsupervised latent discovery was used to identify dominant geophysical regimes within the radar backscatter. Second, an ensemble of supervised regression algorithms, including XGBoost and LightGBM, was deployed using a 10-fold cross-validation protocol to estimate soil integrity across the study site. The Geophysical Handshake novelty, which mathematically harmonizes ground-level dielectric measurements with satellite-based polarimetric scattering signatures, is central to our framework.

To ensure the resilience of our findings, we implemented a strict identity-feature audit to eliminate data leakage. Results reveal a significant performance disparity: while the IoT track provided stable local insights, the PolSAR-based models demonstrated superior predictive power, achieving a Blind-Test 𝑅2 of 0.9908 and an RMSE of 0.33, with a negligible cross-validation standard deviation of 0.0003. This study proves that the integration of continuous proxy engineering and supervised radar modeling provides the synoptic, vegetation-penetrative X-ray required for resilient and scalable SOM estimation across non-instrumented landscapes.

Keywords

• Oil Organic Matter (SoM)
• Soil Organic Carbon (SoC)
• Polarimetric SAR (PolSAR)
• IoT Sensors
• Machine Learning
• Data Leakage
• Geophysical Handshake
• Random Forest
• XGBoost
• Gaussian Mixture Model (GMM)
• LightGBM
• Agri-food Authenticity
• Remote Sensing
• Organic Farming

References

1. Ahmed, N., & Shakoor, N. (2025). Advancing agriculture through IoT, Big Data, and AI: A review of smart technologies enabling sustainability. In Smart Agricultural Technology (Vol. 10). https://doi.org/10.1016/j.atech.2025.100848 google scholar
2. Anceaux, D., Katsifodimos, A., & Ionescu, A. (2023). A comparative study for using PCA, LDA, GDA, and Lasso for dimensionality reduction before classification algorithms. http://repository.tudelft.nl/. google scholar
3. Bahmani, B., Moseley, B., Vattani, A., Kumar, R., & Vassilvitskii, S. (2012). Scalable K-Means++. http://arxiv.org/abs/1203.6402 google scholar
4. Bakian-Dogaheh, K., Chen, R. H., Yi, Y., Kimball, J. S., Moghaddam, M., & Tabatabaeenejad, A. (2022). A model to characterize soil moisture and organic matter profiles in the permafrost active layer in support of radar remote sensing in Alaskan Arctic tundra. Environmental Research Letters, 17(2). https://doi.org/10.1088/1748-9326/ac4e37 google scholar
5. Balamurugan, S., Ayyasamy, A., & Joseph, K. S. (2022). IoT-blockchain driven traceability techniques for improved safety measures in food supply chain. International Journal of Information Technology (Singapore), 14(2), 1087–1098. https://doi.org/10.1007/s41870-020-00581-yhttps://doi.org/10.1007/s41870-020-00581-y google scholar
6. Ben Abbes A, Magagi, R., & Goïta, K. (2019). Estimation of Soil Moisture from SMAP Observations Using Long Short-Term Memory IEEE International Geoscience & Remote Sensing Symposium. google scholar
7. Bouzembrak, Y., Klüche, M., Gavai, A., & Marvin, H. J. P. (2019). Internet of things in food safety: Literature review and a bibliometric analysis. In Trends in Food Science and Technology, 94, 54–64. https://doi.org/10.1016/j.tifs.2019.11.002https://doi.org/10.1016/j.tifs.2019.11.002 google scholar
8. Çelik, M. F., & Erten, E. (2022). Biophysical parameter estimation of crops from polarimetric synthetic aperture radar imagery with data-driven polynomial chaos expansion and global sensitivity analysis. Computers and Electronics in Agriculture, 194. https://doi.org/10.1016/j.compag.2022.106781 google scholar

9. Crop mapping using fused optical-radar data set. (2020). https://github.com/MaziarISTUNI/Soil-SOM-PolSAR-Winnipeg google scholar
10. Dabboor, M., Atteia, G., & Alnashwan, R. (2023). Optimizing soil moisture retrieval: using compact polarimetric features with advanced machine learning techniques. Land, 12(10). https://doi.org/10.3390/land12101861 google scholar
11. Datta, R., Baitalik, S., Das Choudhury, S., Chattopadhyay, A. K., & Das, A. K. (2025). Fuzzy Hypergraph Feature Association Map for High-Dimensional Feature Selection in Agriculture and Remote Sensing. Agricultural and Remote Sensing Research, 62, 69–79. google scholar
12. Deenadayalan, T., & Deepakraj, E. (2023). IoT-enabled ethanol detection system: A Transformative Stride in Ensuring Food Authenticity. 2023 International Conference on System, Computation, Automation and Networking, ICSCAN 2023. https://doi.org/10.1109/ICSCAN58655.2023.10395191 google scholar
13. Deodoro, S. C., de Andrade Moral, R., Fealy, R., McCarthy, T., & Fealy, R. (2024). Using the surface scattering mechanism from dual-pol SAR data to estimate topsoil particle-sizefractions. International Journal of Applied Earth Observation and Geoinformation, 128. https://doi.org/10.1016/j.jag.2024.103742https://doi.org/10.1016/j.jag.2024.103742 google scholar
14. Dinesh, D., Kumar, S., & Saran, S. (2024). Machine learning modeling for soil moisture retrieval from simulated NASA-ISRO SAR (NISAR) L-band data. Remote Sensing, 16(18), 3539. https://doi.org/10.3390/rs16183539 google scholar
15. Dinesh Kumar, S., Rao, S., & Sharma, J. R. (2013). Radar Vegetation Index as an Alternative to NDVI for Monitoring of Soyabean and Cotton. In Indian Cartographer: XXXIII. http://earthexplorer.usgs.gov/ google scholar
16. Ding, Z., Liu, K., Grunwald, S., Smith, P., Ciais, P., Wang, B., Wadoux, A. M. J. C., Ferreira, C., Karunaratne, S., Shurpali, N., Yin, X., Roberts, D., Madgett, O., Duncan, S., Zhou, M., Liu, Z., & Harrison, M. T. (2025). Advancing soil organic carbon prediction: A comprehensive review of technologies, AI, process-based and hybrid modeling approaches. In Advanced Science. John Wiley and Sons Inc. https://doi.org/10.1002/advs.202504152 google scholar
17. Dorigo, W., Himmelbauer, I., Aberer, D., Schremmer, L., Petrakovic, I., Zappa, L., Preimesberger, W., Xaver, A., Annor, F., Ardö, J., Baldocchi, D., Bitelli, M., Blöschl, G., Bogena, H., Brocca, L., Calvet, J. C., Camarero, J. J., Capello, G., Choi, M., … Sabia, R. (2021). The International Soil Moisture Network: Serving Earth system science for over a decade. In Hydrology and Earth System Sciences (Vol. 25, No. 11, pp. 5749–5804). Copernicus Publications. https://doi.org/10.5194/hess-25-5749-2021 google scholar
18. Du, S., Zou, Y., Wang, Z., Li, X., Li, Y., Shang, C., & Shen, Q. (2026). Unsupervised hyperspectral image super-resolution via self-supervised modality decoupling. International Journal of Computer Vision, 134(4), 152. https://doi.org/10.1007/s11263-026-02757-8https://doi.org/10.1007/s11263-026-02757-8 google scholar
19. El-Jamaoui, I., Delgado-Iniesta, M. J., Martínez Sánchez, M. J., Pérez Sirvent, C., & Martínez López, S. (2025). Assessing soil organic carbon in semi-arid agricultural soils using UAVs and machine learning: A pathway to sustainable water and soil resource management. Sustainability (Switzerland), 17(8). https://doi.org/10.3390/su17083440 google scholar
20. Hinz, M., Meslem, D., & Bracke, S. (2022). The application of the Gaussian mixture model algorithm for the unsupervised analysis of surface topographies. In Probabilistic Safety Assessment and Management PSAM (Vol. 16). google scholar
21. Hong, Y., Xie, T., Luo, L., Wang, M., Li, D., Zhang, Q., & Xu, T. (2024). Area extraction and growth monitoring of sugarcane from multi-source remote sensing images under a polarimetric SAR data compensation based on buildings. Geo-Spatial Information Science. https://doi.org/10.1080/10095020.2024.2381607 google scholar
22. Izuchukwu, A., Toufik Osman, M., & Mukadas Musah, M. (2024). Organic farming as a mitigation strategy against climate change: A critical review. IIARD International Journal of Geography & Environmental Management (IJGEM), 10(9). doi:10.1016/j.ijgem.2024.06.009. google scholar
23. Jafarzadeh, H., Verma, A., Mahdianpari, M., Bhattacharya, A., & Homayouni, S. (2024). Enhanced crop discrimination and monitoring using compact-polarimetric SAR signature analysis from RADARSAT constellation mission. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17, 6308–6327. https://doi.org/10.1109/JSTARS.2024.3366883 google scholar
24. Jain, A., Nandakumar, K., & Ross, A. (2005). Score normalization in multimodal biometric systems. Pattern Recognition, 38(12), 2270–2285. https://doi.org/10.1016/j.patcog.2005.01.012 google scholar
25. Kapoor, S., & Narayanan, A. (2022). Leakage and the reproducibility crisis in ML-based science. Available at: http://arxiv.org/abs/2207.07048 google scholar
26. Karray, E., & Bouricha, B. (2024). Discriminant analysis using feature extraction from spectral domain responses to achieve accurate delineation for robust evaluation or classification of soil properties. International Journal of Remote Sensing. https://doi.org/10.1080/01431161.2024.2413544 google scholar
27. Kaya, O., Ates, F., Daler, S., Keskin, N., & Turan, M. (2024). Integrating organic fertilization and soil tillage techniques to enhance the nutritional attributes of “Royal” grapes (Vitis vinifera L. cv.). Food and Energy Security, 13(4). https://doi.org/10.1002/fes3.562https://doi.org/10.1002/fes3.562 google scholar
28. Khurram, S., Pour, A. B., Bagheri, M., Ariffin, E. H., Akhir, M. F., & Hamzah, S. B. (2025). Developments in deep learning algorithms for coastline extraction from remote sensing imagery: a systematic review. In Earth Science Informatics, 18(3), Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/s12145-025-01805-0 google scholar
29. Khusheef, A. S. (2025). Optimized multimodal anomaly detection in fused deposition modeling: real-time monitoring with clustering classifiers and data fusion. Prog. Add. Manuf., 10(11), 9681–9696. https://doi.org/10.1007/s40964-025-01199-x google scholar
30. Kim, Y., Jackson, T., Bindlish, R., Lee, H., & Hong, S. (2012). Radar vegetation index for estimating the vegetation water content of rice and soybean. IEEE Geosci Remote Sens Lett, 9(4), 564–568. https://doi.org/10.1109/LGRS.2011.2174772https://doi.org/10.1109/LGRS.2011.2174772 google scholar
31. Klosin, S., & Vives-i-Bastida, J. (2024). Bagged Polynomial Regression and Neural Networks. (2205.08609v2). http://arxiv.org/abs/2205.08609 google scholar
32. Klosin S, Vives-i-Bastida J. 2024b. Bagged polynomial regression and neural networks http://arxiv.org/abs/2205.08609 google scholar
33. Kubota, H., Hernandez-Ramirez, G., & Zhao, T. (2024). Contrasting soil organic carbon concentrations and mass storage between conventional farming and organic farming: A meta-analysis. Sustainability (Switzerland), 16(24). https://doi.org/10.3390/su162411260 google scholar
34. Kumar V, Naresh R, Siddhartha R, Harshit, S., & Priya, B. (2025). Artificial intelligence for enhancing soil organic matter, nutrient cycling and water productivity: A comprehensive review of soilhealthled intensification approaches. Int J Agric Food Sci. 2025;7(12):114–128. google scholar
35. Li, Q., Han, X., Liu, Z., He, P., Shi, P., Chen, Q., & Du, F. (2022). A novel information changing rate and conditional mutual information-based input feature selection method for artificial intelligence drought prediction models. Climate Dynamics, 58(11–12), 3405–3425. https://doi.org/10.1007/s00382-021-06104-0 google scholar
36. Li, X., Gao, X., Wang, Q., Wang, C., Li, B., & Wan, K. (2024). Feature analysis network: An interpretable idea in deep learning. Cognitive Computation, 16(3), 803–826. https://doi.org/10.1007/s12559-023-10238-0https://doi.org/10.1007/s12559-023-10238-0 google scholar
37. Li, X., Gao, X., Wang, Q., Wang, C., Li, B., & Wan, K. (2024b). Feature analysis network: An interpretable idea in deep learning. Cognitive Computation, 16(3), 803–826. https://doi.org/10.1007/s12559-023-10238-0https://doi.org/10.1007/s12559-023-10238-0 google scholar
38. Mahesh, A., Sahoo, S. K., J, K. D., Hemavathi, E., Shanthi, S., & Murugan, S. (2025). Accurate soil organic carbon prediction with convolutional neural networks and remote sensing data. Proceedings of the 2025 International Conference on Computing and Communications (COMPUTINGCON), 1–6. https://doi.org/10.1109/COMPUTINGCON64838.2025.11377777 google scholar
39. Mandić-Rajčević, S., & Colosio, C. (2019). Methods for the identification of outliers and their influence on exposure assessment in agricultural pesticide applicators: A proposed approach and validation using biological monitoring. Toxics, 7(3). https://doi.org/10.3390/toxics7030037https://doi.org/10.3390/toxics7030037 google scholar
40. McGuirk, S. L., & Cairns, I. H. (2025). Soil carbon remote sensing: A meta-analysis and systematic review of published results from 1969–2022. In Geotechnics (Vol. 5, Number 2). Multidisciplinary Digital Publishing Institute (MDPI). https://doi.org/10.3390/geotechnics5020033 google scholar
41. Mondal, M., Khayati, M., Sandlin, H. Â., & Cudré-Mauroux, P. (2025). A survey of multimodal event detection based on data fusion. VLDB Journal, 34(1). https://doi.org/10.1007/s00778-024-00878-5https://doi.org/10.1007/s00778-024-00878-5 google scholar
42. Moore, K. R., Lychuk, T., Mcnairn, H., Geng, X., Chipanshi, A., Ojo, E. R., Lapen, D., Powers, J., Halstead, A., Gottfried, K., Tetlock, E., Bence, B., Champagne, C., Pacheco, A., Glenn, A., Crittenden, S., Rollin, P., Fitzmaurice, J., Bullock, P., … Kopeechuk, A. (2025). Collection: Datasets From Real-Time In-Situ Soil Monitoring for Agriculture 2025. IEEE Data Descriptions, 2, 433–440. https://doi.org/10.1109/IEEEDATA.2025.3612373 google scholar
43. More, A., Panchal, N., Mouria, A., & Bathe, K. (2019). Soil analysis using IoT. https://ssrn.com/abstract=3366756. 2nd International Conference on Advances in Science & Technology (ICAST) 2019. google scholar
44. Murindangabo, Y. T., Kopecký, M., Perná, K., Nguyen, T. G., Ghorbani, M., Konvalina, P., Bohatá, A., Kavková, M., Hoang, T. N., Kabelka, D., & Klenotová, E. (2023). Enhancing soil organic matter transformation through sustainable farming practices: Evaluating labile soil organic matter fraction dynamics and identifying potential early indicators. Agriculture (Switzerland), 13(7), https://doi.org/10.3390/agriculture13071314 google scholar
45. Neofytou, E., Neophytides, S. P., Eliades, M., Papoutsa, C., Tzouvaras, M., & Hadjimitsis, D. G. (2024). A review of soil organic carbon (SOC) prediction techniques in agricultural lands using remote sensing. Agricultural Lands and Remote Sensing, 1273–1279. doi: 10.1109/igarss53475.2024.10641028 google scholar
46. Nguyen, T. T., Pham, T. D., Nguyen, C. T., Delfos, J., Archibald, R., Dang, K. B., Hoang, N. B., Guo, W., & Ngo, H. H. (2022). A novel intelligence approach based active and ensemble learning for agricultural soil organic carbon prediction using multispectral and SAR data fusion. Science of the Total Environment, 804. https://doi.org/10.1016/j.scitotenv.2021.150187 google scholar
47. Öğretmen, M., & Gümüşçü, A. (2025). Identification of Corn and Wheat Plants Using Radar and Optical Image Data Inspiring Technologies and Innovations, 4(1). https://doi.org/10.5281/zenodo.15735401 google scholar
48. Peng, X., Zhong, R., Li, Z., & Li, Q. (2020). Optical remote sensing image change detection based on attention mechanism and image difference. IEEE Transactions on Geoscience and Remote Sensing, 59(9), 7296–7307. https://doi.org/10.1109/tgrs.2020.3033009 google scholar
49. Ramasamy, M., Ghosh, S., Yadav, K. K., Chitra, M., Dhivya, B., Sanjay, A. A. K., & Karthickraja, A. (2024). The role of organic farming in enhancing soil structure and crop performance: A comprehensive review. Journal of Scientific Research and Reports, 30(10), 890–904. https://doi.org/10.9734/jsrr/2024/v30i102511 google scholar
50. Sandeep, D., Harender, D., Asha, D., Shabnam, S., & Priyanka, S. (2023). Impact of organic farming practices on soil organic matter: A review. International Journal of Plant & Soil Science, 35(19). google scholar
51. Talaat, F. M., Takieldeen, A. E., & Shaban, W. M. (2025). Leveraging IoT and Climate Insights for Soil Property Prediction in Precision Agriculture. 2025 International Telecommunications Conference, ITC-Egypt 2025, 103–108. https://doi.org/10.1109/ITC-Egypt66095.2025.11186621. google scholar
52. Tianle, L., Quanhua, Z., Shuhan, J., Yu, L., & Xueliang, G. (2026). Enhanced soil organic matter estimation using spectral feature selection with empirical mode decomposition and pearson correlation coefficients. Infrared Physics and Technology, 152. https://doi.org/10.1016/j.infrared.2025.106084https://doi.org/10.1016/j.infrared.2025.106084 google scholar
53. Triantakonstantis, D., & Karakostas, A. (2025). Soil organic carbon monitoring and modeling via machine learning methods using soil and remote sensing data. Agriculture (Switzerland), 15(9). https://doi.org/10.3390/agriculture15090910 google scholar
54. Wang, H., Magagi, R., Goïta, K., Duguay, Y., Trudel, M., & Muhuri, A. (2023). Retrieval performances of different crop growth descriptors from full- and compact-polarimetric SAR decompositions. Remote Sens. Environ., 285. https://doi.org/10.1016/j.rse.2022.113381 google scholar
55. Wang, H., Singh, K., & Ravichandran, P. (2023). Compact-polarimetric SAR decompositions for soil moisture retrievals using Radarsat Constellation Mission data. 2023 11th International Conference on Agro-Geoinformatics, Agro-Geoinformatics 2023. https://doi.org/10.1109/Agro-Geoinformatics59224.2023.10233383 google scholar
56. Wen, D., Huang, X., Bovolo, F., Li, J., Ke, X., Zhang, A., & Benediktsson, J. A. (2021). Change detection from very-high-spatial-resolution optical remote sensing images: methods, applications, and future directions. IEEE Gasci and Remote Sensing Magazine, 9(4), 68–101. https://doi.org/10.1109/MGRS.2021.3063465 google scholar
57. Xie, Q., Lai, K., Wang, J., Lopez-Sanchez, J. M., Shang, J., Liao, C., Zhu, J., Fu, H., & Peng, X. (2021). Crop monitoring and classification using polarimetric radarsat-2 time-series data across growing season: A case study in southwestern Ontario, Canada. Remote Sensing, 13(7). https://doi.org/10.3390/rs13071394 google scholar
58. Xu, J., Vakalopoulou, M., Bélair, S., Powers, J., Carrera, M., Sun, L., & Dabboor, M. (2024). The RADARSAT Constellation Mission for Soil Moisture Retrieval of Bare Soil by Compact Polarimetry and Random Forest Regression. Canadian Journal of Remote Sensing, 50(1). https://doi.org/10.1080/07038992.2024.2356688 google scholar
59. M. Yasir Hanif, S. Aanisa Manzoor, S. Tajamul Islam, B. Shabir, J. Lizny, K. Shuraik, M. Ronaldo Luiz, and M. Rastko, M. (2023). Methodological Challenges in Estimating Soil Organic Matter: A Review,” Agriculture & Forestry, vol. 69, no. 4, pp. 275–283, 2023. google scholar
60. Yi, Y., Bakian-Dogaheh, K., Moghaddam, M., Mishra, U., & Kimball, J. S. (2023). Mapping surface organic soil properties in Arctic tundra using C-band SAR data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16, 1403–1413. https://doi.org/10.1109/JSTARS.2023.3236117 google scholar
61. Zhao, X., & Wang, Z. (2025). Review of full-polarimetric ground penetrating radar. Advances in Engineering Research Possibilities and Challenges, 2(1), 5. https://doi.org/10.63313/aerpc.9032https://doi.org/10.63313/aerpc.9032 google scholar
62. Zingaro, M., Scicchitano, G., Palmentola, P., Piscitelli, A., Refice, A., Roseto, R., Scardino, G., & Capolongo, D. (2023). Contribution of the sediment flow connectivity index (SfCI) in landscape archeology investigations: test case of a new interdisciplinary approach. Sustainability (Switzerland), 15(20). https://doi.org/10.3390/su152015042 google scholar