Optimizing Machine Learning Models for Soil Fertility Analysis: Insights from Feature Engineering and Data Localization

Abstract

Soil fertility is a critical determinant of agricultural productivity, yet traditional assessment methods often fall short in providing timely and precise recommendations. This study explores the potential of machine learning (ML) models to predict soil fertility, leveraging localized soil data and advanced feature engineering techniques. A comprehensive methodology was employed, involving data preprocessing, feature selection, and the implementation of six ML algorithms: Random Forest Regressor, Gradient Boosting Regressor, XGBoost Regressor, K-Nearest Neighbours Regressor, and Neural Network (MLP). The models were evaluated using robust metrics such as RMSE, R², and K-Fold Cross-Validation. Results demonstrate that engineered features significantly enhanced model performance, with Random Forest Regressor consistently outperforming other models across multiple soil nutrient parameters, achieving a testing R² of up to 0.99 and minimal RMSE. Exploratory Data Analysis (EDA) revealed key insights into soil nutrient dynamics, emphasizing the importance of pH, nitrogen, and organic matter as predictors. Feature engineering techniques, such as polynomial generation and scaling, further improved model accuracy and stability. This study highlights the transformative potential of ML in optimizing soil management practices. By integrating localized data and advanced predictive models, the findings provide actionable insights for farmers and agronomists, fostering sustainable agricultural practices and informed decision-making. This approach underscores the value of data-driven methods in addressing soil fertility challenges, paving the way for scalable and cost-effective solutions in precision agriculture.

Keywords

• Soil Fertility
• Machine Learning
• Feature Engineering
• Predictive Modelling
• Agricultural Optimization

References

1. Abishek, J. (2023). Soil texture prediction using machine learning approach for sustainable soil health management. International Journal of Plant and Soil Science, 35(19), 1416-1426. https://doi.org/10.9734/ijpss/2023/v35i193685
2. Asif, M. (2024). Leveraging machine learning for soil fertility prediction and crop management in agriculture. https://doi.org/10.21203/rs.3.rs-4310747/v1
3. Awais, M. (2023). Ai and machine learning for soil analysis: an assessment of sustainable agricultural practices. Bioresources and Bioprocessing, 10(1). https://doi.org/10.1186/s40643-023-00710-y
4. Barrena-González, J. (2024). Looking for optimal maps of soil properties at the regional scale. International Journal of Environmental Research, 18(4). https://doi.org/10.1007/s41742-024-00611-8
5. Chen, Y., Shi, T., Li, Q., Wang, Z., Wang, R., Wang, F., … and Li, Y. (2024). Mapping soil properties in tropical rainforest area using uav-based hyperspectral images and lidar points. https://doi.org/10.21203/rs.3.rs-4273924/v1
6. Clark, J., Fernández, F., Veum, K., Camberato, J., Carter, P., Ferguson, R., … and Shanahan, J. (2019). Predicting economic optimal nitrogen rate with the anaerobic potentially mineralizable nitrogen test. Agronomy Journal, 111(6), 3329-3338. https://doi.org/10.2134/agronj2019.03.0224
7. Dinh, T., Nguyen, H., Tran, X., and Hoang, N. (2021). Predicting rainfall-induced soil erosion based on a hybridization of adaptive differential evolution and support vector machine classification. Mathematical Problems in Engineering, 2021, 1-20. https://doi.org/10.1155/2021/6647829
8. Ezenwankwo, S., Adeagbo, A.A., Lawal, S., Idoghor, S. M. and Chukwu, O.(2020). Evaluation of early growth of Maesobotrya barteri (Hutch) seedlings underdifferent growing media and watering regime. In: Forestry Development in Nigeria: Fiftyyears of interventions and Advocacy. At the 42nd Annual conference of the Forestry Association of Nigeria (FAN), on 23-28 November, 2020, Ibadan, Nigeria. pp. 730-736.

9. Groebner, B. (2024). Soil biological and physical measurements did not improve the predictability of corn response to phosphorus fertilization. Agronomy Journal, 116(4), 2048-2059. https://doi.org/10.1002/agj2.21612
10. Hamidović, S., SOFTIC, A., Topčić, F., Tvica, M., Lalević, B., and Stojanova, M. (2023). Impact of soil management practice on the abundance of microbial populations. The Journal Agriculture and Forestry, 69(2). https://doi.org/10.17707/agricultforest.69.2.12
11. Haq, Y., Shahbaz, M., Asif, H., Al-Laith, A., and Alsabban, W. (2023). Spatial mapping of soil salinity using machine learning and remote sensing in kot addu, pakistan. Sustainability, 15(17), 12943. https://doi.org/10.3390/su151712943
12. Harris, J., Bledsoe, R., Guha, S., Omari, H., Crandall, S., Burghardt, L., … and Couradeau, E. (2024). The activity of soil microbial taxa in the rhizosphere predicts the success of root colonization.. https://doi.org/10.1101/2024.12.07.627353
13. Hu, Z., Ding, Z., Al-Yasi, H., Ali, E., Eissa, M., Abou‐Elwafa, S., … and Hamada, A. (2021). Modelling of phosphorus nutrition to obtain maximum yield, high p use efficiency and low p-loss risk for wheat grown in sandy calcareous soils. Agronomy, 11(10), 1950. https://doi.org/10.3390/agronomy11101950
14. Inoyatova, M. (2024). Data mining for assessing soil fertility. E3s Web of Conferences, 494, 02012. https://doi.org/10.1051/e3sconf/202449402012
15. Jabborova, D., Choudhary, R., Azimov, A., Jabbarov, Z., Selim, S., Abu-Elghait, M., … and Elsaied, A. (2022). Composition of zingiber officinale roscoe (ginger), soil properties and soil enzyme activities grown in different concentration of mineral fertilizers. Horticulturae, 8(1), 43. https://doi.org/10.3390/horticulturae8010043
16. Jia, X. (2023). Development of soil fertility index using machine learning and visible-near-infrared spectroscopy. Land, 12(12), 2155. https://doi.org/10.3390/land12122155
17. Kroyan, S. (2024). Anthropogenic changes of the agricultural production features of river valley-еscarpment soils in martuni region, sevan basin, ra. E3s Web of Conferences, 510, 01009. https://doi.org/10.1051/e3sconf/202451001009
18. Lepcha, N. and Devi, N. (2020). Effect of land use, season, and soil depth on soil microbial biomass carbon of eastern himalayas. Ecological Processes, 9(1). https://doi.org/10.1186/s13717-020-00269-y
19. Li, M., Ji, R., Wang, M., and Zheng, L. (2020). Comparison of soil total nitrogen content prediction models based on vis-nir spectroscopy. Sensors, 20(24), 7078. https://doi.org/10.3390/s20247078
20. Liu, W., Yang, Z., Ye, Q., Peng, Z., Zhu, S., Chen, H., … and Huang, H. (2023). Positive effects of organic amendments on soil microbes and their functionality in agro-ecosystems. Plants, 12(22), 3790. https://doi.org/10.3390/plants12223790
21. Longchamps, L., Mandal, D., and Khosla, R. (2022). Assessment of soil fertility using induced fluorescence and machine learning. Sensors, 22(12), 4644. https://doi.org/10.3390/s22124644
22. Ma, G., Cheng, S., He, W., Dong, Y., Qi, S., Nai-mei, T., … and Wei, T. (2023). Effects of organic and inorganic fertilizers on soil nutrient conditions in rice fields with varying soil fertility. Land, 12(5), 1026. https://doi.org/10.3390/land12051026
23. Mendoza, M., Mora-Bautista, M., Cué, J., Escudero, J., and Etchevers, J. (2021). Field production of kale (brassica oleracea var. acephala) with different nutrition sources. Agro Productividad. https://doi.org/10.32854/agrop.v14i10.1954
24. Mesfin, S., Haile, M., Gebresamuel, G., Zenebe, A., and Gebre, A. (2021). Establishment and validation of site-specific fertilizer recommendation for increased barley (hordeum spp.) yield, northern Ethiopia. Helion, 7(8), e07758. https://doi.org/10.1016/j.heliyon.2021.e07758
25. Musanase, C. (2023). Data-driven analysis and machine learning-based crop and fertilizer recommendation system for revolutionizing farming practices. Agriculture, 13(11), 2141. https://doi.org/10.3390/agriculture13112141
26. Nelson, A., Narrowe, A., Rhoades, C., Fegel, T., Daly, R., Roth, H., … and Wilkins, M. (2022). Wildfire-dependent changes in soil microbiome diversity and function. Nature Microbiology, 7(9), 1419-1430. https://doi.org/10.1038/s41564-022-01203-y
27. Ning, Q., Hättenschwiler, S., Lü, X., Kardol, P., Zhang, Y., Wei, C., … and Han, X. (2021). Carbon limitation overrides acidification in mediating soil microbial activity to nitrogen enrichment in a temperate grassland. Global Change Biology, 27(22), 5976-5988. https://doi.org/10.1111/gcb.15819
28. Nwamekwe, C. O., Ewuzie, N. V., Igbokwe, N. C., Okpala, C. C., and U-Dominic, C. M. (2024). Sustainable Manufacturing Practices in Nigeria: Optimization and Implementation Appraisal. Journal of Research in Engineering and Applied Sciences, 9(3). https://qtanalytics.in/journals/index.php/JREAS/article/view/3967
29. Nwamekwe, C. O., Ewuzie, N. V., Igbokwe, N. C., U-Dominic, C. M., and Nwabueze, C. V. (2024). Adoption of Smart Factories in Nigeria: Problems, Obstacles, Remedies and Opportunities. International Journal of Industrial and Production Engineering, 2(2). Retrieved from https://journals.unizik.edu.ng/ijipe/article/view/4167
30. Nwamekwe, C. O., Okpala, C. C., and Okpala, S. C., (2024). Machine Learning-Based Prediction Algorithms for the Mitigation of Maternal and Fetal Mortality in the Nigerian Tertiary Hospitals. International Journal of Engineering Inventions, 13(7), PP: 132-138. https://www.ijeijournal.com/papers/Vol13-Issue7/1307132138.pdf
31. Omar, G. and Sule, H. (2017). Fertility status of floodplain soils along river andlt;iandgt; tatsewarkiandlt;/iandgt;, kano. Bayero Journal of Pure and Applied Sciences, 9(2), 17. https://doi.org/10.4314/bajopas.v9i2.4
32. Osaigbovo, A. and Law-Ogbomo, K. (2014). Effects of spent engine oil polluted soil and organic amendment on soil chemical properties, micro-flora on growth and herbage of andlt;iandgt;telfairia occidentalisandlt;/iandgt; (hook f).. Bayero Journal of Pure and Applied Sciences, 6(1), 72. https://doi.org/10.4314/bajopas.v6i1.15
33. Paepae, T., Bokoro, P., and Kyamakya, K. (2022). A virtual sensing concept for nitrogen and phosphorus monitoring using machine learning techniques. Sensors, 22(19), 7338. https://doi.org/10.3390/s22197338
34. Pagliarini, M., Castilho, R., Moreira, E., Mariano-Nasser, F., and Alves, M. (2019). Development of hymenaea courbaril l. var. stilbocarpa seedlings in different fertilizers and substrate composition. Agrarian, 12(43), 8-15. https://doi.org/10.30612/agrarian.v12i43.4184
35. Palansooriya, K., Wong, J., Hashimoto, Y., Huang, L., Rinklebe, J., Chang, S., … and Ok, Y. (2019). Response of microbial communities to biochar-amended soils: a critical review. Biochar, 1(1), 3-22. https://doi.org/10.1007/s42773-019-00009-2
36. Pant, H., Lohani, M., and Bhatt, A. (2019). Impact of physico-chemical properties for soils type classification of oak using different machine learning techniques. International Journal of Computer Applications, 177(17), 38-44. https://doi.org/10.5120/ijca2019919617
37. Patil, A., Kulkarni, V., and Desai, S. (2023). Soil fertility prediction. International Journal for Research in Applied Science and Engineering Technology, 11(8), 1241-1247. https://doi.org/10.22214/ijraset.2023.55225
38. Prince, M., Mankessi, F., Sun, S., and Fan, X. (2021). Effects of alkaline fertilizer and rice cultivation (oryza sativa l.) on remediation of soils polluted with cadmium (cd). Journal of Applied Biosciences, 157, 16182-16193. https://doi.org/10.35759/jabs.157.4
39. Rajamanickam, J. and Mani, S. (2021). Kullback chi square and gustafson kessel probabilistic neural network-based soil fertility prediction. Concurrency and Computation Practice and Experience, 33(24). https://doi.org/10.1002/cpe.6460
40. Razanov, S. (2024). The content of heavy metals and trace elements in different soils used under the conditions of homestead plots and field agricultural lands of ukraine. Journal of Ecological Engineering, 25(6), 42-50. https://doi.org/10.12911/22998993/186820
41. Reddy, L. (2024). Applying machine learning to soil analysis for accurate farming. Matec Web of Conferences, 392, 01124. https://doi.org/10.1051/matecconf/202439201124
42. Rehman, O., Mehdi, S., Abad, R., Saleem, S., Khalid, R., Alvi, S., … and Munir, A. (2021). Soil characteristics and fertility indexation in gujar khan area of rawalpindi. Pakistan Journal of Scientific and Industrial Research Series a Physical Sciences, 64(1), 46-51. https://doi.org/10.52763/pjsir.phys.sci.64.1.2021.46.51
43. Sandhya, K., Gayathri, B., Papireddy, M., R, P., Vishwanath, V., Swathi, B., … and Naveen, D. (2023). Assessing the nutrient status and soil fertility using nutrient indexed of farmer’s fields in chikkaballapura district, karnataka. International Journal of Plant and Soil Science, 35(23), 639-649. https://doi.org/10.9734/ijpss/2023/v35i234283
44. Saraiva, T., Ventura, S., Brito, E., Rocha, S., Costa, R., Pereira, A., … and Araújo, A. (2022). Temporal stability of soil microbial properties in responses to long-term application of compost obtained from tannery sludge. Sustainability, 14(24), 16736. https://doi.org/10.3390/su142416736
45. Sofo, A., Mininni, A., and Ricciuti, P. (2020). Soil macrofauna: a key factor for increasing soil fertility and promoting sustainable soil use in fruit orchard agrosystems. Agronomy, 10(4), 456. https://doi.org/10.3390/agronomy10040456
46. Sridevy, S., Raj, M., Kumaresan, P., Balakrishnan, N., Tilak, M., Raj, J., … and Rani, P. (2023). Mapping of soil properties using machine learning techniques. International Journal of Environment and Climate Change, 13(8), 684-700. https://doi.org/10.9734/ijecc/2023/v13i81997
47. Tryhuba, I., Тryhuba, А., Hutsol, T., Cieszewska, A., Andrushkiv, O., Głowacki, S., … and Sojak, M. (2024). Prediction of biogas production volumes from household organic waste based on machine learning. Energies, 17(7), 1786. https://doi.org/10.3390/en17071786
48. Yageta, Y., Osbahr, H., Morimoto, Y., and Clark, J. (2019). Comparing farmers' qualitative evaluation of soil fertility with quantitative soil fertility indicators in kitui county, kenya. Geoderma, 344, 153-163. https://doi.org/10.1016/j.geoderma.2019.01.019
49. Yang, Q., Peng, J., Ni, S., Zhang, C., Wang, J., and Cai, C. (2024). Soil erosion‐induced decline in aggregate stability and soil organic carbon reduces aggregate‐associated microbial diversity and multifunctionality of agricultural slope in the mollisol region. Land Degradation and Development, 35(11), 3714-3726. https://doi.org/10.1002/ldr.5163
50. Yerrabolu, V., Kasireddy, I., Jasmine, K., Vamsi, T., Joshua, N., Kumar, V., … and Rao, D. (2024). Performance comparison of random forest regressor and support vector regression for solar energy prediction. Iop Conference Series Earth and Environmental Science, 1375(1), 012013. https://doi.org/10.1088/1755-1315/1375/1/012013
51. Yu, X. (2024). Prediction model of nitrogen, phosphorus, and potassium fertilizer application rate for greenhouse tomatoes under different soil fertility conditions. Agronomy, 14(6), 1165. https://doi.org/10.3390/agronomy14061165
52. Zhao, Z., He, J., Quan, Z., Wu, C., Sheng, R., Zhang, L., … and Geisen, S. (2020). Fertilization changes soil microbiome functioning, especially phagotrophic protists. Soil Biology and Biochemistry, 148, 107863. https://doi.org/10.1016/j.soilbio.2020.107863
53. Zheng, C., Yang, X., Liu, Z., Liu, K., and Huang, Y. (2022). Spatial distribution of soil nutrients and evaluation of cultivated land in xuwen county. Peerj, 10, e13239. https://doi.org/10.7717/peerj.13239
54. Ziyadullaev, D. (2024). Ensemble data mining methods for assessing soil fertility. E3s Web of Conferences, 494, 02013. https://doi.org/10.1051/e3sconf/202449402013