Hyperparameter Optimization In Polynomial-Based SVM Algorithms and Its Application

Abstract

In this study, regression analysis of support vector machines (SVMs) with different polynomial kernels is discussed. Using linear-svm, quadratic-svm and cubic-svm regression algorithms, the performance of each model is examined through an optimization process on the Box-Constraint, Kernel Scale, Epsilon and Standardize parameters. Setting these parameters correctly is critical for the error and approximation metrics of the model. The optimization process was performed with Matlab Regression Learner using the Bayesian Optimization algorithm. Hyperparameter optimized polynomial-based regression models can predict nutrient deficiency in plants with high accuracy. In particular, accurate determination of calcium content in leaves is important to increase fertilization success during the fruit development period in apple trees. In the application part of our study, data sets obtained by digitizing leaf surfaces and data sets obtained from chemical laboratory analysis were modeled. This study using SVM algorithms provides a more efficient model than chemical methods in terms of cost and time.

Keywords

• machine learning
• svm
• bayesian optimization

Polinom Tabanlı SVM Algoritmalarında Hiper-Parametre Optimizasyonu ve Uygulaması

Abstract

Bu çalışmada, destek vektör makinelerinin (SVM) farklı polinom çekirdeklerini içeren regresyon analizleri ele alınmıştır. Linear-svm, quadratic-svm ve cubic-svm regresyon algoritmaları kullanılarak her bir modelin performansı, Box-Constraint, Kernel Scale, Epsilon ve Standardize parametreleri üzerinde gerçekleştirilen optimizasyon süreciyle incelenmiştir. Bu parametrelerin doğru bir şekilde ayarlanması, modelin hata ve yaklaşım metrikleri için kritik önem taşımaktadır. Optimizasyon süreci, Bayesian Optimization algoritması kullanılarak Matlab Regression Learner ile gerçekleştirilmiştir. Hiperparametre optimizasyonu yapılmış polinom tabanlı regresyon modelleri, bitkilerde besin elementlerinin eksikliğini yüksek doğrulukla tahmin edebilmektedirler. Özellikle yapraklardaki kalsiyum miktarının doğru bir şekilde belirlenmesi, elma ağaçlarında meyve gelişim döneminde gübreleme başarısını artırmak açısından önem taşımaktadır. Çalışmamızın uygulama bölümünde yaprak yüzeylerinin sayısallaştırılması ile elde edilen veri setleri ile kimyasal laboratuvar analizlerinden elde edilen veri seti modellenmiştir. SVM algoritmaları kullanılarak yapılan bu çalışma, maliyet ve zaman açısından kimyasal yöntemlere göre daha verimli bir model sunmaktadır.

Keywords

• makine öğrenmesi
• svm
• bayesian optimization

References

1. [1] H. Fattahi and N. Babanouri, “Applying Optimized Support Vector Regression Models for Prediction of Tunnel Boring Machine Performance,” Geotech. Geol. Eng., vol. 35, no. 5, pp. 2205–2217, Oct. 2017. doi:10.1007/S10706-017-0238-4/TABLES/8.
2. [2] V. Strijov and G. W. Weber, “Nonlinear regression model generation using hyperparameter optimization,” Comput. Math. with Appl., vol. 60, no. 4, pp. 981–988, Aug. 2010. doi:10.1016/J.CAMWA.2010.03.021.
3. [3] P. Yu, M. Y. Low, and W. Zhou, “Design of experiments and regression modelling in food flavour and sensory analysis: a review,” Trends Food Sci. Technol., vol. 71, pp. 202–215, Jan. 2018. doi:10.1016/j.tifs.2017.11.013.
4. [4] A. Kamilaris and F. X. Prenafeta-Boldú, “Deep learning in agriculture: A survey,” Comput. Electron. Agric., vol. 147, pp. 70–90, Apr. 2018. doi: 10.1016/J.COMPAG.2018.02.016.
5. [5] P. Freund, R. J., Wilson, W. J., Sa, Regression analysis: Statistical Modeling of a response variable (2nd ed). California, USA: Elsevier, 2006.
6. [6] N. Panigrahi and B. S. Das, “Evaluation of regression algorithms for estimating leaf area index and canopy water content from water stressed rice canopy reflectance,” Inf. Process. Agric., vol. 8, no. 2, pp. 284–298, 2021. doi:10.1016/j.inpa.2020.06.002.
7. [7] I. Keramatlou, M. Sharifani, H. Sabouri, M. Alizadeh, and B. Kamkar, “A simple linear model for leaf area estimation in Persian walnut (Juglansregia L.),” Sci. Hortic. (Amsterdam)., vol. 184, pp. 36–39, Mar. 2015. doi:10.1016/j.scienta.2014.12.017.
8. [8] Basak, J. K., Qasim, W., Okyere, F. G., Khan, F., Lee, Y. J., Park, J., and Kim, H. T. , “Regression Analysis to Estimate Morphology Parameters of Pepper Plant in a Controlled Greenhouse System,” J. Biosyst. Eng., vol. 44, no. 2, pp. 57–68, Jun. 2019. doi:10.1007/S42853-019-00014-0/FIGURES/11.

9. [9] H. Armağan, “Color Based Segmentation with k-Means Clustering Algorithm and Numerical Analysis of the Effect of Color Spaces on Image Quantities.,” El-Cezeri, vol. 9, no. 4, pp. 1506–1517, Dec. 2022. doi:10.31202/ECJSE.1141148.
10. [10] K. G. Liakos, P. Busato, D. Moshou, S. Pearson, and D. Bochtis, “Machine Learning in Agriculture: A Review,” Sensors, Vol. 18, no. 8, p. 2674, Aug. 2018. doi: 10.3390/S18082674.
11. [11] T. U. Rehman, M. S. Mahmud, Y. K. Chang, J. Jin, and J. Shin, “Current and future applications of statistical machine learning algorithms for agricultural machine vision systems,” Comput. Electron. Agric., vol. 156, pp. 585–605, Jan. 2019. doi:10.1016/J.COMPAG.2018.12.006.
12. [12] M. Pathan, N. Patel, H. Yagnik, and M. Shah, “Artificial cognition for applications in smart agriculture: A comprehensive review,” Artif. Intell. Agric., vol. 4, pp. 81–95, Jan. 2020. doi:10.1016/J.AIIA.2020.06.001.
13. [13] “K-means clustering based image segmentation - MATLAB imsegkmeans.” [Online]. Available: https://www.mathworks.com/help/images/ref/imsegkmeans.html [Accessed Sep. 22, 2022].
14. [14] M. Shahhosseini, G. Hu, and H. Pham, “Optimizing ensemble weights and hyperparameters of machine learning models for regression problems,” Mach. Learn. with Appl., vol. 7, p. 100251, 2022. doi:10.1016/j.mlwa.2022.100251.
15. [15] P. M. Granitto, H. D. Navone, P. F. Verdes, and H. A. Ceccatto, “Weed seeds identification by machine vision,” Comput. Electron. Agric., vol. 33, no. 2, pp. 91–103, 2002. doi:10.1016/S0168-1699(02)00004-2.
16. [16] J. G. A. Barbedo, “Detection of nutrition deficiencies in plants using proximal images and machine learning: A review,” Comput. Electron. Agric., vol. 162, pp. 482–492, Jul. 2019. doi:10.1016/J.COMPAG.2019.04.035.
17. [17] X. Ye, S. Abe, and S. Zhang, “Estimation and mapping of nitrogen content in apple trees at leaf and canopy levels using hyperspectral imaging,” Precis. Agric., vol. 21, no. 1, pp. 198–225, Feb. 2020. doi:10.1007/S11119-019-09661-X/FIGURES/15.
18. [18] Chen, S., Hu, T., Luo, L., He, Q., Zhang, S., Li, M.,Cui, X. and Li, H, “Rapid estimation of leaf nitrogen content in apple-trees based on canopy hyperspectral reflectance using multivariate methods,” Infrared Phys. Technol., vol. 111, p. 103542, Dec. 2020. doi:10.1016/J.INFRARED.2020.103542.
19. [19] Morellos, A., Pantazi, X. E., Moshou, D., Alexandridis, T., Whetton, R., Tziotzios, G., Weibensohn, J., Bill, R. and Mouazen, , “Machine learning based prediction of soil total nitrogen, organic carbon and moisture content by using VIS-NIR spectroscopy,” Biosyst. Eng., vol. 152, pp. 104–116, Dec. 2016. doi:10.1016/J.BIOSYSTEMSENG.2016.04.018.
20. [20] G. Liu, S. Mao, and J. H. Kim, “A mature-tomato detection algorithm using machine learning and color analysis,” Sensors (Switzerland), vol. 19, no. 9, pp. 1–19, 2019. doi:10.3390/s19092023.
21. [21] C. Lisu, S. Yuanyuan, and W. Ke, “Rapid diagnosis of nitrogen nutrition status in rice based on static scanning and extraction of leaf and sheath characteristics,” Int. J. Agric. Biol. Eng., vol. 10, no. 3, pp. 158–164, May 2017. doi:10.25165/IJABE.V10I3.1860.
22. [22] M. Yang, D. Xu, S. Chen, H. Li, and Z. Shi, “Evaluation of Machine Learning Approaches to Predict Soil Organic Matter and pH Using vis-NIR Spectra,” Sensors, vol. 19, no. 2, p. 263, Jan. 2019. doi:10.3390/S19020263.
23. [23] M. Altalak, M. A. Uddin, A. Alajmi, and A. Rizg, “Smart Agriculture Applications Using Deep Learning Technologies: A Survey,” Appl. Sci., vol. 12, no. 12, Jun. 2022. doi:10.3390/app12125919.