WF-AlexNet:AlexNet with Automatically Optimized Hyperparameters for Weather Forecasting

Year 2024, Volume: 5 Issue: 2, 8 - 15, 30.12.2024

Soner Kızıloluk , Eser Sert

https://doi.org/10.46572/naturengs.1545180

Abstract

Image classification is a critical area of research with widespread applications across various disciplines, including computer vision, pattern recognition, and artificial intelligence. Despite the advancements in Convolutional Neural Networks (CNNs), which have revolutionized the field by providing powerful tools for image classification, many studies have encountered challenges in achieving optimal classification performance. These challenges often arise from the complex nature of CNN architectures and the multitude of hyperparameters that require fine-tuning. Among the CNN models, AlexNet has been widely recognized for its contributions to deep learning, yet there remains significant potential for improvement through the optimization of its hyperparameters. In this study, WF-AlexNET designed to enhance the performance of the AlexNet architecture by optimizing the hyperparameters of its first convolutional layer using the Equilibrium Optimization (EO) algorithm. The EO algorithm, was employed to fine-tune the filter size, filter number, stride, and padding parameters, which are crucial for effective feature extraction. The proposed WF-AlexNET method was rigorously tested on a multi-class weather image dataset to evaluate its classification accuracy and robustness. Experimental results demonstrate that WF-AlexNET significantly outperforms the standard AlexNet model, achieving a 10.5% increase in mean validation accuracy and a 6.51% improvement in test accuracy. Furthermore, the proposed approach was compared against other prominent CNN architectures, including VGG-16, GoogleNet, ShuffleNet, MobileNet-V2, and VGG-19. WF-AlexNET consistently exhibited superior classification performance across multiple metrics, including F1-score and maximum accuracy, highlighting its efficacy in addressing the challenges associated with hyperparameter optimization in CNNs.

Keywords

Image classification, Equilibrium Optimization (EO), Convolutional Neural Networks (CNN), AlexNet

References

• Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P., (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86, 2278-324. https://doi.org/10.1109/5.726791
• Andonie, R., and Florea, A.-C., (2020). Weighted Random Search for CNN Hyperparameter Optimization. International Journal of Computers Communications & Control, 15. https://doi.org/10.15837/ijccc.2020.2.3868
• Faramarzi, A., Heidarinejad, M., Stephens, B., and Mirjalili, S., (2020). Equilibrium optimizer: A novel optimization algorithm. Knowledge-Based Systems, 191, 105190. https://doi.org/10.1016/j.knosys.2019.105190
• Wojciuk, M., Swiderska-Chadaj, Z., Siwek, K., and Gertych, A., (2024). Improving classification accuracy of fine-tuned CNN models: Impact of hyperparameter optimization. Heliyon, 10, e26586. https://doi.org/10.1016/j.heliyon.2024.e26586
• Nguyen, T., Nguyen, G., and Nguyen, B.M., (2020). EO-CNN: An Enhanced CNN Model Trained by Equilibrium Optimization for Traffic Transportation Prediction. Procedia Computer Science, 176, 800-9. https://doi.org/10.1016/j.procs.2020.09.075
• Bochinski, E., Senst, T., and Sikora, T., (2017). Hyper-parameter optimization for convolutional neural network committees based on evolutionary algorithms. 2017 IEEE International Conference on Image Processing (ICIP), IEEE, Beijing. s. 3924-8. https://doi.org/10.1109/ICIP.2017.8297018
• Esfahanian, P., and Akhavan, M., (2019). GACNN: Training Deep Convolutional Neural Networks with Genetic Algorithm. arXiv:1909.13354. https://arxiv.org/abs/1909.13354.
• Naik, N.K., Sethy, P.K., Devi, A.G., and Behera, S.K., (2024). Few-shot learning convolutional neural network for primitive indian paddy grain identification using 2D-DWT injection and grey wolf optimizer algorithm. Journal of Agriculture and Food Research, 15, 100929. https://doi.org/10.1016/j.jafr.2023.100929
• Wang, Y., Zhang, H., and Zhang, G., (2019). cPSO-CNN: An efficient PSO-based algorithm for fine-tuning hyper-parameters of convolutional neural networks. Swarm and Evolutionary Computation, 49, 114-23. https://doi.org/10.1016/j.swevo.2019.06.002
• Wang, Q., Jiang, H., Ren, J., Liu, H., Wang, X., and Zhang, B., (2024). An intrusion detection algorithm based on joint symmetric uncertainty and hyperparameter optimized fusion neural network. Expert Systems with Applications, 244, 123014. https://doi.org/10.1016/j.eswa.2023.123014
• Lee, W.-Y., Park, S.-M., and Sim, K.-B., (2018). Optimal hyperparameter tuning of convolutional neural networks based on the parameter-setting-free harmony search algorithm. Optik, 172, 359-67. https://doi.org/10.1016/j.ijleo.2018.07.044
• Emam, M.M., Houssein, E.H., Samee, N.A., Alohali, M.A., and Hosney, M.E., (2024). Breast cancer diagnosis using optimized deep convolutional neural network based on transfer learning technique and improved Coati optimization algorithm. Expert Systems with Applications, 255, 124581. https://doi.org/10.1016/j.eswa.2024.124581
• Soon, F.C., Khaw, H.Y., Chuah, J.H., and Kanesan, J., (2018). Hyper‐parameters optimisation of deep CNN architecture for vehicle logo recognition. IET Intelligent Transport Systems, 12, 939-46. https://doi.org/10.1049/iet-its.2018.5127
• Xiao, X., Yan, M., Basodi, S., Ji, C., and Pan, Y., (2020). Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm. arXiv:2006.12703. https://arxiv.org/abs/2006.12703.
• Yildirim, M., Çinar, A., & CengIl, E. (2022). Classification of the weather images with the proposed hybrid model using deep learning, SVM classifier, and mRMR feature selection methods. Geocarto International, 37(9), 2735-2745. https://doi.org/10.1080/10106049.2022.2034989
• Chen, L., Zhong, X., Zhang, F., Cheng, Y., Xu, Y., Qi, Y., & Li, H. (2023). FuXi: A cascade machine learning forecasting system for 15-day global weather forecast. npj Climate and Atmospheric Science, 6(1), 190. https://doi.org/10.1038/s41612-023-00512-1
• Venkatachalam, K., Trojovský, P., Pamucar, D., Bacanin, N., & Simic, V. (2023). DWFH: An improved data-driven deep weather forecasting hybrid model using Transductive Long Short Term Memory (T-LSTM). Expert systems with applications, 213, 119270. https://doi.org/10.1016/j.eswa.2022.119270
• Guo, T., Dong, J., Li, H., and Gao, Y., (2017). Simple convolutional neural network on image classification. 2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA)(, IEEE, Beijing, China. s. 721-4. https://doi.org/10.1109/ICBDA.2017.8078730
• Nhat-Duc, H., Nguyen, Q.-L., and Tran, V.-D., (2018). Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network. Automation in Construction, 94, 203-13. https://doi.org/10.1016/j.autcon.2018.07.008
• Yamashita, R., Nishio, M., Do, R.K.G., and Togashi, K., (2018). Convolutional neural networks: an overview and application in radiology. Insights into Imaging, 9, 611-29. https://doi.org/10.1007/s13244-018-0639-9
• Sinha, T., Haidar, A., and Verma, B., (2018). Particle Swarm Optimization Based Approach for Finding Optimal Values of Convolutional Neural Network Parameters. 2018 IEEE Congress on Evolutionary Computation (CEC), IEEE, Rio de Janeiro. s. 1-6. https://doi.org/10.1109/CEC.2018.8477728
• Krizhevsky, A., Sutskever, I., and Hinton, G.E., (2012). ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems (pp. 1097–1105).
• Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C., (2018). MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Salt Lake City, UT. s. 4510-20. https://doi.org/10.1109/CVPR.2018.00474
• Zhang, X., Zhou, X., Lin, M., and Sun, J., (2018). ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Salt Lake City, UT. s. 6848-56. https://doi.org/10.1109/CVPR.2018.00716
• Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A., (2015). Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Boston, MA, USA. s. 1-9. https://doi.org/10.1109/CVPR.2015.7298594
• Simonyan, K., and Zisserman, A., (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv:1409.1556. https://arxiv.org/abs/1409.1556.
• Ajayi, G., (2018). Multi-class Weather Dataset for Image Classification. Mendeley Data, v1. http://dx.doi.org/10.17632/4drtyfjtfy.1.
• Oluwafemi, A. G., and Wang Z., (2019). Multi-class weather classification from still image using said ensemble method. 2019 IEEE Southern African universities power engineering conference/robotics and mechatronics/pattern recognition association of South Africa (SAUPEC/RobMech/PRASA),135-140. https://doi.org/10.1109/RoboMech.2019.8704783
• Tian, M., Chen, X., Zhang, H., Zhang, P., Cao, K., and Wang, R., (2021). Weather classification method based on spiking neural network. 2021 IEEE International Conference on Digital Society and Intelligent Systems (DSInS), 134-137. https://doi.org/10.1109/DSInS54396.2021.9670557