Yeni Bir Derin Öğrenme Algoritması: Hibrit Katman İşbirliği Tabanlı İleri-İleri Ağ (HiLaCob-FF)

Öz

Bu çalışma, ileri-ileri algoritmasına dayanan "Hibrit Katman İş Birliği Tabanlı İleri-İleri Ağ (HiLaCob-FF)" adlı yeni bir öğrenme algoritması önermektedir. HiLaCob-FF algoritması, yerel bir kayıp yerine küresel bir kayıp ve hiperparametre optimizasyonu gibi yeni bir kayıp fonksiyonu kullanmaktadır. MNIST, Fashion MNIST ve CIFAR-10 veri kümelerine uygulanarak, örüntüleri tanıma ve çıkarımlar yapma konusundaki genelleme yeteneği kanıtlanmıştır. Bu çalışma, derin öğrenmede alternatif yaklaşımların önemini vurgulamaktadır. Önerilen yeni kayıp fonksiyonunun model eğitim kayıplarını iyileştirdiği sonucuna varılmıştır. Sonuç olarak, MNIST veri kümesine uygulanan en son kayıp fonksiyonuyla modelin eğitim kaybı %1,42 oranında azalarak %0,87'ye düşmüştür. Benzer şekilde, hata oranı CIFAR-10 veri kümesinde %40,91'den %40,36'ya düşmüştür. Önerdiğimiz HiLaCob-FF modeli, MINIST veri setinde %92,32, FashionMINIST veri setinde %78,29 ve CIFAR-10 veri setinde %74,20 başarı oranına ulaşmıştır. Bu performanslar, geri yayılım tabanlı modellerdeki kadar yüksek olmasa da, sonuçlar ileri-ileri ağlar geliştirmek için önemlidir.

Anahtar Kelimeler

• Derin Öğrenme
• İleri-İleri Algoritması
• Beyinden İlham Alan Modeller
• HiLaCob-FF
• Alternatif Öğrenme Algoritmaları

A Novel Deep Learning Algorithm: Hybrid Layer Collaboration-based Forward-Forward Network (HiLaCob-FF)

Öz

This study proposes a new learning algorithm, the "Hybrid Layer Collaboration-based Forward-Forward Network (HiLaCob-FF)," which is based on the forward-forward algorithm. The HiLaCob-FF algorithm employs a new loss function, a global loss rather than a local loss, and hyperparameter optimization. It applied to MNIST, Fashion MNIST, and CIFAR-10 datasets, demonstrating its generalization ability in recognizing patterns and making inferences. This study underscores the importance of alternative approaches in deep learning. It is concluded that the proposed new loss function improved the model training losses. As a result, with the latest loss function applied to the MNIST dataset, the model's training loss decreased by 1.42% to 0.87%. Similarly, the error rate reduced from 40.91% to 40.36% on the CIFAR-10 dataset. Our proposed HiLaCob-FF model achieves 92.32% success on the MINIST dataset, 78.29% on the FashionMINIST dataset, and 74.20% on the CIFAR-10 dataset. Although these performances are not as high as those of backpropagation-based models, the results are essential for developing forward-forward networks

Anahtar Kelimeler

• : Deep Learning
• Forward-Forward Algorithm
• Brain-Inspired Models
• HiLaCob-FF
• Alternative Learning Algorithms

Kaynakça

1. [1] W. Zhang, S. Ma, X. Ji, X. Liu, Y. Cong, and L. Shi, “The development of general-purpose brain-inspired computing,” Nature Electronics 2024 7:11, vol. 7, no. 11, pp. 954–965, Nov. 2024, doi: 10.1038/s41928-024-01277-y.
2. [2] Song, Y., Lukasiewicz, T., Xu, Z., & Bogacz, R. (2020). Can the brain do backpropagation?---exact implementation of backpropagation in predictive coding networks. Advances in neural information processing systems, 33, 22566-22579.
3. [3] Hinton, G. (2022). The forward-forward algorithm: Some preliminary investigations. arXiv preprint arXiv:2212.13345, 2(3), 5. https://www.cs.toronto.edu/~hinton/FFA13.pdf (03.05.2025)
4. [4] Y. Song, T. Lukasiewicz, Z. Xu, and R. Bogacz, “Can the brain do backpropagation? — Exact implementation of backpropagation in predictive coding networks,” Adv. Neural Inf. Process. Syst., vol. 2020-Decem, 2020.
5. [5] Y. Lecun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015, doi: 10.1038/nature14539.
6. [6] Werbos, P. J. (2005, September). Applications of advances in nonlinear sensitivity analysis. In System Modeling and Optimization: Proceedings of the 10th IFIP Conference New York City, USA, August 31–September 4, 1981 (pp. 762-770). Berlin, Heidelberg: Springer Berlin Heidelberg.
7. [7] S. Grossberg, “Competitive learning: From interactive activation to adaptive resonance,” Cogn. Sci., vol. 11, no. 1, pp. 23–63, 1987, doi: 10.1016/S0364-0213(87)80025-3.
8. [8] A. H. Marblestone, G. Wayne, and K. P. Kording, “Toward an integration of deep learning and neuroscience,” Front. Comput. Neurosci., vol. 10, no. SEP, p. 94, 2016, doi: 10.3389/fncom.2016.00094.

9. [9] T. P. Lillicrap, A. Santoro, L. Marris, C. J. Akerman, and G. Hinton, “Backpropagation and the brain,” Nat. Rev. Neurosci., vol. 21, no. 6, pp. 335–346, 2020, doi: 10.1038/s41583-020-0277-3.
10. [10] Pozzi, I., Bohte, S., & Roelfsema, P. (2020). Attention-Gated Brain Propagation: How the brain can implement reward-based error backpropagation. Advances in neural information processing systems, 33, 2516-2526.
11. [11] P. Mazzoni, R. A. Andersen, and M. I. Jordan, “A more biologically plausible learning rule for neural networks,” Proc. Natl. Acad. Sci. U. S. A., vol. 88, no. 10, pp. 4433–4437, 1991, doi: 10.1073/pnas.88.10.4433.
12. [12] Kohan, A. A., Rietman, E. A., & Siegelmann, H. T. (2018). Error forward-propagation: Reusing feedforward connections to propagate errors in deep learning. arXiv preprint arXiv:1808.03357.
13. [13] Bellec, G., Scherr, F., Hajek, E., Salaj, D., Legenstein, R., & Maass, W. (2019). Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets. arXiv preprint arXiv:1901.09049.
14. [14] Xie, X., & Seung, H. S. (2003). Equivalence of backpropagation and contrastive Hebbian learning in a layered network. Neural computation, 15(2), 441-454.
15. [15] Song, Y., Millidge, B., Salvatori, T., Lukasiewicz, T., Xu, Z., & Bogacz, R. (2024). Inferring neural activity before plasticity as a foundation for learning beyond backpropagation. Nature neuroscience, 27(2), 348-358.
16. [16] K. S. Rockland, “Notes on Visual Cortical Feedback and Feedforward Connections,” Front. Syst. Neurosci., vol. 16, 2022, doi: 10.3389/fnsys.2022.784310.
17. [17] Millidge, B., Seth, A., & Buckley, C. L. (2021). Predictive coding: a theoretical and experimental review. arXiv preprint arXiv:2107.12979.
18. [18] R. L. Sumner, M. J. Spriggs, S. D. Muthukumaraswamy, and I. J. Kirk, “The role of Hebbian learning in human perception: a methodological and theoretical review of the human Visual Long-Term Potentiation paradigm,” Neurosci. Biobehav. Rev., vol. 115, pp. 220–237, 2020, doi: 10.1016/j.neubiorev.2020.03.013.
19. [19] A. Bewtra, “A Further Investigation of the Forward-Forward Algorithm,” 2023.
20. [20] I. Oguz et al., “Forward–forward training of an optical neural network,” Opt. Lett., vol. 48, no. 20, p. 5249, 2023, doi: 10.1364/ol.496884.
21. [21] Tang, D. Y. (2023). The integrated forward-forward algorithm: Integrating forward-forward and shallow backpropagation with local losses. arXiv preprint arXiv:2305.12960.
22. [22] Ahamed, M. A., Chen, J., & Imran, A. A. Z. (2023). FFCL: Forward-Forward contrastive learning for improved medical image classification. In Medical Imaging with Deep Learning, short paper track.
23. [23] A. Reyes-Angulo and S. Paheding, “The Forward-Forward Algorithm as a feature extractor for skin lesion classification: A preliminary study.,” 2023. doi: 10.52591/lxai202307235.
24. [24] Zhao, G., Wang, T., Jin, Y., Lang, C., Li, Y., & Ling, H. (2025). The cascaded forward algorithm for neural network training. Pattern Recognition, 161, 111292.
25. [25] A. Mehonic and A. J. Kenyon, “Brain-inspired computing needs a master plan,” Nature 2022 604:7905, vol. 604, no. 7905, pp. 255–260, Apr. 2022, doi: 10.1038/s41586-021-04362-w.
26. [26] Y. Zhang et al., “A system hierarchy for brain-inspired computing,” Nature 2020 586:7829, vol. 586, no. 7829, pp. 378–384, Oct. 2020, doi: 10.1038/s41586-020-2782-y.
27. [27] G. Lorberbom, I. Gat, Y. Adi, A. Schwing, and T. Hazan, “Layer Collaboration in the Forward-Forward Algorithm,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 13, pp. 14141–14148, Mar. 2024, doi: 10.1609/aaai.v38i13.29324.
28. [28] Y. LeCun, (1998). The MNIST database of handwritten digits. http://yann. lecun. com/exdb/mnist/.
29. [29] Xiao, H., Rasul, K., & Vollgraf, R. (2017). Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747.
30. [30] A. Krizhevsky, V. Nair, & G.Hinton. (2014, September). The CIFAR-10 dataset.
31. [31] H.-C. Lee and J. Song, “SymBa: Symmetric Backpropagation-Free Contrastive Learning with Forward-Forward Algorithm for Optimizing Convergence,” Mar. 2023. doi: https://doi.org/10.48550/arXiv.2303.08418.
32. [32] M. Ghader, S. R. Kheradpisheh, B. Farahani, and M. Fazlali, “Backpropagation-free spiking neural networks with the forward–forward algorithm,” Scientific Reports 2026 16:1, vol. 16, no. 1, pp. 14294-, Mar. 2026, doi: 10.1038/s41598-026-41671-4.
33. [33] S. Sunkari et al., “A refined ResNet18 architecture with Swish activation function for Diabetic Retinopathy classification,” Biomed. Signal Process. Control, vol. 88, p. 105630, Feb. 2024, doi: 10.1016/j.bspc.2023.105630.
34. [34] M. A. Mercioni and S. Holban, “P-Swish: Activation Function with Learnable Parameters Based on Swish Activation Function in Deep Learning,” 2020 14th International Symposium on Electronics and Telecommunications, ISETC 2020 - Conference Proceedings, Nov. 2020, doi: 10.1109/ISETC50328.2020.9301059.
35. [35] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533–536, 1986, doi: 10.1038/323533a0.
36. [36] Gandhi, S., Gala, R., Kornberg, J., & Sridhar, A. (2023). Extending the forward forward algorithm. arXiv preprint arXiv:2307.04205.
37. [37] M. A. Mercioni and S. Holban, “Soft-Clipping Swish: A Novel Activation Function for Deep Learning,” in SACI 2021 - IEEE 15th International Symposium on Applied Computational Intelligence and Informatics, Proceedings, 2021, pp. 225–230. doi: 10.1109/SACI51354.2021.9465622.
38. [38] M. Sokolova, N. Japkowicz, and S. Szpakowicz, “Beyond accuracy, F-score and ROC: A family of discriminant measures for performance evaluation,” in AAAI Workshop - Technical Report, Springer Berlin Heidelberg, 2006, pp. 24–29. doi: 10.1007/11941439_114.