Sınır ağları üzerinden Fourier özniteliklerinin incelenmesi ve kablosuz örgü ağlarda akıllı bağlantı yönlendirmeye uygulanması

Abstract

Rastgele Fourier öznitelikleri (RFÖ), doğrusal olmayan sınıflandırma için özellikle büyük ölçekli veri koşullarında en önemli araçlardan biridir. Bununla birlikte, RFÖ'nün orijinal önerisi dikkate alındığında, Fourier öznitelikleri belirli bir dağılımdan rastgele seçilir ve eniyilenmeden kullanılır. Bu yazıda, Fourier özniteliklerini tek gizli katmanlı bir ileri beslemeli sinir ağı (TKİS) aracılığıyla incelemekte ve bu öznitelikleri (rastgele seçim yerine) optimize etmekte, yani öğrenmekteyiz. Öğrenilen Fourier öznitelikleri radyal taban fonksiyonundan (rtf çekirdeği) üretildikten sonra TKİS'nin gizli katmanında gerçeklenir ve sonra takip eden çıktı katmanında kullanılır. Biyoinformatik gibi çeşitli alanlardan 10 farklı sınıflandırma veri kümeleri ile kapsamlı deneyler sunmaktayız. Fourier öznitelik öğrenmesinin, rtf çekirdek uzayında perseptron uygulama veya ileri yönlü fırsatçı öznitelikleri seçme stratejileri gibi rakip tekniklere göre oldukça üstün olduğu gözlemlenmiştir. Öte yandan, Fourier öznitelik öğrenmesi, DVM (rtf çekirdekli destek vektör makineleri) ile karşılaştırılabilir bir performans sergilerken, önemli hesaplama avantajlarını marjin büyütmesini kullanmadan dahi sağlayabilmektedir. Ayrıca, TKİS'yi kablosuz örgü ağlarında test ettiğimizde akıllı yönlendirme açısından umut vaat ettiğini gözlemlemekteyiz.

Keywords

• Fourier öznitelikleri
• Sinir ağları
• Tek gizli katman
• Sınıflandırma
• Çekirdek
• Bağlantı yönlendirme

Investigation of Fourier features via neural networks and an application to smart steering in wireless mesh networks

Abstract

Random Fourier features (RFF) provide one of the most prominent means for nonlinear classification in especially large scale data settings. However, considering the original proposal of RFF, Fourier features are randomly drawn from a certain distribution and used unoptimized. In this paper, we investigate Fourier features via a single hidden layer feedforward neural network (SLFN) and optimize, i.e., learn, those features (instead of drawing randomly). The learned Fourier features are deduced from the radial basis function (rbf kernel), and implemented in the hidden layer of the SLFN which is followed by the output layer. We present extensive experiments with 10 different classification datasets from various fields, e.g., bioinformatics. The learning of Fourier features is observed to be highly superior over the competing techniques such as perceptron in the rbf kernel space or a greedy forward feature selection strategy. On the other hand, the Fourier feature learning performs comparably with SVM (support vector machines with rbf kernel) while providing substantial computational benefits, and this is even without using the max margin regularization. Moreover, when tested in wireless mesh networks, the SLFN delivers promising smart steering capabilities.

Keywords

• Fourier features
• Neural networks
• Single hidden layer
• Classification
• Kernel
• Steering

References

1. [1] Hofmann T, Schölkopf B, Smola AJ. “Kernel methods in machine learning”. The Annals of Statistics, 36(3), 1171-1220, 2008.
2. [2] Cortes C, Vapnik V. “Support-vector networks”. Machine Learning, 20(3), 273-297, 1995.
3. [3] Scholkopf B, Sung KK, Burges CJ, Girosi F, Niyogi P, Poggio T, Vapnik V. “Comparing support vector machines with gaussian kernels to RBF classifiers”. IEEE Transactions on Signal Processing, 45(11), 2758-2765, 1997.
4. [4] Jaakkola TS, Haussler D. “Probabilistic kernel regression models”. Artificial Intelligence and Statistics, Ft. Lauderdale, FL, USA, 3-6 January 1999.
5. [5] Kerpicci M, Ozkan H, Kozat SS. “Online anomaly detection with bandwidth optimized hierarchical kernel density estimators”. IEEE Transactions on Neural Networks and Learning Systems, 32(9), 4253-4266, 2020.
6. [6] Lanckriet GR, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI. “Learning the kernel matrix with semidefinite programming”. Journal of Machine Learning Research, 5(1), 27-72, 2004.
7. [7] Rahimi A, Recht B. “Random features for large-scale kernel machines”. Neural Information Processing Systems, Vancouver, B.C., Canada, 3-6 December 2007.
8. [8] Kuskonmaz B, Ozkan H, Gurbuz O. “Machine learningbased smart steering for wireless mesh networks”. Ad Hoc Networks, 88(1), 98-111, 2019.

9. [9] Lu J, Hoi SC, Wang J, Zhao P, Liu ZY. “Large scale online kernel learning”. Journal of Machine Learning Research, 17(1), 1613-1655, 2016.
10. [10] Băzăvan EG, Li F, Sminchisescu C. “Fourier kernel learning”. European Conference on Computer Vision, Berlin, Germany, 7-13 October 2012.
11. [11] Nguyen T, Le T, Bui H, Phung D. “Large-scale online kernel learning with random feature reparameterization”. International Joint Conferences on Artificial Intelligence, Melbourne, Australia, 19-25 August 2017.
12. [12] Oliva JB, Dubey A, Wilson AG, Póczos B, Schneider J, Xing EP. “Bayesian nonparametric kernel-learning”. Artificial Intelligence and Statistics, Cadiz, Spain, 9-11 May 2016.
13. [13] Li CL, Chang WC, Mroueh Y, Yang Y, Póczos B. “Implicit kernel learning”. Artificial Intelligence and Statistics, Naha, Okinawa, Japan, 16-18 April 2019.
14. [14] Wang A, Law L, Miscouridou X, Mider M, Ip S. “Kernel learning via random fourier representations”. Warwick Statistics Programme, Oxford University, Oxford, England, Reports and Presentations, 2016.
15. [15] Xie J, Liu F, Wang K, Huang X. “Deep kernel learning via random fourier features”. arXiv Preprint, 2019. https://doi.org/10.48550/arXiv.1910.02660.
16. [16] Xue H, Wu ZF, Sun WX. “Deep spectral kernel learning”. International Joint Conferences on Artificial Intelligence, Macao, China, 10-16 August 2019.
17. [17] Can B, Ozkan H. “A neural network approach for online nonlinear neyman-pearson classification”. IEEE Access, 8(1), 210234-210250, 2020.
18. [18] Porikli F, Ozkan H. “Data driven frequency mapping for computationally scalable object detection”. IEEE International Conference on Advanced Video and Signal Based Surveillance, Klagenfurt, Austria, 30 August–2 September 2011.
19. [19] Alpaydin E. Introduction to Machine Learning. 4th ed. Cambridge, MA, USA, MIT Press, 2020.
20. [20] Muller KR, Mika S, Ratsch G, Tsuda K, Scholkopf B. “An Introduction to kernel-based learning algorithms”. IEEE Transactions on Neural Networks, 12(2), 181-201, 2001.
21. [21] Rudin W. Fourier Analysis on Groups. New York, USA, Wiley, 2011.
22. [22] McClellan JH, Schafer RW, Yoder MA. Signal Processing First. 1st ed. Upper Saddle River, NJ, USA, Pearson, 2003.
23. [23] Burges CJ. “A tutorial on support vector machines for pattern recognition”. Data Mining and Knowledge Discovery, 2(2), 121-167, 1998.
24. [24] Wright S. “Coordinate descent algorithms”. Mathematical Programming, 151(1), 3-34, 2015.
25. [25] Chang CC, Lin CJ. “LIBSVM: A library for support vector machines”. ACM Transactions on Intelligent Systems and Technology, 2(3), 1-27, 2011.