Extending Wireless Sensor Networks’ Lifetimes Using Deep Reinforcement Learning in a Software-Defined Network Architecture

Abstract

Routing packets in a Wireless Sensor Network (WSN) is a challenging task, according to the limited resources available on the nodes of these networks, especially their energy sources. The use of Machine Learning (ML) techniques in a Software-Defined Network (SDN) topology has shown a good potential toward solving such a complex task. However, existing techniques emphasize finding the shortest paths to deliver the packets, which can overload certain nodes in the network, depending on their positioning. In this study, a new method is proposed to extend the lifetime of the WSN by balancing the loading on the nodes, using a Deep Reinforcement Learning (DRL) approach. By emphasizing on the lifetime of the network, the proposed method has been able to discover and use alternative routes to deliver the packets, avoiding the use of nodes with low energy. Hence, the average number of hops the packets travel through has been increased but the time required for the first node to exhaust its energy has been significantly increased.

Keywords

• Software Defined Network
• Internet of Things
• Wireless Sensor Network
• Reinforcement Learning

References

1. R. Vijayashree and C. Suresh Ghana Dhas, "Energy efficient data collection with multiple mobile sink using artificial bee colony algorithm in large-scale WSN," Automatika, vol. 60, pp. 555-563, 2019.
2. M. Krishnan, S. Yun, and Y. M. Jung, "Dynamic clustering approach with ACO-based mobile sink for data collection in WSNs," Wireless Networks, vol. 25, pp. 4859-4871, 2019.
3. T. Wang, J. Zeng, Y. Lai, Y. Cai, H. Tian, Y. Chen, et al., "Data collection from WSNs to the cloud based on mobile Fog elements," Future Generation Computer Systems, vol. 105, pp. 864-872, 2020.
4. S. K. Singh and P. Kumar, "A comprehensive survey on trajectory schemes for data collection using mobile elements in WSNs," Journal of Ambient Intelligence and Humanized Computing, vol. 11, pp. 291-312, 2020.
5. M. Anand and T. Sasikala, "Efficient energy optimization in mobile ad hoc network (MANET) using better-quality AODV protocol," Cluster Computing, vol. 22, pp. 12681-12687, 2019.
6. P. Gupta, P. Goel, P. Varshney, and N. Tyagi, "Reliability factor based AODV protocol: prevention of black hole attack in MANET," in Smart Innovations in Communication and Computational Sciences, ed: Springer, 2019, pp. 271-279.
7. V. Sharma, B. Alam, and M. Doja, "An improvement in dsr routing protocol of manets using anfis," in Applications of Artificial Intelligence Techniques in Engineering, ed: Springer, 2019, pp. 569-576.
8. .Z. Al Aghbari, A. M. Khedr, W. Osamy, I. Arif, and D. P. Agrawal, "Routing in Wireless Sensor Networks Using Optimization Techniques: A Survey," Wireless Personal Communications, pp. 1-28, 2019.

9. V. K. Quy, N. T. Ban, V. H. Nam, D. M. Tuan, and N. D. Han, "Survey of recent routing metrics and protocols for mobile Ad-hoc networks," Journal of Communications, vol. 14, pp. 110-120, 2019.
10. K. L. Arega, G. Raga, and R. Bareto, "Survey on Performance Analysis of AODV, DSR and DSDV in MANET," 2020.
11. N. E. Majd, N. Ho, T. Nguyen, and J. Stolmeier, "Evaluation of parameters affecting the performance of routing protocols in mobile ad hoc networks (MANETs) with a focus on energy efficiency," in Future of information and communication conference, 2019, pp. 1210-1219.
12. S. K. Singh and J. Prakash, "Energy Efficiency and Load Balancing in MANET: A Survey," in 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), 2020, pp. 832-837.
13. Y. Zhao, Y. Li, X. Zhang, G. Geng, W. Zhang, and Y. Sun, "A Survey of Networking Applications Applying the Software Defined Networking Concept Based on Machine Learning," IEEE Access, vol. 7, pp. 95385-95405, 2019.
14. S. Sezer, S. Scott-Hayward, P. K. Chouhan, B. Fraser, D. Lake, J. Finnegan, et al., "Are we ready for SDN? Implementation challenges for software-defined networks," IEEE Communications Magazine, vol. 51, pp. 36-43, 2013.
15. F. Tang, Z. M. Fadlullah, B. Mao, and N. Kato, "An inelligent traffic load prediction-based adaptive channel assignment algorithm in SDN-IoT: A deep learning approach," IEEE Internet of Things Journal, vol. 5, pp. 5141-5154, 2018.
16. M. Ojo, D. Adami, and S. Giordano, "A SDN-IoT architecture with NFV implementation," in 2016 IEEE Globecom Workshops (GC Wkshps), 2016, pp. 1-6.
17. M. Baddeley, R. Nejabati, G. Oikonomou, M. Sooriyabandara, and D. Simeonidou, "Evolving SDN for low-power IoT networks," in 2018 4th IEEE Conference on Network Softwarization and Workshops (NetSoft), 2018, pp. 71-79.
18. .J. Wu, S. Luo, S. Wang, and H. Wang, "NLES: A novel lifetime extension scheme for safety-critical cyber-physical systems using SDN and NFV," IEEE Internet of Things Journal, vol. 6, pp. 2463-2475, 2018.
19. .L. Busoniu, R. Babuska, B. De Schutter, and D. Ernst, Reinforcement learning and dynamic programming using function approximators vol. 39: CRC press, 2010.
20. .M. L. Littman, "Markov games as a framework for multi-aent reinforcement learning," in Machine learning proceedings 1994, ed: Elsevier, 1994, pp. 157-163.
21. J. Schmidhuber, "Deep learning in neural networks: An overview," Neural networks, vol. 61, pp. 85-117, 2015.
22. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, et al., "Mastering the game of Go with deep neural networks and tree search," nature, vol. 529, pp. 484-489, 2016.
23. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, et al., "Human-level control through deep reinforcement learning," nature, vol. 518, pp. 529-533, 2015.
24. D. Zhang, F. R. Yu, and R. Yang, "A machine learning approach for software-defined vehicular ad hoc networks with trust management," in 2018 IEEE Global Communications Conference (GLOBECOM), 2018, pp. 1-6.
25. S.-C. Lin, I. F. Akyildiz, P. Wang, and M. Luo, "QoS-aware adaptive routing in multi-layer hierarchical software defined networks: A reinforcement learning approach," in 2016 IEEE International Conference on Services Computing (SCC), 2016, pp. 25-33.
26. G. Stampa, M. Arias, D. Sánchez-Charles, V. Muntés-Mulero, and A. Cabellos, "A deep-reinforcement learning approach for software-defined networking routing optimization," arXiv preprint arXiv:1709.07080, 2017.
27. M. A. Alsheikh, D. Niyato, S. Lin, H.-P. Tan, and Z. Han, "Mobile big data analytics using deep learning and apache spark," IEEE network, vol. 30, pp. 22-29, 2016.
28. B. Rhodes, J. Goerzen, A. Beaulne, and P. Membrey, Foundations of Python network programming: Springer, 2014.
29. .F. Chollet, "Keras: The python deep learning library," ascl, p. ascl: 1806.022, 2018.
30. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, et al., "Tensorflow: A system for large-scale machine learning," in 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16), 2016, pp. 265-283.