Late Fusion Based Convolutional Network Model in Detection of Vital Signals with Radar Technology

Year 2023, Volume: 15 Issue: 1, 248 - 255, 31.01.2023

Umut Özkaya

Abstract

In this study, a method based on Convolutional Neural Networks (CNN) and fusion technology was proposed for the classification of vital signals. In order to obtain more information from 1-D radar signals, 2-D data were obtained with the spectrogram technique. An automated classification framework has been implemented by using pre-trained Google Net, VGG-16 and ResNet-50 models. The performance in the test data is increased by applying late fusion process to the highest performing VGG-16 and GoogleNet CNN structures. The performance of the proposed method is 92.54% Accuracy (ACC), 92.41% Sensitivity (SEN), 97.18% Specificity (SPE), 93.54% Precision (PRE), 92.66% F1-Score, and 90.25% Matthews Correlation Constant (MCC). Thanks to the proposed method, radar technology, which is one of the non-destructive detection technologies, comes to the forefront compared to wearable technologies

Keywords

Radar, Vital Sign, Deep Learning, Convolutional Neural Network, Late Fusion

Radar Teknolojisi ile Hayati Sinyallerin Tespitinde Geç Füzyon Tabanlı Evrişimsel Sinir Ağı Modeli

Abstract

Bu çalışmada hayati sinyallerin sınıflandırılması için Evrişimsel Sinir Ağları (ESA) ve füzyon teknolojine dayalı bir yöntem önerildi. Tek boyutlu radar sinyallerinden daha fazla bilgi edinmek amacıyla spektrogram tekniği ile 2 boyutlu veriler elde edildi. GoogleNet, VGG-16 ve ResNet-50 ön eğitimli ESA kullanılarak otomatik bir sınıflandırma çerçevesi uygulanmıştır. En yüksek performansa sahip VGG-16 ve GoogleNet ESA yapılarına geç füzyon işlemi uygulanarak test verilerindeki performans artırılmıştır. Önerilen yöntemin performans 92.54% Doğruluk (DOĞ), 92.41% Duyarlılık (DUY), 97.18% Özgüllük (ÖZG), 93.54% Hassasiyet (HAS), 92.66% F1-Skoru ve 90.25% Matthews Korelasyon Sabiti (MKS)’dir. Önerilen yöntem sayesinde tahribatsız algılama teknolojilerinden biri olan radar teknolojisi giyilebilir teknolojilere göre daha ön plana çıkmaktadır.

Keywords

Radar, Vital Sign, Deep Learning, Convolutional Neural Network, Late Fusion

References

• Baldoumas G., Peschos D., Tatsis G., Christofilakis V., Chronopoulos S. K., Kostarakis P., Varotsos P. A., Sarlis N. V., Skordas E. S., Bechlioulis A., Michalis L. K., Naka K. K. (2021). Remote sensing natural time analysis of heartbeat data by means of a portable photoplethysmography device, International Journal of Remote Sensing, 42 (6): 2292-2302.
• Cardillo E., Li C., Caddemi A. (2021). Vital sign detection and radar self-motion cancellation through clutter identification, IEEE Transactions on Microwave Theory and Techniques, 69 (3): 1932-1942.
• Chang H., Lin C., Lin Y., Chung W., Lee T. (2020). DL-Aided NOMP: a deep learning-based vital sign estimating scheme using FMCW radar, IEEE 91st Vehicular Technology Conference (VTC2020-Spring), Antwerp, Belgium, 1-7. Erdoğan S., Yılmaz Ş., Öncü A. (2019) Microwave noncontact vital sign measurements for medical applications, IEEE International Symposium on Medical Measurements and Applications (MeMeA), Istanbul, Turkey, 1-5.
• Fulton L.V., Dolezel D., Harrop J., Yan Y., Fulton C.P. (2019). Classification of Alzheimer’s disease with and without imagery using gradient boosted machines and ResNet-50, Brain sciences, 9 (9): 212.
• Giv H.H. (2013). Directional short-time Fourier transform, Journal of Mathematical Analysis and Applications, 399 (1): 100-107.
• Jaderberg, M., Simonyan, K., Zisserman, A. (2015). Spatial transformer networks, Advances in neural information processing systems, 28: 2017-2025.
• Kagawa, M., Ueki, K., Tojima, H., Matsui, T. (2013). Noncontact screening system with two microwave radars for the diagnosis of sleep apnea-hypopnea syndrome, In Proceedings of the 35th Annual International Conference of the IEEE EMBS, Osaka, Japan, 3–7 July, 2052–2055.
• Kagawa, M., Yoshida, Y., Kubota, M., Kurita, A., Matsui, T. (2011). An overnight vital signs monitoring system for elderly people using dual microwave radars, In Proceedings of the Asia-Pacific Microwave Conference, Melbourne, Australia, 5–8 December, 590–593.
• Lai W. C. (2020). Design of receiver frontend with deep neural network for doppler radar heart rate detection, IEEE 5th International Conference on Integrated Circuits and Microsystems (ICICM), Nanjing, China, 121-124.
• Leung, R.S.T., Bradley, T.D. (2001). Sleep apnea and cardiovascular disease, Am. J. Respir. Crit. Care Med., 164: 2147–2165.
• Lie, D.Y.C., Ichapurapu, R., Jain, S., Lopez, J., Banister, R.E., Nguyen, T., Griswold, J. A. (2011). 2.4 GHz Non-Contact biosensor system for continuous monitoring of vital-signs. In Telemedicine Techniques and Applications; Graschew, G., Ed.; InTech: Rijeka, Croatia, 211–238.
• Nieto, F.J., Peppard, P.E., Young, T., Finn, L., Hla, K.M., Farré, R. (2012). Sleep-disordered breathing and cancer mortality: results from the wisconsin sleep cohort study, Am. J. Respir. Crit. Care Med., 186: 190–194.
• Quaiyum F., Tran N., Phan T., Theilmann P., Fathy A. E., Kilic O. (2018). Electromagnetic modeling of vital sign detection and human motion sensing validated by noncontact radar measurements, IEEE Journal of Electromagnetics, RF and Microwaves in Medicine and Biology, 2 (1): 40-47.
• Öztürk Ş., (2020). Stacked auto-encoder based tagging with deep features for content-based medical image retrieval, Expert Systems with Applications, 161 (5).
• Saluja J. J., Lin J., Casanova J. (2018). A supervised learning approach for real time vital sign radar harmonics cancellation, IEEE International Microwave Biomedical Conference (IMBioC), Philadelphia, PA, USA, 67-69.
• Saluja J., Casanova J., Lin J. (2020). A supervised machine learning algorithm for heart-rate detection using doppler motion-sensing radar, IEEE Journal of Electromagnetics, RF and Microwaves in Medicine and Biology, 4 (1): 45-51.
• Savage, H. O., Khushaba, R. N., Zaffaroni, A., Colefax, M., Farrugia, S., Schindhelm, K., Teschler, H., Weinreich, G., Grueger, H., Neddermann, M., Heneghan, C., Simonds, A., Cowie, M. R. (2016). Development and validation of a novel non‐contact monitor of nocturnal respiration for identifying sleep‐disordered breathing in patients with heart failure, ESC Heart Failure, 3: 212– 219.
• Schellenberger S., Shi K., Steigleder T. (2020). A dataset of clinically recorded radar vital signs with synchronised reference sensor signals, Sci Data, 7 (291).
• Seicean, S., Strohl, K.P., Seicean, A., Gibby, C., Marwick, T.H. (2013). Sleep disordered breathing as a risk of cardiac events in subjects with diabetes mellitus and normal exercise echocardiographic findings, Am. J. Cardiol., 111: 1214–1220.
• Sindi, H., Nour, M., Rawa, M., Öztürk, Ş., Polat, K. (2021). A novel hybrid deep learning approach including combination of 1D power signals and 2D signal images for power quality disturbance classification, Expert Systems with Applications, 2021, 174 (114785).
• Slapničar, G., Wang W., Luštrek M. (2021). Classification of hemodynamics scenarios from a public radar dataset using a deep learning approach, Sensors, 21 (5): 1836.
• Tran, V. P., Al-Jumaily, A. A. (2015). Non-contact dual pulse doppler system based real-time relative demodulation and respiratory & heart rates estimations for chronic heart failure patients, Procedia Computer Science, 76: 47-52.
• Wang X., Yang C., Mao S. (2017). PhaseBeat: Exploiting CSI phase data for vital sign monitoring with commodity wifi devices, IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 1230-1239.
• Young, T., Palta, M., Dempsey, J., Skatrud, J. Webber, S. Badr, S. (1993). The occurrence of sleep-disordered breathing among middle-aged adults, N. Engl. J. Med., 1 (328): 1230–1235.
• Zaffaroni, A., De Chazal, P., Heneghan, C., Boyle, P., Mppm, P. R., McNicholas, W. T. (2009). SleepMinder: an innovative contact-free device for the estimation of the apnoea-hypopnoea index, Annual international conference of the IEEE engineering in medicine and biology society, 7091-9094.
• Zhang X., Zou J., He K., Su J. (2015). Accelerating very deep convolutional networks for classification and detection, IEEE transactions on pattern analysis and machine intelligence, 38 (10): 1943-1955.