Research Article
BibTex RIS Cite

Comparison of manual lymphatic drainage massage and negative pressure massage therapy efficacy in lymphedema patients: a randomized controlled study

Year 2023, Volume: 9 Issue: 6 - November 2023, 1474 - 1482, 04.11.2023
https://doi.org/10.18621/eurj.1354942

Abstract

Objectives: Lower extremity lymphedema due to secondary causes is a lifelong complication that can be encountered. Its treatment is essential, because it has significant impact on quality of life and daily living activities related to lower extremity involvement. This research aims to compare the effects of Manual Lymphatic Drainage Massage (MLD) and Negative Pressure Massage Therapy (NPMT) treatments in order to provide maximum benefit to patients.

Methods: This prospective, randomized study included 30 patients with lower extremity lymphedema due to secondary causes. Patients, randomized using computer software, were divided into two groups. The first group (n =15) received 45 minutes, 15 sessions of MLD, while the second group (n =15) received 45 minutes of 15 sessions of NPMT using the LymphaTouch device. Compression bandaging was applied to both groups and self-drainage training was given to all patients. The circumference of the extremity at 6 reference points were measured and their pain and discomfort assessed by the Visual Analogue Scale (VAS) were recorded before and after treatment. Changes within the groups and between the groups were compared using the SPSS statistical program.

Results: Statistically significant improvement was observed in all parameters in both treatment groups. The decrease in VAS pain and VAS discomfort scores (p < 0.05 and p < 0.01; respectively), circumference measurement of the extremity (p < 0.01) was statistically greater in the NPMT group compared to the MLD group.

Conclusions: In conclusion, NPMT appears to be a beneficial non-invasive treatment method for reducing extremity volumes and decreasing subjective pain and discomfort in lymphedema patients.

References

  • 1. Korpan MI, Crevenna R, Fialka-Moser V. Lymphedema: a therapeutic approach in the treatment and rehabilitation of cancer patients. Am J Phys Med Rehabil 2011;90(5 Suppl 1):S69-75.
  • 2. Miller AJ, Bruna J, Beninson J. A universally applicable clinical classification of lymphedema. Angiology 1999;50:189-92.
  • 3. Cheville AL, McGarvey CL, Petrek JA, Russo SA, Thiadens SR, Taylor ME. The grading of lymphedema in oncology clinical trials. Semin Radiat Oncol 2003;13:214-25.
  • 4. Paskett ED, Dean JA, Oliveri JM, Harrop JP. Cancer-related lymphedema risk factors, diagnosis, treatment, and impact: a review. J Clin Oncol 2012;30:3726-33.
  • 5. International Society of Lymphology. The diagnosis and treatment of peripheral lymphedema. 2009 Concensus Document of the International Society of Lymphology. Lymphology 2009;42:51-60.
  • 6. Badger C, Preston N, Seers K, Mortimer P. Benzo-pyrones for reducing and controlling lymphoedema of the limbs. Cochrane Database Syst Rev 2004;2004:CD003140.
  • 7. Maclellan RA, Greene AK. Lymphedema. Semin Pediatr Surg 2014;23:191-7.
  • 8. Kesiktaş FN. Kanser rehabilitasyonunda fiziksel ajanlar ve uğraşı tedavisi. Akyüz G, Giray E, eds. Kanser Rehabilitasyonu. 1st. edition. Ankara, Türkiye Klinikleri; 2022: pp. 66-73.
  • 9. de Sire A, Losco L, Lippi L, Spadoni D, Kaciulyte J, Sert G, et al. Surgical treatment and rehabilitation strategies for upper and lower extremity lymphedema: a comprehensive review. Medicina (Kaunas) 2022;58:954.
  • 10. Fukushima T, Tsuji T, Sano Y, Miyata C, Kamisako M, Hohri H, et al. Immediate effects of active exercise with compression therapy on lower-limb lymphedema. Support Care Cancer 2017;25:2603-10.
  • 11. Chang CJ, Cormier JN. Lymphedema interventions: exercise, surgery, and compression devices. Semin Oncol Nurs 2013;29:28-40.
  • 12. Strössenreuther RHK. Degongective kinesiotherapy, respiratory therapy, physio-therapy and other physical therapy tecniques. In: Földi M, Földi E, eds. Földi’s Textbook of Lymphology for Physicians and Lymphedema Therapists. 2nd edition. Munich, Mosby-Elsevier; 2006: pp. 547-61.
  • 13. Lasinski BB. Complete decongestive therapy for treatment of lymphedema. Semin Oncol Nurs 2013;29:20-7.
  • 14. Devoogdt N, Van Kampen M, Geraerts I, Coremans T, Christiaens MR. Different physical treatment modalities for lymphoedema developing after axillary lymph node dissection for breast cancer: a review. Eur J Obstet Gynecol Reprod Biol 2010;149:3-9.
  • 15. Bjork R, Ehmann S. S.T.R.I.D.E. Professional guide to compression garment selection for the lower extremity. J Wound Care 2019;28(Sup6a):1-44.
  • 16. Huskisson EC. Measurement of pain. Lancet 1974;2:1127-31.
  • 17. Deltombe T, Jamart J, Recloux S, Legrand C, Vandenbroeck N, Theys S, et al. Reliability and limits of agreement of circumferential, water displacement, and optoelectronic volumetry in the measurement of upper limb lymphedema. Lymphology 2007;40:26-34.
  • 18. Lampinen R, Lee JQ, Leano J, Miaskowski C, Mastick J, Brinker L, et al. Treatment of breast cancer-related lymphedema using negative pressure massage: a pilot randomized controlled trial. Arch Phys Med Rehabil 2021;102:1465-72.e2.
  • 19. Kasseroller RG. The Vodder School: the Vodder method. Cancer 1998;83(12 Suppl American):2840-2.
  • 20. Corum M, Basoglu C, Korkmaz MD, Yildirim MA, Ones K. Effectiveness of combined complex decongestive therapy and resistance exercises in the treatment of lymphedema associated with breast cancer and the effect of pain on treatment response. Lymphat Res Biol 2021;19:383-90.
  • 21. Boris M, Weindorf S, Lasinkski S. Persistence of lymphedema reduction after noninvasive complex lymphedema therapy. Oncology (Williston Park) 1997;11:99-109.
  • 22. Mayrovitz HN. The standard of care for lymphedema: current concepts and physiological considerations. Lymphat Res Biol 2009;7:101-8.
  • 23. McCallin M, Johnston J, Bassett S. How effective are physiotherapy techniques to treat established secondary lymphoedema following surgery for cancer? A critical analysis of the literature. New Zealand J Physiother 2005;33:101-12.
  • 24. Vuorinen VP, Iivarinen J, Jurvelin J, Airaksinen O. Lymphatic therapy using negative pressure. A clinical study with the LymphaTouch device. Research Report #5320003/221, Finnish Funding Agency for Technology and Innovation, August 2013.
  • 25. Saul D, Fischer AC, Lehmann W, Dresing K. Reduction of postoperative swelling with a negative pressure treatment-A prospective study. J Orthop Surg (Hong Kong) 2020;28:2309499020929166.
  • 26. Mihara M, Hara H, Araki J, Kikuchi K, Narushima M, Yamamoto T, et al. Indocyanine green (ICG) lymphography is superior to lymphoscintigraphy for diagnostic imaging of early lymphedema of the upper limbs. PLoS One 2012;7:e38182.
  • 27. Beesley VL, Rowlands IJ, Hayes SC, Janda M, O'Rourke P, Marquart L, et al; Australian National Endometrial Cancer Study Group. Incidence, risk factors and estimates of a woman's risk of developing secondary lower limb lymphedema and lymphedema-specific supportive care needs in women treated for endometrial cancer. Gynecol Oncol 2015;136:87-93.
  • 28. Deura I, Shimada M, Hirashita K, Sugimura M, Sato S, Sato S, et al. Incidence and risk factors for lower limb lymphedema after gynecologic cancer surgery with initiation of periodic complex decongestive physiotherapy. Int J Clin Oncol 2015;20:556-60.
  • 29. Fu MR, Axelrod D, Guth AA, Fletcher J, Qiu JM, Scagliola J, et al. Patterns of obesity and lymph fluid level during the first year of breast cancer treatment: a prospective study. J Pers Med 2015;5:326-40.
  • 30. Armer JM, Stewart BR. A comparison of four diagnostic criteria for lymphedema in a post-breast cancer population. Lymphat Res Biol 2005;3:208-17.
  • 31. Bérard A, Kurz X, Zuccarelli F, Abenhaim L; VEINES Study Group. Venous insufficiency epidemiologic and economic study. Validity of the Leg-O-Meter, an instrument to measure leg circumference. Angiology 2002;53:21-8.
  • 32. Karges JR, Mark BE, Stikeleather SJ, Worrell TW. Concurrent validity of upper-extremity volume estimates: comparison of calculated volume derived from girth measurements and water displacement volume. Phys Ther 2003;83:134-45.
  • 33. Devoogdt N, Christiaens MR, Geraerts I, Truijen S, Smeets A, Leunen K, et al. Effect of manual lymph drainage in addition to guidelines and exercise therapy on arm lymphoedema related to breast cancer: randomised controlled trial. BMJ 2011;343:d5326.
  • 34. Reul-Hirche H. Manual lymph drainage when added to advice and exercise may not be effective in preventing lymphoedema after surgery for breast cancer. J Physiother 2011;57:258.
  • 35. Sen EI, Arman S, Zure M, Yavuz H, Sindel D, Oral A. Manual lymphatic drainage may not have an additional effect on the intensive phase of breast cancer-related lymphedema: a randomized controlled trial. Lymphat Res Biol 2021;19:141-50.
  • 36. Lin Y, Yang Y, Zhang X, Li W, Li H, Mu D. Manual lymphatic drainage for breast cancer-related lymphedema: a systematic review and meta-analysis of randomized controlled trials. Clin Breast Cancer 2022;22:e664-73.
  • 37. Huang TW, Tseng SH, Lin CC, Bai CH, Chen CS, Hung CS, et al. Effects of manual lymphatic drainage on breast cancer-related lymphedema: a systematic review and meta-analysis of randomized controlled trials. World J Surg Oncol 2013;11:15.
  • 38. Sitzia J, Sobrido L, Harlow W. Manual lymphatic drainage compared with simple lymphatic drainage in the treatment of post-mastectomy lymphoedema. Physiotherapy 2002;88:99-107.
  • 39. Qiao J, Yang LN, Kong YH, Huang X, Li Y, Bai DQ. Effect of manual lymphatic drainage on breast cancer-related postmastectomy lymphedema: a meta-analysis of randomized controlled trials. Cancer Nurs 2023;46:159-66.
  • 40. Borman P, Yaman A, Denizli M, Yüzer A, Terzioğlu F. Complex decongestive therapy in a patient with poliomyelitis, bilateral lymphoedema and a deep wound: a case study. J Wound Care 2022;31:792-8.
  • 41. Şahinoğlu E, Ergin G, Karadibak D. The efficacy of change in limb volume on functional mobility, health-related quality of life, social appearance anxiety, and depression in patients with lower extremity lymphedema. Phlebology 2022;37:200-5.
  • 42. Ali K, Dibbs RP, Dougherty C, Iacobas I, Maricevich RS. Treatment outcomes of manual lymphatic drainage in pediatric lymphedema. Ann Vasc Surg 2022;78:263-71.
  • 43. Liu F, Liu NF, Wang L, Chen J, Han L, Yu Z, et al. Treatment of secondary lower limb lymphedema after gynecologic cancer with complex decongestive therapy. Lymphology 2021;54:122-32.
  • 44. Pereira de Godoy JM, Pereira de Godoy LM, Pereira de Godoy HJ, Guerreiro Godoy MF. Reduction of arm lymphedema using manual lymphatic therapy (Godoy Method). Cureus 2022;14:e28374.
  • 45. McNeely ML, Magee DJ, Lees AW, Bagnall KM, Haykowsky M, Hanson J. The addition of manual lymph drainage to compression therapy for breast cancer related lymphedema: a randomized controlled trial. Breast Cancer Res Treat 2004;86:95-106.
  • 46. Cavezzi A, Urso SU, Paccasassi S, Mosti G, Campana F, Colucci R. Bioimpedance spectroscopy and volumetry in the immediate/short-term monitoring of intensive complex decongestive treatment of lymphedema. Phlebology 2020;35:715-23.
  • 47. Campisi CC, Ryn M, Campisi CS, Di Summa P, Boccardo F, Campisi C. Intermittent negative pressure therapy in the combined treatment of peripheral lymphedema. Lymphology 2015;48:197-204.
  • 48. Weber M, Rahn J, Hackl M, Leschinger T, Dresing K, Müller LP, et al. Postoperative swelling after elbow surgery: influence of a negative pressure application in comparison to manual lymphatic drainage-a randomized controlled trial. Arch Orthop Trauma Surg 2023;143:6243-9.
Year 2023, Volume: 9 Issue: 6 - November 2023, 1474 - 1482, 04.11.2023
https://doi.org/10.18621/eurj.1354942

Abstract

References

  • 1. Korpan MI, Crevenna R, Fialka-Moser V. Lymphedema: a therapeutic approach in the treatment and rehabilitation of cancer patients. Am J Phys Med Rehabil 2011;90(5 Suppl 1):S69-75.
  • 2. Miller AJ, Bruna J, Beninson J. A universally applicable clinical classification of lymphedema. Angiology 1999;50:189-92.
  • 3. Cheville AL, McGarvey CL, Petrek JA, Russo SA, Thiadens SR, Taylor ME. The grading of lymphedema in oncology clinical trials. Semin Radiat Oncol 2003;13:214-25.
  • 4. Paskett ED, Dean JA, Oliveri JM, Harrop JP. Cancer-related lymphedema risk factors, diagnosis, treatment, and impact: a review. J Clin Oncol 2012;30:3726-33.
  • 5. International Society of Lymphology. The diagnosis and treatment of peripheral lymphedema. 2009 Concensus Document of the International Society of Lymphology. Lymphology 2009;42:51-60.
  • 6. Badger C, Preston N, Seers K, Mortimer P. Benzo-pyrones for reducing and controlling lymphoedema of the limbs. Cochrane Database Syst Rev 2004;2004:CD003140.
  • 7. Maclellan RA, Greene AK. Lymphedema. Semin Pediatr Surg 2014;23:191-7.
  • 8. Kesiktaş FN. Kanser rehabilitasyonunda fiziksel ajanlar ve uğraşı tedavisi. Akyüz G, Giray E, eds. Kanser Rehabilitasyonu. 1st. edition. Ankara, Türkiye Klinikleri; 2022: pp. 66-73.
  • 9. de Sire A, Losco L, Lippi L, Spadoni D, Kaciulyte J, Sert G, et al. Surgical treatment and rehabilitation strategies for upper and lower extremity lymphedema: a comprehensive review. Medicina (Kaunas) 2022;58:954.
  • 10. Fukushima T, Tsuji T, Sano Y, Miyata C, Kamisako M, Hohri H, et al. Immediate effects of active exercise with compression therapy on lower-limb lymphedema. Support Care Cancer 2017;25:2603-10.
  • 11. Chang CJ, Cormier JN. Lymphedema interventions: exercise, surgery, and compression devices. Semin Oncol Nurs 2013;29:28-40.
  • 12. Strössenreuther RHK. Degongective kinesiotherapy, respiratory therapy, physio-therapy and other physical therapy tecniques. In: Földi M, Földi E, eds. Földi’s Textbook of Lymphology for Physicians and Lymphedema Therapists. 2nd edition. Munich, Mosby-Elsevier; 2006: pp. 547-61.
  • 13. Lasinski BB. Complete decongestive therapy for treatment of lymphedema. Semin Oncol Nurs 2013;29:20-7.
  • 14. Devoogdt N, Van Kampen M, Geraerts I, Coremans T, Christiaens MR. Different physical treatment modalities for lymphoedema developing after axillary lymph node dissection for breast cancer: a review. Eur J Obstet Gynecol Reprod Biol 2010;149:3-9.
  • 15. Bjork R, Ehmann S. S.T.R.I.D.E. Professional guide to compression garment selection for the lower extremity. J Wound Care 2019;28(Sup6a):1-44.
  • 16. Huskisson EC. Measurement of pain. Lancet 1974;2:1127-31.
  • 17. Deltombe T, Jamart J, Recloux S, Legrand C, Vandenbroeck N, Theys S, et al. Reliability and limits of agreement of circumferential, water displacement, and optoelectronic volumetry in the measurement of upper limb lymphedema. Lymphology 2007;40:26-34.
  • 18. Lampinen R, Lee JQ, Leano J, Miaskowski C, Mastick J, Brinker L, et al. Treatment of breast cancer-related lymphedema using negative pressure massage: a pilot randomized controlled trial. Arch Phys Med Rehabil 2021;102:1465-72.e2.
  • 19. Kasseroller RG. The Vodder School: the Vodder method. Cancer 1998;83(12 Suppl American):2840-2.
  • 20. Corum M, Basoglu C, Korkmaz MD, Yildirim MA, Ones K. Effectiveness of combined complex decongestive therapy and resistance exercises in the treatment of lymphedema associated with breast cancer and the effect of pain on treatment response. Lymphat Res Biol 2021;19:383-90.
  • 21. Boris M, Weindorf S, Lasinkski S. Persistence of lymphedema reduction after noninvasive complex lymphedema therapy. Oncology (Williston Park) 1997;11:99-109.
  • 22. Mayrovitz HN. The standard of care for lymphedema: current concepts and physiological considerations. Lymphat Res Biol 2009;7:101-8.
  • 23. McCallin M, Johnston J, Bassett S. How effective are physiotherapy techniques to treat established secondary lymphoedema following surgery for cancer? A critical analysis of the literature. New Zealand J Physiother 2005;33:101-12.
  • 24. Vuorinen VP, Iivarinen J, Jurvelin J, Airaksinen O. Lymphatic therapy using negative pressure. A clinical study with the LymphaTouch device. Research Report #5320003/221, Finnish Funding Agency for Technology and Innovation, August 2013.
  • 25. Saul D, Fischer AC, Lehmann W, Dresing K. Reduction of postoperative swelling with a negative pressure treatment-A prospective study. J Orthop Surg (Hong Kong) 2020;28:2309499020929166.
  • 26. Mihara M, Hara H, Araki J, Kikuchi K, Narushima M, Yamamoto T, et al. Indocyanine green (ICG) lymphography is superior to lymphoscintigraphy for diagnostic imaging of early lymphedema of the upper limbs. PLoS One 2012;7:e38182.
  • 27. Beesley VL, Rowlands IJ, Hayes SC, Janda M, O'Rourke P, Marquart L, et al; Australian National Endometrial Cancer Study Group. Incidence, risk factors and estimates of a woman's risk of developing secondary lower limb lymphedema and lymphedema-specific supportive care needs in women treated for endometrial cancer. Gynecol Oncol 2015;136:87-93.
  • 28. Deura I, Shimada M, Hirashita K, Sugimura M, Sato S, Sato S, et al. Incidence and risk factors for lower limb lymphedema after gynecologic cancer surgery with initiation of periodic complex decongestive physiotherapy. Int J Clin Oncol 2015;20:556-60.
  • 29. Fu MR, Axelrod D, Guth AA, Fletcher J, Qiu JM, Scagliola J, et al. Patterns of obesity and lymph fluid level during the first year of breast cancer treatment: a prospective study. J Pers Med 2015;5:326-40.
  • 30. Armer JM, Stewart BR. A comparison of four diagnostic criteria for lymphedema in a post-breast cancer population. Lymphat Res Biol 2005;3:208-17.
  • 31. Bérard A, Kurz X, Zuccarelli F, Abenhaim L; VEINES Study Group. Venous insufficiency epidemiologic and economic study. Validity of the Leg-O-Meter, an instrument to measure leg circumference. Angiology 2002;53:21-8.
  • 32. Karges JR, Mark BE, Stikeleather SJ, Worrell TW. Concurrent validity of upper-extremity volume estimates: comparison of calculated volume derived from girth measurements and water displacement volume. Phys Ther 2003;83:134-45.
  • 33. Devoogdt N, Christiaens MR, Geraerts I, Truijen S, Smeets A, Leunen K, et al. Effect of manual lymph drainage in addition to guidelines and exercise therapy on arm lymphoedema related to breast cancer: randomised controlled trial. BMJ 2011;343:d5326.
  • 34. Reul-Hirche H. Manual lymph drainage when added to advice and exercise may not be effective in preventing lymphoedema after surgery for breast cancer. J Physiother 2011;57:258.
  • 35. Sen EI, Arman S, Zure M, Yavuz H, Sindel D, Oral A. Manual lymphatic drainage may not have an additional effect on the intensive phase of breast cancer-related lymphedema: a randomized controlled trial. Lymphat Res Biol 2021;19:141-50.
  • 36. Lin Y, Yang Y, Zhang X, Li W, Li H, Mu D. Manual lymphatic drainage for breast cancer-related lymphedema: a systematic review and meta-analysis of randomized controlled trials. Clin Breast Cancer 2022;22:e664-73.
  • 37. Huang TW, Tseng SH, Lin CC, Bai CH, Chen CS, Hung CS, et al. Effects of manual lymphatic drainage on breast cancer-related lymphedema: a systematic review and meta-analysis of randomized controlled trials. World J Surg Oncol 2013;11:15.
  • 38. Sitzia J, Sobrido L, Harlow W. Manual lymphatic drainage compared with simple lymphatic drainage in the treatment of post-mastectomy lymphoedema. Physiotherapy 2002;88:99-107.
  • 39. Qiao J, Yang LN, Kong YH, Huang X, Li Y, Bai DQ. Effect of manual lymphatic drainage on breast cancer-related postmastectomy lymphedema: a meta-analysis of randomized controlled trials. Cancer Nurs 2023;46:159-66.
  • 40. Borman P, Yaman A, Denizli M, Yüzer A, Terzioğlu F. Complex decongestive therapy in a patient with poliomyelitis, bilateral lymphoedema and a deep wound: a case study. J Wound Care 2022;31:792-8.
  • 41. Şahinoğlu E, Ergin G, Karadibak D. The efficacy of change in limb volume on functional mobility, health-related quality of life, social appearance anxiety, and depression in patients with lower extremity lymphedema. Phlebology 2022;37:200-5.
  • 42. Ali K, Dibbs RP, Dougherty C, Iacobas I, Maricevich RS. Treatment outcomes of manual lymphatic drainage in pediatric lymphedema. Ann Vasc Surg 2022;78:263-71.
  • 43. Liu F, Liu NF, Wang L, Chen J, Han L, Yu Z, et al. Treatment of secondary lower limb lymphedema after gynecologic cancer with complex decongestive therapy. Lymphology 2021;54:122-32.
  • 44. Pereira de Godoy JM, Pereira de Godoy LM, Pereira de Godoy HJ, Guerreiro Godoy MF. Reduction of arm lymphedema using manual lymphatic therapy (Godoy Method). Cureus 2022;14:e28374.
  • 45. McNeely ML, Magee DJ, Lees AW, Bagnall KM, Haykowsky M, Hanson J. The addition of manual lymph drainage to compression therapy for breast cancer related lymphedema: a randomized controlled trial. Breast Cancer Res Treat 2004;86:95-106.
  • 46. Cavezzi A, Urso SU, Paccasassi S, Mosti G, Campana F, Colucci R. Bioimpedance spectroscopy and volumetry in the immediate/short-term monitoring of intensive complex decongestive treatment of lymphedema. Phlebology 2020;35:715-23.
  • 47. Campisi CC, Ryn M, Campisi CS, Di Summa P, Boccardo F, Campisi C. Intermittent negative pressure therapy in the combined treatment of peripheral lymphedema. Lymphology 2015;48:197-204.
  • 48. Weber M, Rahn J, Hackl M, Leschinger T, Dresing K, Müller LP, et al. Postoperative swelling after elbow surgery: influence of a negative pressure application in comparison to manual lymphatic drainage-a randomized controlled trial. Arch Orthop Trauma Surg 2023;143:6243-9.
There are 48 citations in total.

Details

Primary Language English
Subjects Rehabilitation
Journal Section Original Articles
Authors

Sedef Ersoy 0000-0001-9018-7937

Nur Kesiktaş 0000-0002-3937-9973

Büşra Şirin 0000-0001-8519-1747

Nazlı Derya Buğdaycı 0000-0002-0631-3791

Halime Kibar 0000-0002-9695-062X

Nurdan Paker 0000-0001-8957-1843

Early Pub Date October 16, 2023
Publication Date November 4, 2023
Submission Date September 4, 2023
Acceptance Date October 6, 2023
Published in Issue Year 2023 Volume: 9 Issue: 6 - November 2023

Cite

AMA Ersoy S, Kesiktaş N, Şirin B, Buğdaycı ND, Kibar H, Paker N. Comparison of manual lymphatic drainage massage and negative pressure massage therapy efficacy in lymphedema patients: a randomized controlled study. Eur Res J. November 2023;9(6):1474-1482. doi:10.18621/eurj.1354942

e-ISSN: 2149-3189 


The European Research Journal, hosted by Turkish JournalPark ACADEMIC, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

by-nc-nd.png

2024