Research Article
BibTex RIS Cite

I-GROUP PRESENCE QUESTIONNAIRE: PSYCHOMETRICALLY REVISED ENGLISH VERSION

Year 2021, , 1 - 10, 16.10.2021

Mehmet İlker Berkman , Güven Çatak

https://doi.org/10.22531/muglajsci.882271
Cited By: 6

Abstract

I-group Presence Questionnaire (IPQ), which is used to evaluate the mediated experience of presence -especially for virtual reality applications- is originally developed in German and translated to several other languages. However, there is not any psychometric study for these translations including English version, except the Portuguese and Persian translations. We evaluated English translation of IPQ with 36 participants through 12 VR sessions with an overall of 432 samples. Using a partial least squares based factor analysis approach, the original 14-item set is trimmed into 11-items in order to achieve better psychometric qualities. In addition, a covariance based confirmatory factor analysis is executed to compare models. Several indices, even the conservative Cronbach’s alpha indicated that the subscales of 11-item version are reliable, but not the 14-item version. Eliminated items did not lead to a decrease in scales’ sensitivity to identify different levels of Realism, Spatial Presence and Involvement for different virtual environments. Although we provided evidence to remove the items which are identically worded and inversely coded that are causing measurement error, we suggest researchers to employ the 14-items but report the results for both 14-item version and 11-item version, until the psychometric qualities of IPQ in English is confirmed with a larger sample of participants.

Keywords

virtual reality spatial presence I-group PLS-CFA

References

  • Barfield, W. “Musings on presence twenty-five years after ‘being there,’” Presence: Teleoperators and Virtual Environments. 25(2):148-150, 2016.
  • Lombard, M. and Jones, M.T. (Lombard, Biocca, Freeman, IJsselsteijn, and Schaevitz,) “Defining presence” in Immersed in Media: Telepresence Theory, Measurement and Technology, Springer, 13–34, 2015.
  • Berkman, M.I. and Akan, E. (Lee) “Presence and Immersion in Virtual Reality” Encyclopedia of Computer Graphics and Games. Springer International Publishing, Cham, 1–10, 2019.
  • Biocca, F., Harms, C. and Burgoon, C. “Toward a More Robust Theory and Measure of Social Presence: Review and Suggested Criteria” in Presence: Teleoperators and Virtual Environments, 12(5):456-480, 2003. Lee, K.M., “Presence, Explicated” Commun. Theory, 14(1):27-50, 2004.
  • Zahorik, P. and Jenison, R.L. “Presence as being-in-the-world” Presence Teleoperators Virtual Environ., 7(1):78-89 1998.
  • Lombard, M. and Ditton, T., “At the Heart of It All: The Concept of Presence” J. Comput. Commun., 3(2), 1997.
  • Harms, C. and Biocca, F. “Internal Consistency and Reliability of the Networked Minds Measure of Social Presence” Seventh Annu. Int. Work. Presence 2004, 2004.
  • Sheridan, T.B. “Musings on Telepresence and Virtual Presence” Presence Teleoperators Virtual Environ., 1(1):120–126, 1992.
  • Biocca, F. and Delaney, B. (Lombard & Levy) “Immersive virtual reality technology” Communication in the Age of Virtual Reality, 57-126, 1995.
  • Slater, M. “Immersion and the illusion of presence in virtual reality” British Journal of Psychology, 109(3):431-433, 2018.
  • Laarni, J. et al., (Lombard, Biocca, Freeman, IJsselsteijn, and Schaevitz,) “Ways to measure spatial presence: Review and future directions” in Immersed in Media: Telepresence Theory, Measurement and Technology, 139-185, Springer, 2015.
  • Schubert, T., Friedmann, F. and Regenbrecht, H. “The experience of presence: Factor analytic insights” Presence Teleoperators Virtual Environ., 10(3):266–281, 2001.
  • Elsey, J. W. B., et al., “The impact of virtual reality versus 2D pornography on sexual arousal and presence” Comput. Human Behav., 97:35-43, 2019.
  • Insko, B. E., (Riva, Davide, & IJsselsteijn) “Measuring Presence: Subjective, Behavioral and Physiological Methods,” Studies in new technologies and practices in communication. 109-119, 2003.
  • Slater, M., Usoh, M. and Steed, A. “Depth of Presence in Virtual Environments,” Presence Teleoperators Virtual Environ., 3(2):130-144, 1994.
  • Kim, T. “Telepresence via Television: Two Dimensions of Telepresence May Have Different Connections to Memory and Persuasion.[1],” Journal of Computer-Mediated Communication, 3, 1997.
  • Barfield, W., Baird, B. E. and Bjorneseth, O. J. “Presence in virtual environments as a function of type of input device and display update rate,” Displays, 19(2):91-98, 1998.
  • Witmer, B. G. and Singer, M. J. “Measuring presence in virtual environments: A presence questionnaire,” Presence Teleoperators Virtual Environ., 7(3):225-240. 1998.
  • Biocca, F., Jin, K. and Choi, Y. “Visual touch in virtual environments: An exploratory study of presence, multimodal interfaces, and cross-modal sensory illusions,” Presence Teleoperators Virtual Environ., 10(3):247-265, 2001.
  • Lessiter, J. et al. “A cross-media presence questionnaire: The ITC-sense of presence inventory,” Presence Teleoperators Virtual Environ., 10(3):282-297, 2001.
  • Wirth, W. et al. “A process model of the formation of spatial presence experiences. Media Psychol., 9(3):493-525, 2007.
  • Hartmann, T. et al., “The spatial presence experience scale (SPES): A short self-report measure for diverse media settings,” J. Media Psychol., 28(1):1-15, 2016.
  • Weibel, D. et al., “Measuring spatial presence: Introducing and validating the pictorial presence SAM,” Presence Teleoperators Virtual Environ., 24(1):44-61 ,2015.
  • Lombard, M., Ditton T. B. and Weinstein, L. “Measuring Presence: The Temple Presence Inventory,” Proceedings of Presence 2009 12th Int. Work. Presence, 2009.
  • Schubert, T., Friedmann, F. and Regenbrecht, H. “Embodied Presence in Virtual Environments” Visual Representations and Interpretations, 269-278 1999.
  • Slater, M. and Usoh, M. “Representations Systems, Perceptual Position, and Presence in Immersive Virtual Environments” Presence Teleoperators Virtual Environ., 2(3):221-233, 1993.
  • Hendrix, C. M. “Exploratory Studies on the Sense of Presence in Virtual Environments as a Function of Visual and Auditory Display Parameters” Master's thesis, Univ. Washingt., 1994.
  • Hendrix, C.M. and Barfield, W. “Presence within virtual environments as a function of visual display parameters” Presence Teleoperators Virtual Environ., 5(3), 274-289:1996.
  • Carlin, A. S., Hoffman, H. G. and Weghorst, S. “Virtual reality and tactile augmentation in the treatment of spider phobia: A case report” Behav. Res. Ther., 35(2):153-158, 1997.
  • Schwind, V. et al., N. “Using presence questionnaires in virtual reality” in Conference on Human Factors in Computing Systems, 2019.
  • Panahi-Shahri, M. et al., “Reliability and validity of Igroup Presence Questionnaire (IPQ)” J. Behav. Sci., 3(1):27–34, 2009.
  • Vasconcelos-Raposo, J. et al., “Adaptation and validation of the Igroup Presence Questionnaire (IPQ) in a Portuguese sample” Presence Teleoperators Virtual Environ., 25(3):191-203, 2016.
  • Melo, M., Vasconcelos-Raposo, J. and Bessa, M. “Presence and cybersickness in immersive content: Effects of content type, exposure time and gender” Comput. Graph., 71:159-165, 2018.
  • Rivera, H. et al., “Evaluation of Cybersickness and Sense of Presence in a VR Simulator of Electric-Powered Wheelchairs” in 2nd International Workshop on Assistive Technology (IWAT2019), 2019.
  • Narciso, D. et al., “Immersive 360∘ video user experience: impact of different variables in the sense of presence and cybersickness” Univers. Access Inf. Soc., 18:77–87, 2019.
  • I-group.org-project, “igroup presence questionnaire (IPQ) Item Download” igroup.org website. [Online]. Available: www.igroup.org/pq/ipq/items.php. [Accessed: 15-Mar-2020].
  • Clifford R. M. S. et al., “Aerial firefighter radio communication performance in a virtual training system: radio communication disruptions simulated in VR for Air Attack Supervision,” Vis. Comput., 37:63–76, 2020.
  • Wallis, G. and Tichon, J. “Predicting the efficacy of simulator-based training using a perceptual judgment task versus questionnaire-based measures of presence,” Presence Teleoperators Virtual Environ., 22(1):67-85, 2013.
  • Archambault, P. S. et al., “Driving performance in a power wheelchair simulator,” Disabil. Rehabil. Assist. Technol., 7(3):226-233, 2012.
  • Simms, L. J. et al., “Does the Number of Response Options Matter? Psychometric Perspectives Using Personality Questionnaire Data” Psychol. Assess., 31(4):557 2019.
  • Lewis, J.R., “Measuring User Experience With 3, 5, 7, or 11 Points: Does It Matter?” Hum. Factors, Online first, 2019.
  • Hair, J. F., Ringle, C. M. and Sarstedt, M. . “PLS-SEM: Indeed a Silver Bullet” J. Mark. Theory Pract., 19[2]: 139–152, 2011.
  • Henseler, J., Hubona, G. and Ray, J. “Using PLS path modeling in new technology research: updated guidelines” Ind. Manag. Data Syst., 116(1):2–20, Feb. 2016.
  • Ringle, C., Wende, S. and Becker, J.M. “SmartPLS3.” Boenningstedt: SmartPLS GmbH, 2015.
  • von Oertzen, T., Brandmaier, A. M. and Tsang, S. “Structural Equation Modeling With Ωnyx,” Struct. Equ. Model., 22(1):148-161, 2015.
  • Fornell, C. and Larcker, D. F. “Evaluating structural model with unobserved variables and measurement errors” J. Mark. Res., 18(1):39-50 1981.
  • Henseler, J., Ringle, C. M. andSarstedt, M. “A new criterion for assessing discriminant validity in variance-based structural equation modeling”. J. Acad. Mark. Sci., 43(1):115-135, 2014.
  • Schreiber, J.B. et al., “Reporting structural equation modeling and confirmatory factor analysis results: A review,” Journal of Educational Research. 99(6):323-338, 2006.

Year 2021, , 1 - 10, 16.10.2021

Mehmet İlker Berkman , Güven Çatak

https://doi.org/10.22531/muglajsci.882271
Cited By: 6

Abstract

Özellikle sanal gerçeklik uygulamaları için, teknoloji aracılığı ile tecrübe edilen "mevcudiyet" (presence) deneyimini değerlendirmek için kullanılan I-grup Varlık Anketi (IPQ), orijinal olarak Almanca olarak geliştirilmiş ve diğer birçok dile çevrilmiştir. Ancak bu çeviriler için İngilizce versiyonu da dahil olmak üzere, Portekizce ve Farsça çeviriler dışında psikometrik bir çalışma bulunmamaktadır. Bu çalışmada IPQ'nun İngilizce çevirisi, 36 katılımcı ile toplam 432 örneğin derlendiği 12 VR seansı aracılığıyla psikometrik olarak değerlendirilmiştir. PLS tabanlı faktör analizi yaklaşımı kullanılarak, daha iyi psikometrik nitelikler elde etmek için orijinal 14 maddelik set, 11 maddeye indirgenmiştir. Ayrıca, modelleri karşılaştırmak için kovaryansa dayalı doğrulayıcı faktör analizi yapılmıştır. Cornbach's alfa katsayısı başta olmak üzere çeşitli indikatörler 11 maddelik versiyonun alt ölçeklerinin güvenilir olduğunu, ancak 14 maddelik versiyonun olmadığını göstermektedir. Ölçekten çıkarılan maddeler, farklı sanal gerçeklik ortamları için farklı Gerçekçilik, Mekansal Mevcudiyet ve Katılım düzeylerini belirleyen alt ölçeklerin duyarlılığında bir azalmaya yol açmamıştır. Ölçüm hatasına neden olan benzer ifadelerle yazılmış veya ters kodlanmış maddeleri ölçek haricinde tutmak için kanıt sağlanmıştır.

References

  • Barfield, W. “Musings on presence twenty-five years after ‘being there,’” Presence: Teleoperators and Virtual Environments. 25(2):148-150, 2016.
  • Lombard, M. and Jones, M.T. (Lombard, Biocca, Freeman, IJsselsteijn, and Schaevitz,) “Defining presence” in Immersed in Media: Telepresence Theory, Measurement and Technology, Springer, 13–34, 2015.
  • Berkman, M.I. and Akan, E. (Lee) “Presence and Immersion in Virtual Reality” Encyclopedia of Computer Graphics and Games. Springer International Publishing, Cham, 1–10, 2019.
  • Biocca, F., Harms, C. and Burgoon, C. “Toward a More Robust Theory and Measure of Social Presence: Review and Suggested Criteria” in Presence: Teleoperators and Virtual Environments, 12(5):456-480, 2003. Lee, K.M., “Presence, Explicated” Commun. Theory, 14(1):27-50, 2004.
  • Zahorik, P. and Jenison, R.L. “Presence as being-in-the-world” Presence Teleoperators Virtual Environ., 7(1):78-89 1998.
  • Lombard, M. and Ditton, T., “At the Heart of It All: The Concept of Presence” J. Comput. Commun., 3(2), 1997.
  • Harms, C. and Biocca, F. “Internal Consistency and Reliability of the Networked Minds Measure of Social Presence” Seventh Annu. Int. Work. Presence 2004, 2004.
  • Sheridan, T.B. “Musings on Telepresence and Virtual Presence” Presence Teleoperators Virtual Environ., 1(1):120–126, 1992.
  • Biocca, F. and Delaney, B. (Lombard & Levy) “Immersive virtual reality technology” Communication in the Age of Virtual Reality, 57-126, 1995.
  • Slater, M. “Immersion and the illusion of presence in virtual reality” British Journal of Psychology, 109(3):431-433, 2018.
  • Laarni, J. et al., (Lombard, Biocca, Freeman, IJsselsteijn, and Schaevitz,) “Ways to measure spatial presence: Review and future directions” in Immersed in Media: Telepresence Theory, Measurement and Technology, 139-185, Springer, 2015.
  • Schubert, T., Friedmann, F. and Regenbrecht, H. “The experience of presence: Factor analytic insights” Presence Teleoperators Virtual Environ., 10(3):266–281, 2001.
  • Elsey, J. W. B., et al., “The impact of virtual reality versus 2D pornography on sexual arousal and presence” Comput. Human Behav., 97:35-43, 2019.
  • Insko, B. E., (Riva, Davide, & IJsselsteijn) “Measuring Presence: Subjective, Behavioral and Physiological Methods,” Studies in new technologies and practices in communication. 109-119, 2003.
  • Slater, M., Usoh, M. and Steed, A. “Depth of Presence in Virtual Environments,” Presence Teleoperators Virtual Environ., 3(2):130-144, 1994.
  • Kim, T. “Telepresence via Television: Two Dimensions of Telepresence May Have Different Connections to Memory and Persuasion.[1],” Journal of Computer-Mediated Communication, 3, 1997.
  • Barfield, W., Baird, B. E. and Bjorneseth, O. J. “Presence in virtual environments as a function of type of input device and display update rate,” Displays, 19(2):91-98, 1998.
  • Witmer, B. G. and Singer, M. J. “Measuring presence in virtual environments: A presence questionnaire,” Presence Teleoperators Virtual Environ., 7(3):225-240. 1998.
  • Biocca, F., Jin, K. and Choi, Y. “Visual touch in virtual environments: An exploratory study of presence, multimodal interfaces, and cross-modal sensory illusions,” Presence Teleoperators Virtual Environ., 10(3):247-265, 2001.
  • Lessiter, J. et al. “A cross-media presence questionnaire: The ITC-sense of presence inventory,” Presence Teleoperators Virtual Environ., 10(3):282-297, 2001.
  • Wirth, W. et al. “A process model of the formation of spatial presence experiences. Media Psychol., 9(3):493-525, 2007.
  • Hartmann, T. et al., “The spatial presence experience scale (SPES): A short self-report measure for diverse media settings,” J. Media Psychol., 28(1):1-15, 2016.
  • Weibel, D. et al., “Measuring spatial presence: Introducing and validating the pictorial presence SAM,” Presence Teleoperators Virtual Environ., 24(1):44-61 ,2015.
  • Lombard, M., Ditton T. B. and Weinstein, L. “Measuring Presence: The Temple Presence Inventory,” Proceedings of Presence 2009 12th Int. Work. Presence, 2009.
  • Schubert, T., Friedmann, F. and Regenbrecht, H. “Embodied Presence in Virtual Environments” Visual Representations and Interpretations, 269-278 1999.
  • Slater, M. and Usoh, M. “Representations Systems, Perceptual Position, and Presence in Immersive Virtual Environments” Presence Teleoperators Virtual Environ., 2(3):221-233, 1993.
  • Hendrix, C. M. “Exploratory Studies on the Sense of Presence in Virtual Environments as a Function of Visual and Auditory Display Parameters” Master's thesis, Univ. Washingt., 1994.
  • Hendrix, C.M. and Barfield, W. “Presence within virtual environments as a function of visual display parameters” Presence Teleoperators Virtual Environ., 5(3), 274-289:1996.
  • Carlin, A. S., Hoffman, H. G. and Weghorst, S. “Virtual reality and tactile augmentation in the treatment of spider phobia: A case report” Behav. Res. Ther., 35(2):153-158, 1997.
  • Schwind, V. et al., N. “Using presence questionnaires in virtual reality” in Conference on Human Factors in Computing Systems, 2019.
  • Panahi-Shahri, M. et al., “Reliability and validity of Igroup Presence Questionnaire (IPQ)” J. Behav. Sci., 3(1):27–34, 2009.
  • Vasconcelos-Raposo, J. et al., “Adaptation and validation of the Igroup Presence Questionnaire (IPQ) in a Portuguese sample” Presence Teleoperators Virtual Environ., 25(3):191-203, 2016.
  • Melo, M., Vasconcelos-Raposo, J. and Bessa, M. “Presence and cybersickness in immersive content: Effects of content type, exposure time and gender” Comput. Graph., 71:159-165, 2018.
  • Rivera, H. et al., “Evaluation of Cybersickness and Sense of Presence in a VR Simulator of Electric-Powered Wheelchairs” in 2nd International Workshop on Assistive Technology (IWAT2019), 2019.
  • Narciso, D. et al., “Immersive 360∘ video user experience: impact of different variables in the sense of presence and cybersickness” Univers. Access Inf. Soc., 18:77–87, 2019.
  • I-group.org-project, “igroup presence questionnaire (IPQ) Item Download” igroup.org website. [Online]. Available: www.igroup.org/pq/ipq/items.php. [Accessed: 15-Mar-2020].
  • Clifford R. M. S. et al., “Aerial firefighter radio communication performance in a virtual training system: radio communication disruptions simulated in VR for Air Attack Supervision,” Vis. Comput., 37:63–76, 2020.
  • Wallis, G. and Tichon, J. “Predicting the efficacy of simulator-based training using a perceptual judgment task versus questionnaire-based measures of presence,” Presence Teleoperators Virtual Environ., 22(1):67-85, 2013.
  • Archambault, P. S. et al., “Driving performance in a power wheelchair simulator,” Disabil. Rehabil. Assist. Technol., 7(3):226-233, 2012.
  • Simms, L. J. et al., “Does the Number of Response Options Matter? Psychometric Perspectives Using Personality Questionnaire Data” Psychol. Assess., 31(4):557 2019.
  • Lewis, J.R., “Measuring User Experience With 3, 5, 7, or 11 Points: Does It Matter?” Hum. Factors, Online first, 2019.
  • Hair, J. F., Ringle, C. M. and Sarstedt, M. . “PLS-SEM: Indeed a Silver Bullet” J. Mark. Theory Pract., 19[2]: 139–152, 2011.
  • Henseler, J., Hubona, G. and Ray, J. “Using PLS path modeling in new technology research: updated guidelines” Ind. Manag. Data Syst., 116(1):2–20, Feb. 2016.
  • Ringle, C., Wende, S. and Becker, J.M. “SmartPLS3.” Boenningstedt: SmartPLS GmbH, 2015.
  • von Oertzen, T., Brandmaier, A. M. and Tsang, S. “Structural Equation Modeling With Ωnyx,” Struct. Equ. Model., 22(1):148-161, 2015.
  • Fornell, C. and Larcker, D. F. “Evaluating structural model with unobserved variables and measurement errors” J. Mark. Res., 18(1):39-50 1981.
  • Henseler, J., Ringle, C. M. andSarstedt, M. “A new criterion for assessing discriminant validity in variance-based structural equation modeling”. J. Acad. Mark. Sci., 43(1):115-135, 2014.
  • Schreiber, J.B. et al., “Reporting structural equation modeling and confirmatory factor analysis results: A review,” Journal of Educational Research. 99(6):323-338, 2006.
There are 48 citations in total.

Details

Primary Language English
Subjects Engineering
Journal Section Journals
Authors

Mehmet İlker Berkman 0000-0002-2340-9373

Güven Çatak 0000-0002-4679-8973

Publication Date October 16, 2021
Published in Issue Year 2021

Cite

APA Berkman, M. İ., & Çatak, G. (2021). I-GROUP PRESENCE QUESTIONNAIRE: PSYCHOMETRICALLY REVISED ENGLISH VERSION. Mugla Journal of Science and Technology, 7, 1-10. https://doi.org/10.22531/muglajsci.882271
AMA Berkman Mİ, Çatak G. I-GROUP PRESENCE QUESTIONNAIRE: PSYCHOMETRICALLY REVISED ENGLISH VERSION. MJST. October 2021;7:1-10. doi:10.22531/muglajsci.882271
Chicago Berkman, Mehmet İlker, and Güven Çatak. “I-GROUP PRESENCE QUESTIONNAIRE: PSYCHOMETRICALLY REVISED ENGLISH VERSION”. Mugla Journal of Science and Technology 7, October (October 2021): 1-10. https://doi.org/10.22531/muglajsci.882271.
EndNote Berkman Mİ, Çatak G (October 1, 2021) I-GROUP PRESENCE QUESTIONNAIRE: PSYCHOMETRICALLY REVISED ENGLISH VERSION. Mugla Journal of Science and Technology 7 1–10.
IEEE M. İ. Berkman and G. Çatak, “I-GROUP PRESENCE QUESTIONNAIRE: PSYCHOMETRICALLY REVISED ENGLISH VERSION”, MJST, vol. 7, pp. 1–10, 2021, doi: 10.22531/muglajsci.882271.
ISNAD Berkman, Mehmet İlker - Çatak, Güven. “I-GROUP PRESENCE QUESTIONNAIRE: PSYCHOMETRICALLY REVISED ENGLISH VERSION”. Mugla Journal of Science and Technology 7 (October 2021), 1-10. https://doi.org/10.22531/muglajsci.882271.
JAMA Berkman Mİ, Çatak G. I-GROUP PRESENCE QUESTIONNAIRE: PSYCHOMETRICALLY REVISED ENGLISH VERSION. MJST. 2021;7:1–10.
MLA Berkman, Mehmet İlker and Güven Çatak. “I-GROUP PRESENCE QUESTIONNAIRE: PSYCHOMETRICALLY REVISED ENGLISH VERSION”. Mugla Journal of Science and Technology, vol. 7, 2021, pp. 1-10, doi:10.22531/muglajsci.882271.
Vancouver Berkman Mİ, Çatak G. I-GROUP PRESENCE QUESTIONNAIRE: PSYCHOMETRICALLY REVISED ENGLISH VERSION. MJST. 2021;7:1-10.

Cited By

Role of virtual reality in examining the effect of fear of falling (FOF) on postural stability in individuals without and with Parkinson’s disease in Egypt: a mixed-methods feasibility study protocol
BMJ Open
https://doi.org/10.1136/bmjopen-2023-080592
Effects of visual flow velocity on cycling experience in virtual reality
German Journal of Exercise and Sport Research
https://doi.org/10.1007/s12662-024-00964-4
Effectiveness of Virtual Reality Environment for Post-Operative Pain Management
International Journal of Recent Surgical and Medical Sciences
https://doi.org/10.25259/IJRSMS_29_2023
Exploring the Educational Value of an Immersive Virtual Reality Method Within a Continuing Education Module in Nursing: A Mixed Methods Study
The Journal of Continuing Education in Nursing
https://doi.org/10.3928/00220124-20240201-04
Mining players’ experience in computer games: Immersion affects flow but not presence
Computers in Human Behavior Reports
https://doi.org/10.1016/j.chbr.2023.100334
Polish Adaptation of the Igroup Presence Questionnaire
Homo Ludens
https://doi.org/10.14746/HL.2022.15.10

5975f2e33b6ce.png
Muğla Sıtkı Koçman Üniversitesi Fen Bilimleri ve Teknoloji Dergisi Creative Commons Atıf-GayriTicari-AynıLisanslaPaylaş 4.0 Uluslararası Lisansı ile lisanslanmıştır.