Research Article
BibTex RIS Cite

AN INTEGRATED MENTAL WORKLOAD ASSESSMENT APPROACH BASED ON NASA-TLX AND SMAA-2: A CASE STUDY

Year 2018, Volume: 26 Issue: 2, 88 - 99, 31.05.2018
https://doi.org/10.31796/ogummf.384328

Abstract

Zihinsel iş yükü (ZİY), özellikle
karmaşık sistemlerdeki çalışan performansını etkileyen önemli bir ergonomik
faktör olarak uzun zamandan beri dikkate alınmaktadır. ZİY araştırmaları
genellikle, altı farklı ZİY kriterinin ağırlıklı toplamını içeren ve klasik bir
ZİY değerlendirme yaklaşımı olan Ulusal Havacılık ve Uzay Dairesi İş Yükü
İndeksi (NASA-TLX) yönteminin farklı uygulamalarına odaklanmıştır. Ancak karar
verici (KV) olarak çalışanlar tercihlerini, ZİY boyutlarına ait ağırlıkların ve
puanların belirlenmesi için uygulanan skaladaki kesin değerleri kullanarak
belirtemeyebilir veya belirtmek istemeyebilirler. Bu tür belirsizlik içeren
bilgiler, etkin bir karar destek aracı olan Stokastik Çok Kriterli Kabul
Edilebilirlik Analizi-2 (SMAA-2) kullanılarak stokastik bir şekilde
modellenebilir. Bu kapsamda çalışmada, ZİY’nü analiz edebilmek için NASA TLX ve
SMAA-2 temelinde entegre bir yaklaşım önerilmektedir. Önerilen yaklaşıma
ilişkin gerçek hayat uygulaması ise kutu harf imalat prosesinde
gerçekleştirilmiştir.

References

  • Aertsen, W., Kınt, V., Orshoven, J.V. ve Muys. B. (2011). Evaluation of modelling techniques for forest site productivity prediction in contrasting ecoregions using stochastic multicriteria acceptability analysis (SMAA), Environmental Modelling & Software, 26(7), 929-937.
  • Akyeampong, J., Udoka, S., Caruso, G. ve Bordegoni, M. (2014). Evaluation of hydraulic excavator Human–Machine Interface concepts using NASA TLX. International Journal of Industrial Ergonomics, 44(3), 374-382.
  • Angilella, S., Corrente, S. ve Greco, S. (2015). Stochastic multiobjective acceptability analysis for the Choquet integral preference model and the scale construction problem. European Journal of Operatıonal Research, 240(1), 172-182.
  • Colligan, L., Potts, H. W., Finn, C. T. ve Sinkin, R. A. (2015). Cognitive workload changes for nurses transitioning from a legacy system with paper documentation to a commercial electronic health record. International Journal of Medical Informatics, 84(7), 469-476.
  • Durantin, G., Gagnon, J. F., Tremblay, S. ve Dehais, F. (2014). Using near infrared spectroscopy and heart rate variability to detect mental overload. Behavioural Brain Research, 259, 16-23.
  • Filtness, A. J., Rudin-Brown, C. M. (2012). Drinking and riding: is subjective workload related to performance?. Australian Road Safety Research, Policing and Education Conference.
  • Flemisch, F. O., Onken, R. (2002). Open a Window to the Cognitive Work Process! Pointillist Analysis of Man–Machine Interaction. Cognition, Technology & Work, 4, 160–170.
  • Gülkaç, H. (2013). Assessing Pilot's Mental Workloads with NASA-TLX, Master Thesis, Kocaeli University.
  • Hart, S. G., Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology, 52, 139-183.
  • Hokkanen, J., Lahdelama, R. ve Salminen, P. (2000). Multicriteria decision support in a technology competition for cleaning polluted soil in Helsinki. Journal of Environment Management, 60(4), 339-348.
  • Jorgensen, A.H., Garde, A.H. ve Laursen, B., Jensen, B.R. (1999). Applying the concept of workload to IT work. In Virtual Proceedings of CybErg.
  • Kadzinski, M., Tervonen, T. (2013). Stochastic ordinal regression for multiple criteria sorting problems. Decısıon Support Systems, 55(1), 55-66.
  • Kangas, J., Hokkanen, J., Kangas, A. R. ve Lahdelma, P. S., (2003). Applying stochastic multicriteria acceptability analysis to forest ecosystem management with both cardinal and ordinal criteria. Forest Science, 49(6), 928-937.
  • Kangas, J., Store, R. ve Kangas, A. (2005). Socioecological landscape planning approach and multicriteria acceptability analysis in multiple-purpose forest management, Forest Policy and Economics, 7(4), 603-614.
  • Kangas, A., Kangas, J., Lahdelama, R. ve Salminen, P. (2006). Using SMAA-2 method with dependent uncertainities for strategic forest planning. Forest Policy and Economics, 9(2),113-125.
  • Kuehn, K. A., Chourasia, A. O., Wiegmann, D. A. ve Sesto, M. E. (2013, September). Effects of Orientation on Workload during Touch Screen Operation among Individuals With and Without Disabilities. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 57, No. 1, pp. 1580-1584). SAGE Publications.
  • Lahdelma, R., Salminen, P. (2001). SMAA-2: Stochastic multicriteria acceptability analysis for group decision making. Operations Research, 49(3), 444-454.
  • Lahdelma, R., Salminen, P. (2002). Pseudo-criteria versus linear utility function in stochastic multi-criteria acceptability analysis. European Journal of Operational Research, 141(2), 454-469.
  • Lahdelma, R., Salminen, P. (2006). Classifying efficient alternatives in SMAA using cross confidence factors. European Journal of Operational Research, 170(1), 228-240.
  • Lahdelma, R., Makkonen, S. ve Salminen, P. (2009). Two ways to handle dependent unceartinities in multicriteria decision problems. Omega, 37(1), 79-92.
  • Lee, Y. H., Liu, B. S. (2003). Inflight workload assessment: Comparison of subjective and physiological measurements. Aviation, Space, and Environmental Medicine, 74(10), 1078-1084.
  • Loft, S. P. Sanderson, A. N. ve Mooij, M. (2007). Modeling and predicting mental workload in en route air traffic control: critical review and broader implications. Human Factors, 49(3), 376–399.
  • Matthews, R., Legg, S. ve Charlton, S. (2003). The effect of cell phone type on drivers subjective workload during concurrent driving and conversing. Accident Analysis & Prevention, 35(4), 451-457.
  • Mazur, L. M., Mosaly, P. R., Hoyle, L. M., Jones, E. L. ve Marks, L. B. (2013). Subjective and objective quantification of physician’s workload and performance during radiation therapy planning tasks. Practical Radiation Oncology, 3(4), e171-e177.
  • Moroney, W.F., Biers, D.W. ve Eggemeier, F.T. (1995). Some measurement and methodological considerationsin the application of subjective workload and measurement techniques. International Journal ofAviation Psychology, 5, 87–106.
  • Mouzé-Amady, M., Raufaste, E., Prade, H. ve Meyer, J.P. (2013). Fuzzy-TLX: using fuzzy integrals for evaluating human mental workload with NASA-Task Load indeX in laboratory and field studies. Ergonomics, 56(5), 752-763.
  • Parasuraman, R., Hancock, P. A. (2001). Adaptive Control of Mental Workload. In Stress, Workload and Fatigue, Lawrence Erlbaum, 305–320.
  • Park, J. I. N. K. Y. U. N., Jung, W. ve Ko, J. O. N. G. H. Y. U. N. (2009). Investigating the appropriateness of the TACOM measure–application to the complexity of proceduralized tasks for high speed train drivers. Nuclear Engineering and Technology, 42(1), 115-124.
  • Rubio, S. E., Díaz, J., Martín. ve Puente, J. M. (2004). Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA-TLX, and Workload Profile Methods. Applied Psychology: An International Review, 53(1), 61–86.
  • Ruiz-Rabelo, J. F., Navarro-Rodriguez, E., Di-Stasi, L. L., Diaz-Jimenez, N., Cabrera-Bermon, J., Diaz-Iglesias, C., ... ve Briceño-Delgado, J. (2015). Validation of the NASA-TLX Score in Ongoing Assessment of Mental Workload During a Laparoscopic Learning Curve in Bariatric Surgery. Obesity Surgery, 25(12), 2451-2456.
  • Sporcic, M., Landekic, M., Lovric, M. ve Martinic, I. (2011). Planning and Decision Making Models in Forestry. Croatian Journal of Forest Engineering, 32(1), 443-456.
  • Stefanidis, D., Wang, F., Korndorffer Jr, J. R., Dunne, J. B. ve Scott, D. J. (2010). Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surgical Endoscopy, 24(2), 377-382.
  • Tervonen,T., Hokkanen, J. ve Lahdelma, R. (2008). Elevator planning with stochastic multi criteria acceptability analysis. Omega, 36(3), 352-362.
  • Tervonen, T., Figueira, J.R., Lahdelma, R., Dias, J.A. ve Salminen, P. (2009). A stochastic method for robustness analysis in sorting problems. European Journal of Operatıonal Research, 192(1), 236-242.
  • Tervonen, T., Valkenhoef, G.V., Buskens, E., Hillege, H.L. ve Postmus, D. (2011). A stochastic multicriteria model for evidence based decision making in drug benefit-risk analysis. Statistics in Medicine, 30(12), 1419-1428.
  • Tervonen, T. (2012). JSMAA: open source software for SMAA computations. International Journal of Systems Science, 45(1), 69-81.
  • Vidulich, M.A, Tsang. P.S. (2007). Methodological and theoretical concerns in multitask performance: A critique of Boles, Bursk, Phillips, and Perdelwitz. Human Factors, 49(1), 46-49.
  • Wickens, C. D. (2008). Multiple Resources and Mental Workload. Human Factors, 50(3), 449–455.
  • Young, M. S., Brookhuis, K.A. Wickens, C. D. ve Hancock, P.A. (2015). State of science:mental workload in ergonomics. Ergonomics, 58(1), 1-17.
  • Zheng, B., Jiang, X., Tien, G., Meneghetti, A., Panton, O. N. M. ve Atkins, M. S. (2012). Workload assessment of surgeons: correlation between NASA TLX and blinks. Surgical endoscopy, 26(10), 2746-2750.
Year 2018, Volume: 26 Issue: 2, 88 - 99, 31.05.2018
https://doi.org/10.31796/ogummf.384328

Abstract

References

  • Aertsen, W., Kınt, V., Orshoven, J.V. ve Muys. B. (2011). Evaluation of modelling techniques for forest site productivity prediction in contrasting ecoregions using stochastic multicriteria acceptability analysis (SMAA), Environmental Modelling & Software, 26(7), 929-937.
  • Akyeampong, J., Udoka, S., Caruso, G. ve Bordegoni, M. (2014). Evaluation of hydraulic excavator Human–Machine Interface concepts using NASA TLX. International Journal of Industrial Ergonomics, 44(3), 374-382.
  • Angilella, S., Corrente, S. ve Greco, S. (2015). Stochastic multiobjective acceptability analysis for the Choquet integral preference model and the scale construction problem. European Journal of Operatıonal Research, 240(1), 172-182.
  • Colligan, L., Potts, H. W., Finn, C. T. ve Sinkin, R. A. (2015). Cognitive workload changes for nurses transitioning from a legacy system with paper documentation to a commercial electronic health record. International Journal of Medical Informatics, 84(7), 469-476.
  • Durantin, G., Gagnon, J. F., Tremblay, S. ve Dehais, F. (2014). Using near infrared spectroscopy and heart rate variability to detect mental overload. Behavioural Brain Research, 259, 16-23.
  • Filtness, A. J., Rudin-Brown, C. M. (2012). Drinking and riding: is subjective workload related to performance?. Australian Road Safety Research, Policing and Education Conference.
  • Flemisch, F. O., Onken, R. (2002). Open a Window to the Cognitive Work Process! Pointillist Analysis of Man–Machine Interaction. Cognition, Technology & Work, 4, 160–170.
  • Gülkaç, H. (2013). Assessing Pilot's Mental Workloads with NASA-TLX, Master Thesis, Kocaeli University.
  • Hart, S. G., Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology, 52, 139-183.
  • Hokkanen, J., Lahdelama, R. ve Salminen, P. (2000). Multicriteria decision support in a technology competition for cleaning polluted soil in Helsinki. Journal of Environment Management, 60(4), 339-348.
  • Jorgensen, A.H., Garde, A.H. ve Laursen, B., Jensen, B.R. (1999). Applying the concept of workload to IT work. In Virtual Proceedings of CybErg.
  • Kadzinski, M., Tervonen, T. (2013). Stochastic ordinal regression for multiple criteria sorting problems. Decısıon Support Systems, 55(1), 55-66.
  • Kangas, J., Hokkanen, J., Kangas, A. R. ve Lahdelma, P. S., (2003). Applying stochastic multicriteria acceptability analysis to forest ecosystem management with both cardinal and ordinal criteria. Forest Science, 49(6), 928-937.
  • Kangas, J., Store, R. ve Kangas, A. (2005). Socioecological landscape planning approach and multicriteria acceptability analysis in multiple-purpose forest management, Forest Policy and Economics, 7(4), 603-614.
  • Kangas, A., Kangas, J., Lahdelama, R. ve Salminen, P. (2006). Using SMAA-2 method with dependent uncertainities for strategic forest planning. Forest Policy and Economics, 9(2),113-125.
  • Kuehn, K. A., Chourasia, A. O., Wiegmann, D. A. ve Sesto, M. E. (2013, September). Effects of Orientation on Workload during Touch Screen Operation among Individuals With and Without Disabilities. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 57, No. 1, pp. 1580-1584). SAGE Publications.
  • Lahdelma, R., Salminen, P. (2001). SMAA-2: Stochastic multicriteria acceptability analysis for group decision making. Operations Research, 49(3), 444-454.
  • Lahdelma, R., Salminen, P. (2002). Pseudo-criteria versus linear utility function in stochastic multi-criteria acceptability analysis. European Journal of Operational Research, 141(2), 454-469.
  • Lahdelma, R., Salminen, P. (2006). Classifying efficient alternatives in SMAA using cross confidence factors. European Journal of Operational Research, 170(1), 228-240.
  • Lahdelma, R., Makkonen, S. ve Salminen, P. (2009). Two ways to handle dependent unceartinities in multicriteria decision problems. Omega, 37(1), 79-92.
  • Lee, Y. H., Liu, B. S. (2003). Inflight workload assessment: Comparison of subjective and physiological measurements. Aviation, Space, and Environmental Medicine, 74(10), 1078-1084.
  • Loft, S. P. Sanderson, A. N. ve Mooij, M. (2007). Modeling and predicting mental workload in en route air traffic control: critical review and broader implications. Human Factors, 49(3), 376–399.
  • Matthews, R., Legg, S. ve Charlton, S. (2003). The effect of cell phone type on drivers subjective workload during concurrent driving and conversing. Accident Analysis & Prevention, 35(4), 451-457.
  • Mazur, L. M., Mosaly, P. R., Hoyle, L. M., Jones, E. L. ve Marks, L. B. (2013). Subjective and objective quantification of physician’s workload and performance during radiation therapy planning tasks. Practical Radiation Oncology, 3(4), e171-e177.
  • Moroney, W.F., Biers, D.W. ve Eggemeier, F.T. (1995). Some measurement and methodological considerationsin the application of subjective workload and measurement techniques. International Journal ofAviation Psychology, 5, 87–106.
  • Mouzé-Amady, M., Raufaste, E., Prade, H. ve Meyer, J.P. (2013). Fuzzy-TLX: using fuzzy integrals for evaluating human mental workload with NASA-Task Load indeX in laboratory and field studies. Ergonomics, 56(5), 752-763.
  • Parasuraman, R., Hancock, P. A. (2001). Adaptive Control of Mental Workload. In Stress, Workload and Fatigue, Lawrence Erlbaum, 305–320.
  • Park, J. I. N. K. Y. U. N., Jung, W. ve Ko, J. O. N. G. H. Y. U. N. (2009). Investigating the appropriateness of the TACOM measure–application to the complexity of proceduralized tasks for high speed train drivers. Nuclear Engineering and Technology, 42(1), 115-124.
  • Rubio, S. E., Díaz, J., Martín. ve Puente, J. M. (2004). Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA-TLX, and Workload Profile Methods. Applied Psychology: An International Review, 53(1), 61–86.
  • Ruiz-Rabelo, J. F., Navarro-Rodriguez, E., Di-Stasi, L. L., Diaz-Jimenez, N., Cabrera-Bermon, J., Diaz-Iglesias, C., ... ve Briceño-Delgado, J. (2015). Validation of the NASA-TLX Score in Ongoing Assessment of Mental Workload During a Laparoscopic Learning Curve in Bariatric Surgery. Obesity Surgery, 25(12), 2451-2456.
  • Sporcic, M., Landekic, M., Lovric, M. ve Martinic, I. (2011). Planning and Decision Making Models in Forestry. Croatian Journal of Forest Engineering, 32(1), 443-456.
  • Stefanidis, D., Wang, F., Korndorffer Jr, J. R., Dunne, J. B. ve Scott, D. J. (2010). Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surgical Endoscopy, 24(2), 377-382.
  • Tervonen,T., Hokkanen, J. ve Lahdelma, R. (2008). Elevator planning with stochastic multi criteria acceptability analysis. Omega, 36(3), 352-362.
  • Tervonen, T., Figueira, J.R., Lahdelma, R., Dias, J.A. ve Salminen, P. (2009). A stochastic method for robustness analysis in sorting problems. European Journal of Operatıonal Research, 192(1), 236-242.
  • Tervonen, T., Valkenhoef, G.V., Buskens, E., Hillege, H.L. ve Postmus, D. (2011). A stochastic multicriteria model for evidence based decision making in drug benefit-risk analysis. Statistics in Medicine, 30(12), 1419-1428.
  • Tervonen, T. (2012). JSMAA: open source software for SMAA computations. International Journal of Systems Science, 45(1), 69-81.
  • Vidulich, M.A, Tsang. P.S. (2007). Methodological and theoretical concerns in multitask performance: A critique of Boles, Bursk, Phillips, and Perdelwitz. Human Factors, 49(1), 46-49.
  • Wickens, C. D. (2008). Multiple Resources and Mental Workload. Human Factors, 50(3), 449–455.
  • Young, M. S., Brookhuis, K.A. Wickens, C. D. ve Hancock, P.A. (2015). State of science:mental workload in ergonomics. Ergonomics, 58(1), 1-17.
  • Zheng, B., Jiang, X., Tien, G., Meneghetti, A., Panton, O. N. M. ve Atkins, M. S. (2012). Workload assessment of surgeons: correlation between NASA TLX and blinks. Surgical endoscopy, 26(10), 2746-2750.
There are 40 citations in total.

Details

Primary Language English
Subjects Industrial Engineering
Journal Section Research Articles
Authors

Elif Kılıç Delice

Gülin Feryal Can

Publication Date May 31, 2018
Acceptance Date April 2, 2018
Published in Issue Year 2018 Volume: 26 Issue: 2

Cite

APA Kılıç Delice, E., & Can, G. F. (2018). AN INTEGRATED MENTAL WORKLOAD ASSESSMENT APPROACH BASED ON NASA-TLX AND SMAA-2: A CASE STUDY. Eskişehir Osmangazi Üniversitesi Mühendislik Ve Mimarlık Fakültesi Dergisi, 26(2), 88-99. https://doi.org/10.31796/ogummf.384328
AMA Kılıç Delice E, Can GF. AN INTEGRATED MENTAL WORKLOAD ASSESSMENT APPROACH BASED ON NASA-TLX AND SMAA-2: A CASE STUDY. ESOGÜ Müh Mim Fak Derg. May 2018;26(2):88-99. doi:10.31796/ogummf.384328
Chicago Kılıç Delice, Elif, and Gülin Feryal Can. “AN INTEGRATED MENTAL WORKLOAD ASSESSMENT APPROACH BASED ON NASA-TLX AND SMAA-2: A CASE STUDY”. Eskişehir Osmangazi Üniversitesi Mühendislik Ve Mimarlık Fakültesi Dergisi 26, no. 2 (May 2018): 88-99. https://doi.org/10.31796/ogummf.384328.
EndNote Kılıç Delice E, Can GF (May 1, 2018) AN INTEGRATED MENTAL WORKLOAD ASSESSMENT APPROACH BASED ON NASA-TLX AND SMAA-2: A CASE STUDY. Eskişehir Osmangazi Üniversitesi Mühendislik ve Mimarlık Fakültesi Dergisi 26 2 88–99.
IEEE E. Kılıç Delice and G. F. Can, “AN INTEGRATED MENTAL WORKLOAD ASSESSMENT APPROACH BASED ON NASA-TLX AND SMAA-2: A CASE STUDY”, ESOGÜ Müh Mim Fak Derg, vol. 26, no. 2, pp. 88–99, 2018, doi: 10.31796/ogummf.384328.
ISNAD Kılıç Delice, Elif - Can, Gülin Feryal. “AN INTEGRATED MENTAL WORKLOAD ASSESSMENT APPROACH BASED ON NASA-TLX AND SMAA-2: A CASE STUDY”. Eskişehir Osmangazi Üniversitesi Mühendislik ve Mimarlık Fakültesi Dergisi 26/2 (May 2018), 88-99. https://doi.org/10.31796/ogummf.384328.
JAMA Kılıç Delice E, Can GF. AN INTEGRATED MENTAL WORKLOAD ASSESSMENT APPROACH BASED ON NASA-TLX AND SMAA-2: A CASE STUDY. ESOGÜ Müh Mim Fak Derg. 2018;26:88–99.
MLA Kılıç Delice, Elif and Gülin Feryal Can. “AN INTEGRATED MENTAL WORKLOAD ASSESSMENT APPROACH BASED ON NASA-TLX AND SMAA-2: A CASE STUDY”. Eskişehir Osmangazi Üniversitesi Mühendislik Ve Mimarlık Fakültesi Dergisi, vol. 26, no. 2, 2018, pp. 88-99, doi:10.31796/ogummf.384328.
Vancouver Kılıç Delice E, Can GF. AN INTEGRATED MENTAL WORKLOAD ASSESSMENT APPROACH BASED ON NASA-TLX AND SMAA-2: A CASE STUDY. ESOGÜ Müh Mim Fak Derg. 2018;26(2):88-99.

20873  13565  13566 15461  13568    14913