INVITED ARTICLE: Building Journal Impact Factor Quartile into the Assessment of Academic Performance: A Case Study

Year 2020, Volume: 7 Issue: 2, 1 - 13, 01.08.2020

Keziban Orbay , Ruben Miranda Metin Orbay

https://doi.org/10.17275/per.20.26.7.2

Cited By: 16

Abstract

This study aims to provide information about the Q Concept defined as the division of journal impact factors into quartiles based on given field categories so that the disadvantages resulting from the direct use of journal impact factors can be eliminated. While the number of "Original articles published in the Web of Science (WoS) database-indexed journals like SCI, SSCI and A&HCI" is an important indicator for research assessment in Turkey, neither the journal impact factors nor the Q Concept of these papers have been taken into account. Present study analyzes the scientific production of the Amasya University researchers in journals indexed in WoS database in the period 2014-2018 using the Q concept. The share of publications by Q category journals as well as the average citations received by the works from Amasya University were compared to the average situation in Turkey and other different countries in the world. Results indicate that the articles published by Amasya University researchers were mostly published in low impact factor journals (Q4 journals) (36.49%), in fact, only a small share of papers were published in high impact journals (14.32% in Q1 journals). The share of papers published in low impact journals by researchers from Amasya University is higher than the Turkish average and much higher than the scientific leading countries. The average citations received by papers published in Q1 journals was around six times higher than papers published in Q4 journals (8.92 vs. 1.56), thus papers published in Q1 journals received 30.02% citations despite only 14.32% of the papers was published in these journals. The share of papers published which were never cited in WoS was 27.48%, increasing from 9.68% in Q1 to almost half (48.10%) in Q4. The study concludes with some suggestions on how and where the Q Concept can be used.

Keywords

Journal Impact Factor, JIF Quartile, Academic Performance Assessment, Web of Science

References

• Al, U., & Soydal, İ. (2014). The war of academia with citation indexes. Hacettepe University Journal of Faculty of Letters, 31(1), 23–42.
• Alvarez, P., Boulaiz, H., Velez, C., Rodriguez-Serrano, F., Ortiz, R., Melguizo, C., et.al. (2014). Qualitative and qauntative analyses of anatomists’ reseach: Evulation of multidiscipliniarity and trends in scientific production. Scientometrics, 98(1), 447-456.
• Archambault E., & Larivière V. (2009). History of the journal impact factor: Contingencies and consequences. Scientometrics, 79, 635-649.
• Bornmann, L., & Marx, W. (2014). How to evaluate individual reserchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations. Scientometrics, 98(1), 487-509.
• Bornmann L., & Williams R. (2017a). Can the journal impact factor be used as a criterion for the selection of junior researchers? A large-scale empirical study based on ResercherID data. Journal of Informetrics, 11, 788-799.
• Bornmann, L., & Williams R. (2017b). Use of the journal impact factor as a criterion for the selection of junior researchers: A rejoinder on a comment by Peters (2017). Journal of Informetrics, 11, 945-947.
• Callaway, E. (2016). Publishing elite turns against impact factor, Nature, 535, 210-211.
• Cameron, W.B. (1963). Informal sociology: A casual intoduction to sociological thinking. New York: Random House.
• Chinchilla-Rodriguez, Z., Arencibia-Jorse, R., de Mova-Anegon, F., & Corera-Alvarez, E. (2015). Somes patterns of Cuban scientific publications in Scopus: The current situation and challenges. Scientometrics, 103(3), 779-794.
• Clarivate Analytics. (2019a). Web of Science. https://mjl.clarivate.com/scope-notes Accessed November 10, 2019.
• Clarivate Analytics. (2019b). Journal Impact Factor Quartile http://help.incites.clarivate.com/inCites2Live/indicatorsGroup/aboutHandbook.html Accessed November 10, 2019.
• Clarivate Analytics. (2019c). Web of Science: Emerging Sources Citation Index. http://help.incites.clarivate.com/inCites2Live/indicatorsGroup/aboutHandbook.html Accessed November 10, 2019.
• Doğan, G., Dhyi, S.M.M.A., & Al, U. (2018). A Research on Turkey-Addressed Dropped Journals from Web of Science. Turkish Librarianship, 32(3) 151-162.
• DORA (2012). San Francisco Declaration on Research Assessment. http://am.ascb.org/dora/. Accessed November 10, 2019.
• Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science ,178, 471–479.
• Garfield, E. (1994). The impact factor. Current Contents https://clarivate.com/essays/impact-factor/ Accessed November 10, 2019.
• Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA, 295(1), 90.
• Hammarfelt, B. & Rushforth, A.D. (2017). Indicators as judgment devices: An empirical study of citizen bibliometrics in research evaluation. Research Evaluation, 26, 169-180.
• Liu, W., Hu, G., & Gu, M. (2016). The probability of publishing in the firstquartile fournals, Scientometrics, 106, 1273–1276.
• Miranda, R. & Garcia-Carpintero, E. (2019). Comparison of the share of documents and citations from different quartile journals in 25 research areas. Scientometrics 121, 479-501.
• Pajić, D., & Jevremov J. (2014) Globally national–Locally international: Bibliometric analysis of a SEE psychology journal. Psihologija, 47(2) 263-267. Peters, G.J.Y. (2017). Why not to use the journal impact factor as a criterion for the selection of junior researchers: A comment on Bornmann and Williams (2017). Journal of Informetrics, 11, 888-891.
• Seglen, P.O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ, 314, 498-502.
• Tang, L., Shapira, P., & Youtie, J. (2015). Is there a clubbing effect underlyning Chinese reseach citation increases? Journal of the Association for Information Science and Technology, 66(9), 1923-1932.
• Tang, L., & Hu, G. (2018). Evaluation woes: metrics beat bias. Nature, 559, 331.
• Testa, J. (2016). A view from Web of Science: Journals, articles, impact. Information Services & Use, 36(1-2) 99-104.
• Tonta, Y. (2017). Journals published in Turkey and Indexed in Web of Science: An Evaluation. Turkish Librarianship, 31(4) 449-482.
• Tregoning, J. (2018). How will you judge me if not by impact factor? Nature, 558, 345.
• TÜBİTAK, 2019. The Scientific and Technological and Technological Research Council of Turkey. Turkish Academic Network and Information Center, https://cabim.ulakbim.gov.tr/ubyt/ (in Turkish). Accessed November 10, 2019.
• URAP, (2019a). University Ranking by Academic Performance. Ranking Methodology. https://www.urapcenter.org/Methodology, Accessed November 10, 2019.
• URAP, (2019b). 2019-2020 URAP Turkey Ranking. http://tr.urapcenter.org/2019/2019-2020-URAP-Turkiye-Siralamasi-Raporu.pdf (in Turkish), Accessed November 10, 2019.
• ÜAK, 2019. Head of Inter-University Council. The panel for the assesment of the position of Associate Professor, http://www.uak.gov.tr/?q=node/68 (In Turkish). Accessed November 10, 2019.
• YÖK, 2019. Higher Education Council. Academic appointment and promotion criteria. https://www.yok.gov.tr/akademik/atanma-kriterleri (in Turkish). Accessed November 10, 2019.
• Van Noorden, R. (2010). Metrics: A profusion of measures. Nature, 465, 864-866.
• Van Raan, A.F.J. (2005). Fatal attraction: conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.
• Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10, 365-391.
• Waltman, L., & Traag V.A. (2017). Use of the journal impact factor for assessing individual articles need not be wrong. Preprint at arXiv: https://arxiv.org/ftp/arxiv/papers/1703/1703.02334.pdf Accessed November 10, 2019.
• Waltman L., & van Eck N.J. (2019). Field Normalization of Scientometric Indicators. In: Glänzel W., Moed H.F., Schmoch U., Thelwall M. (eds) Springer Handbook of Science and Technology Indicators. Springer Handbooks. Springer.
• Zhaou, P., & Lv, X. (2015). Academic publishing and collaboration between China and Germany in physics. Scientometrics, 105(3), 1875–1887.