Disciplinary Differences in Academics' Perceptions of Performance Measurement at Nordic Universities

Abstract

As performance measurement has become increasingly common at Nordic universities, concerns have been raised that disciplinary differences create difficulties in comparing academic performances. To better understand the potential of utilising performance measures for the management of academic work, this study explores how academics perceive governance and steering based on performance measurement. Building on an established typology of the disciplines that distinguishes the hard sciences from the soft and the applied sciences from the pure, we ask how academics perceive performance measurement depending on their disciplinary affiliation. The empirical material consists of a survey sent to academics in four Nordic countries. Our results show there are clear differences in the attitudes toward performance measurement between academics from different disciplines. Academics from the hard applied sciences are more positive about performance measurement than any other group, and academics from the soft pure sciences are more negative. These findings are consistent with notions about the poor adaptation of metrics to publication practices within the soft sciences and greater sensitivity to performance measurement among the applied sciences. The main contribution of the article is to provide empirical data that support the notion that performance measures are accepted to different degrees in different disciplines.

Keywords

• Disciplines,performance measurement,Nordic universities

References

1. Aagaard, K. (2015). How incentives trickle down: Local use of a national bibliometric indicator system. Science and Public Policy, 42(5), 725-737.
2. Ahola, S., Hedmo, T., Thomsen, J.-P., & Vabø, A. (2014). Organisational features of higher education: Denmark, Finland, Norway & Sweden. Oslo: NIFU. Accessed (June 1, 2020): https://www.nifu.no/en/publications/1162368
3. Aksnes, D. W., & Rip, A. (2009). Researchers’ perceptions of citations. Research Policy, 38(6), 895-905.
4. Amaral, A., Meek, L., & Larsen, I. M. (Eds.). (2003). The higher education managerial revolution?. Dordrecht: Springer.
5. Becher, T. (1994). The significance of disciplinary differences. Studies in Higher Education, 19(2), 151-161.
6. Becher, T., & Trowler, P. R. (2001). Academic tribes and territories. Buckingham: Open University Press.
7. Biglan, A. (1973a). The characteristics of subject matter in different academic areas. Journal of Applied Psychology, 57(3), 195-203.
8. Biglan, A. (1973b). Relationships between subject matter characteristics and the structure and output of university departments. Journal of Applied Psychology, 57(3), 204-213.

9. Bleiklie, I. (1998). Justifying the evaluative state: New public management ideals in higher education. Journal of Public Affairs Education, 4(2), 87-100.
10. Bornmann, L. (2014). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics, 8(4), 895-903.
11. Buela-Casal, G., & Zych, I. (2012). What do the scientists think about the impact factor?. Scientometrics, 92(2), 281-292.
12. Campbell, J. P., & Oblinger, D. G. (2007). Academic Analytics. EDUCAUSE Quarterly. Accessed (September 01, 2019): https://library.educause.edu/-/media/files/library/2007/10/pub6101-pdf.pdf
13. Christensen, T. (2011). University governance reforms: potential problems of more autonomy?. Higher Education, 62(4), 503-517.
14. Clark, B. R. (1998). Creating entrepreneurial universities: Organizational pathways of transformation. Bingley: Emerald.
15. Cole, S. (1983). The hierarchy of the sciences?. American Journal of Sociology, 89(1), 111-139.
16. Deem, R., Hillyard, S., Reed, M., & Reed, M. (2007). Knowledge, higher education, and the new managerialism: The changing management of UK universities. Oxford: Oxford University Press.
17. Deephouse, D. L., & Suchman, M. (2008). Legitimacy in organizational institutionalism. In R. Greenwood, C. Oliver, R. Suddaby, & K. Sahlin-Andersson (Eds.), The Sage Handbook of Organizational Institutionalism (pp. 49-77). London: Sage.
18. Donovan, C. (2007). The qualitative future of research evaluation. Science and Public Policy, 34(8), 585-597.
19. El‐Khawas, E. (1996). One professoriate, or many?: Assessing aspects of differentiation among academics. Tertiary Education & Management, 2(2), 146-152.
20. Elzinga, A. (1997). The science-society contract in historical transformation: With special reference to “epistemic drift”. Social Science Information, 36(3), 411-445.
21. Elzinga, A. (2012). Features of the current science policy regime: Viewed in historical perspective. Science and Public Policy, 39(4), 416-428.
22. Espeland, W. N., & Sauder, M. (2007). Rankings and reactivity: How public measures recreate social worlds. American Journal of Sociology, 113(1), 1-40.
23. Ferlie, E., Musselin, C., & Andresani, G. (2008). The steering of higher education systems: A public management perspective. Higher education, 56(3), 325-348.
24. Fanelli, D., & Glänzel, W. (2013). Bibliometric evidence for a hierarchy of the sciences. PLoS One, 8(6), e66938.
25. Fägerlind, L., & Strömqvist, G. (Eds.). (2004). Reforming higher education in the Nordic countries: Studies of change in Denmark, Finland, Iceland, Norway and Sweden. Paris: International Institute for Educational Planning. Accessed (June 1, 2020): http://www.iiep.unesco.org/en/reforming-higher-education-nordic-countries-studies-change-denmark-finland-iceland-norway-and-sweden
26. Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., & Trow, M. (Eds.). (1994). The new production of knowledge: The dynamics of science and research in contemporary societies. London: Sage.
27. Gläser, J., & Laudel, G. (2007). The social construction of bibliometric evaluations. In R. Whitley & J. Gläser (Eds.), The changing governance of the sciences: The advent of research evaluation systems, (pp. 101-123). Dordrecht: Springer.
28. Godin, B. (2006). The linear model of innovation: The historical construction of an analytical framework. Science, Technology, & Human Values, 31(6), 639-667.
29. Gornitzka, Å., Stensaker, B., Smeby, J-C., & De Boer, H. (2004). Contract arrangements in the Nordic countries—solving the efficiency/effectiveness dilemma?. Higher Education in Europe, 29(1), 87-101.
30. Gulbrandsen, M., & Slipersaeter, S. (2007). The third mission and the entrepreneurial university model. In A. Bonaccorsi & C. Daraio (Eds.), Universities and strategic knowledge creation: Specialization and Performance in Europe (pp. 112-143). Cheltenham: Edward Elgar Publishing.
31. Hammarfelt, B., & de Rijcke, S. (2015). Accountability in context: Effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the Faculty of Arts at Uppsala University. Research Evaluation, 24(1), 63-77.
32. Hammarfelt, B., & Rushforth, A. D. (2017). Indicators as judgment devices: An empirical study of citizen bibliometrics in research evaluation. Research Evaluation, 26(3), 169-180.
33. Hammarfelt, B., Nelhans, G., Eklund, P., & Åström, F. (2016). The heterogeneous landscape of bibliometric indicators: Evaluating models for allocating resources at Swedish universities. Research Evaluation, 25(3), 292-305.
34. Hansen, H. F., Geschwind, L., Kivistö, J., Pekkola, E., Pinheiro, R., & Pulkkinen, K. (2019). Balancing accountability and trust: university reforms in the Nordic countries. Higher Education, 78(4), 557-573.
35. Harvey, L. (2008). Rankings of higher education institutions: A critical review. Quality in Higher Education, 14(3), 187-207.
36. Hedges, L. V. (1987). How hard is hard science, how soft is soft science? The empirical cumulativeness of research. American Psychologist, 42(5), 443-455.
37. Henkel, M. (2000). Academic identities and policy change in higher education. London: Jessica Kingsley Publishers.
38. Hicks, D. (2004). The four literatures of social science. In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of Quantitative Science and Technology Research (pp. 473-496). Dordrecht: Springer.
39. Hood, C. (1991). A public management for all seasons?. Public Administration, 69(1), 3-19.
40. Kallio, K. M., & Kallio, T. J. (2014). Management-by-results and performance measurement in universities–implications for work motivation. Studies in Higher Education, 39(4), 574-589.
41. Kallio, K. M., Kallio, T. J., Tienari, J., & Hyvönen, T. (2016). Ethos at stake: Performance management and academic work in universities. Human Relations, 69(3), 685-709.
42. Kuhn, T. (1962) [Reprinted 2012]. The structure of scientific revolutions. Chicago: The University of Chicago Press.
43. Mingers, J., & Willmott, H. (2013). Taylorizing business school research: On the ‘one best way’ performative effects of journal ranking lists. Human Relations, 66(8), 1051-1073.
44. Neumann, R. (2001). Disciplinary differences and university teaching. Studies in Higher Education, 26(2), 135-146.
45. OECD. (2007). Revised field of science and technology (FOS) classification in the Frascati manual. Accessed (June 1, 2020): https://www.oecd.org/science/inno/38235147.pdf
46. Osterloh, M. (2010). Governance by numbers. Does it really work in research?. Analyse & Kritik, 32(2), 267-283.
47. Pinheiro, R., Geschwind, L., & Aarrevaara, T. (Eds.). (2016). Mergers in higher education: The experience from Northern Europe. Cham: Springer.
48. Pinheiro, R., Geschwind, L., Hansen, H. F., & Pulkkinen, K. (Eds.). (2019). Reforms, organizational change and performance in higher education: A comparative account from the Nordic Countries. Cham: Palgrave Macmillan.
49. Paradeise, C., Reale, E., Bleiklie, I., & Ferlie, E. (Eds.). (2009). University governance: Western European comparative perspectives. Dordrecht: Springer.
50. Piwowar, H. (2013). Altmetrics: Value all research products. Nature, 493(7431), 159.
51. Pollitt, C., & Bouckaert, G. (2004). Public management reform: A comparative analysis. Oxford: Oxford University Press.
52. Porter, T. M. (1995). Trust in numbers: The pursuit of objectivity in science and public life. Princeton: Princeton University Press.
53. Reale, E., & Seeber, M. (2011). Organisation response to institutional pressures in Higher Education: The important role of the disciplines. Higher Education, 61(1), 1-22.
54. Slaughter, S., & Leslie, L. L. (1997). Academic capitalism: Politics, policies, and the entrepreneurial university. Baltimore: The Johns Hopkins University Press.
55. Smith, L. D., Best, L. A., Stubbs, D. A., Johnston, J., & Archibald, A. B. (2000). Scientific graphs and the hierarchy of the sciences: A Latourian Survey of Inscription Practices. Social Studies of Science, 30(1), 73-94.
56. Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598-642.
57. Stichweh, R. (2009). Differentiation of scientific disciplines: Causes and consequences. In G. H. Hadorn (Ed.), Unity of knowledge in transdisciplinary research for sustainable development, Volume 1 (pp. 82-90). Oxford: EOLSS Publishers/UNESCO.
58. Stoecker, J. L. (1993). The Biglan classification revisited. Research in Higher Education, 34(4), 451-464.
59. Stokes, D. E. (1997). Pasteur's Quadrant: Basic science and technological innovation. Washington, DC: Brookings Institution Press.
60. Sugimoto, C. R., & Weingart, S. (2015). The kaleidoscope of disciplinarity. Journal of Documentation, 71(4), 775-794.
61. Söderlind, J., & Geschwind, L. (2019). Making sense of academic work: The influence of performance measurement in Swedish universities. Policy Reviews in Higher Education, 3(1), 75-93.
62. Trowler, P. (2014). Depicting and researching disciplines: Strong and moderate essentialist approaches. Studies in Higher Education, 39(10), 1720-1731.
63. van Raan, A. F. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133-143.
64. van Vught, F., & Westerheijden, D. F. (2010). Multidimensional ranking. Higher Education Management and Policy, 22(3), 1-26.
65. Whitley, R. (2007). Changing governance of the public sciences: The consequences of establishing research evaluation systems for knowledge production in different Countries and scientific fields. In R. Whitley & J. Gläser (Eds.), The changing governance of the sciences: The advent of research evaluation systems (pp. 3-27). Dordrecht: Springer.