Cookies on this website
We use cookies to ensure that we give you the best experience on our website. If you click 'Continue' we'll assume that you are happy to receive all cookies and you won't see this message again. Click 'Find out more' for information on how to change your cookie settings.

‘Competing for excellence: Perverse and constructive effects of evaluation machines in academia’ | Prof Paul Wouters, University of Leiden | 3 June, Nuffield College (17.00-18.30)

Abstract

Many universities have become obsessed with performance indicators and their position in the global university rankings. This holds for education, but also for research. Evaluation has become formalized in national and local assessment protocols and is usually performed one or more steps removed from the primary process of education or research, often by evaluation professionals and managers. At the same time, research has become more competitive at a global scale, also in fields where competition was less prominent only a few decades ago. As a result, research leaders increasingly need to think strategically about their research portfolio and profile. However, the data they have at their disposal are often ill suited to their needs. Moreover, the type of evaluation they are interested in may not always be aligned with the existing formalized evaluation protocols.

In my talk, I will discuss the current tensions in the way researchers are being evaluated and assessed against the background of a short history of research evaluation and the rise of performance indicators in academia. I will introduce the concept of "evaluation machines" as developed by the Danish economist Peter Dahler-Larsen to understand the dynamics behind disconnected assessment practices. I aim to discuss how both a desire to develop science for the people and a drive to develop a knowledge-based economy and innovation have shaped a new context in which the old culture of peer trust was no longer the best available option for all purposes.

In the last part of my talk, I would like to discuss some possible scenarios for the future of knowledge creation in relation to accountability regimes in academia. I will also explore how we may wish to define excellence and scientific quality and will briefly sketch a possible alternative theory of how scientific quality might be defined.

Biography

Paul Wouters is a professor in scientometrics at the University of Leiden, and the director of the university's interdisciplinary Centre for Science and Technology Studies (CWTS). His previous experience includes acting as a leader of the group for analysing the development of information technologies for research, both as research instruments and as new communication media, at the Royal Netherlands Academy of Arts and Sciences, followed by a five-year appointment as the programme leader for the Academy's Virtual Knowledge Studio. He was also a visiting professor of cybermetrics at the University of Wolverhampton.

Wouters is member of the editorial boards of Social Studies of Science, the Journal of the American Society for Information Science and Technology, Scholarly Communication, and Cybermetrics. He has been member of the council of the Society for the Social Study of Science and is currently member of the board of the Dutch graduate school Science, Technology, and Modern Culture (WTMC). He is involved in the European project NESSHI and is PI of the 7th Framework project ACUMEN.

Download presentation

Watch and listen