dr. J.S. Drost (Jacqueline)

PhD Thesis: The Irony of Rankings

Since the 1980s a worldwide trend towards increased accountability and transparency has been responsible for the becoming of our ‘Audit Society’ (Power, 1994). Such a society is characterised by the desire to measure and evaluate performances of social institutions. In order to respond to society’s transparency and accountability requirements, social institutions were encouraged to translate the quality of their services delivered to the public into quantified numbers through the use of rankings: Instruments that make invisible complex and hidden information visible, accessible, understandable and ready for evaluations. Consequently, overwhelming productions of rankings in society unfolded.

The international character of this trend has been made visible by the various magazines, newspapers and other publishing agencies around the world who publish rankings about the quality of services provided by social institutions. Not only in the USA, where since 1987 the famous U.S. News & World Report has annually been publishing law school rankings, but also in The Netherlands, where the Dutch publisher New Skool Media publishes a ranking report of the ‘Best Dutch Hospitals’ in Elsevier Weekblad every year. Given the international proliferation of rankings there has been considerable research interest in rankings ever since.

Scholars from many academic disciplines have intensively studied rankings by adopting various ontologies, engaging with several theoretical frameworks, and using different methodologies to provide a better understanding of the many unintended consequences of rankings in society. An academic ranking debate unfolded in which academic ranking scholars (1) shared their critique towards the representational function of rankings (e.g. Espeland & Stevens, 1998); (2) emphasised the capacity that rankings have in changing people’s cognition and behaviour that results into unintended consequences for organisational transformations (e.g. Sauder & Lancaster, 2006; Sauder & Espeland, 2006; Espeland & Sauder, 2007; Espeland & Stevens, 2008); and (3) showed how rankings are performative actors in shaping strategies, markets, organisations and their practices (e.g. Kornberger & Carter, 2010; Mehrpouya & Samiolo, 2016; Pollock & D’Adderio, 2012; Scott & Orlikowski, 2012; Orlikowski & Scott, 2014, Wallenburg et al., 2019).

Analysing the academic ranking debate unfolded into knowledge on the current state of ranking studies. It has provided insights in the general research area of rankings and it has highlighted why studying rankings has become interesting, important and relevant, specially with regard to its representational problems and unintended reactive and performative consequences. Besides, analysing the academic ranking debate has also provided insights in what has mainly left unexplored with regard to ranking research. More specifically, it has unfolded into four new research directions that could enhance our knowledge on rankings: (1) Studying the production of rankings (Orlikowski & Scott, 2014; Espeland & Sauder, 2016; Mennicken et al, 2018); (2) Studying the reconfiguration of rankings (Orlikowski & Scott, 2014); (3) Studying the performative consequences for organisational transformations unfolding from the production and reconfiguration of rankings (e.g. Sauder & Espeland, 2006; Sauder & Lancaster, 2006; Espeland & Sauder, 2007; Pollock & D’Adderio, 2012; Mehrpouya & Samiolo, 2016, Wallenburg et al., 2019) and (4) Studying the production and reconfiguration of rankings and their performative consequences for organisational transformations from an ontology of becoming (Orlikowski & Scott, 2014).

Additionally, and as has been suggested by Orlikowski & Scott (2014), our knowledge on the production and reconfiguration of rankings and their implications for organisations could be best enhanced by engaging with Barad’s (2003, 2007) conceptualisation of performativity since her work ‘offers a distinctive capacity for inquiring into material enactments and their implications’ (Orlikowski & Scott, 2014, p. 888). Barad’s (2003, 2007) work on performativity finds its fundament in an ontology of becoming. Such an ontology assumes that the world is not fixed, but comes into being in a specific form

with a specific meaning and is continuously reconfigured into a new form or a new meaning. As such, adopting such an ontology could enable me to study the becoming - production - and reconfiguration of rankings in an organisation - a local ‘world’. However, sociomaterial scholars, who have adopted an ontology of becoming to empirically study the becoming of the world, argue that necessary performative methodological knowledge is limited (see specifically the work of Cecez-kecmanovic et al., (2014) and Hultin (2019)). Consequently, Cecez-kecmanovic et al., (2014) and Hultin (2019) call for more methodological experimentation when adopting an ontology of becoming. Such a call opens up the opportunity to methodologically experiment how to study the production and reconfiguration of rankings and their performative consequences for organisational transformations by adopting an ontology of becoming. Hence, in this PhD research I follow these calls and suggestions to join and contribute to the academic ranking debate.

Research questions

In this PhD research I aim to furthering our knowledge on rankings by studying the performativity of rankings at a rheumatology ward of a local Dutch hospital from an ontology of becoming. More specifically:

1. I aim to develop a methodological framework that suits an ontology of becoming and enables me to study the performativity of rankings from such an ontology;

2. Based on this framework I aim to study the performativity of rankings by exploring (1) the production of rankings; (2) reconfiguration of rankings, and (3) how rankings through their production and reconfiguration produce performative consequences for organisational transformations at a rheumatology ward of a local Dutch hospital.

The overall research question and the related subquestions I pose are:

How can an ontology of becoming contribute to our understanding of rankings? How can rankings be studied from an ontology of becoming?

In order to answer these research questions I have developed a methodological framework that I call ‘The Framework of Diffractivity’, which consists of: (1) A refined conceptualisation of Barad’s (2003, 2007) notion on performativity called ‘Diffractivity’; and (2) a translation of Gullion’s (2018) notion of a ‘Diffractive Ethnography’ in concrete research method with a detailed 5-Step Approach including two analytical approaches.

Since the performativity conceptualisation of Diffractivity centralises the notion of diffractions (simply translated as transformations in material form and meaning) through which local sociomaterial practices are produced and continuously reconfigured in a specific form with a specific meaning, engaging with this conceptualisation enables me to conceptualise rankings as a sociomaterial practice. Moreover, developing a concrete research method including two analytical approaches, enables me to ethnographically investigate, explore and map transformations that not only produces and reconfigures rankings, but also those that unfold into transformations of the form and meaning of daily care practices at the rheumatology ward of a local Dutch hospital.

Hence, in this thesis I not only aim to enhance our knowledge on rankings, but I also aim to furthering our methodological understanding and practices when we engage with an ontology of becoming.