Judging Universities by How Free They Are
Restrictions on academic freedom come in many shapes and forms. Sometimes they are crude and obvious; sometimes they are soft and subtle.
When the Khmer Rouge took power in Cambodia in 1975, as part of their brutal “year zero” campaign, they suspended all academic activities, destroyed school buildings and libraries, and killed thousands of professors, teachers and students. This complete wipeout of the academic sector is one end of a very broad spectrum of restrictions on academic freedom.
Closer to the other end of the spectrum, we would find the conflicts of interest that may arise from corporate influence over universities – when they turn to private money to sustain their operations. For example, in Cologne, Germany, a pharmaceutical company finances graduate programs on medical research, which has raised suspicions about the company’s interference in research and teaching contents.
Those two scenarios – the abolition of everything deemed ‘intellectual’ and the subtle influence of corporate money – may seem hard to compare with one another. But this is precisely what my colleagues, Katrin Kinzelbach, Ilyas Saliba, and I are trying to achieve by creating a comparative global index of academic freedom. The attempt to do justice to the nuances between such disparate scenarios as Cambodia in the 1970s and Germany today is one of five key challenges in devising such a measurement. Before turning to these five challenges and how we are addressing them, let’s first consider why such a global measurement is so needed in the first place.
Academic Excellence vs. Academic Freedom
You might say we have several well-established university rankings: the Times Higher Education ranking, the QS World Rankings, the Shanghai ranking, for example. They rate universities across the world for their excellence in research and teaching – why do we need a separate measurement of academic freedom?
Apart from the widely known methodological shortcomings and biases of these rankings, the key problem is that academic freedom is distinct from academic excellence. Think of Nazi Germany when institutions of higher education were under tight control by the government. Nevertheless, they produced several Nobel laureates in the 1930s and 40s. Similarly, the limited scientific autonomy and intellectual freedom in the Soviet Union did not prevent their pioneering of space exploration in the 1950s. And in more recent times, we see rapid technological progress coming from an increasingly authoritarian China. In fact, a lot of staggering scientific research and innovation has resulted from extreme government pressure – in particular to gain the upper hand over enemy technology in times of war.
The fact that university rankings measure academic excellence but not freedom has two types of consequences:
First, it means that universities in countries that extensively violate academic freedom do not experience negative effects to their international reputation as a consequence. In a way, these rankings make it not only tolerable but – at times – even rewarding to repress the freedom of scholars and students. Such reverse incentive structures should be of great concern to the international academic community. And indeed, the fact that the Bologna process has taken up this issue proves that there is an increased awareness and sense of responsibility around this issue.
Second, the total lack of comparable data on academic freedom prevents us as researchers and practitioners from studying these phenomena in more depth. Indeed, not only university rankings have failed to take up this issue – even existing democracy or human rights indices are so far not collecting data on academic freedom in any meaningful way. Having such data would go such a long way towards improving our understanding of academic freedom infringements. For example, with the right kind of data, we could explore to what extent academic excellence does relate to academic freedom. We could also investigate dynamics in the restriction and exercise of academic freedom across time and space. Maybe we can identify certain determinants that generally explain the level of academic freedom, or maybe, in a time series, we can find out about a possible sequencing of the repression of academics with other rights infringements.
But even before we address any more elaborate research questions, such global data would simply improve our knowledge of the state of academic freedom across the world – a knowledge that is still very limited today. A global measurement on academic freedom is very much needed today.
In fall 2017, my colleagues and I convened an expert conference in Germany to discuss possible strategies forward, and the challenges we would need to overcome. We invited experts to talk about different kinds of methodologies that could be used. Others gave us insights from a variety of country contexts covering the spectrum from limited to systematic restrictions on the university sector. Since then, after continuous work on the subject, we formally started a pilot project earlier this year. The project’s goal is to develop a methodology and to collect initial data.
This endeavor faces several challenges:
The Challenge of Defining Academic Freedom
First of all, there is no legally binding international definition of “academic freedom”. There is not even an authoritative definition that is widely agreed upon. Furthermore, the concept in its positive form is very elusive. By positive form, I mean the freedom to do – the freedom of academics to carry out their work in a self-determined way: How can one measure freedom? So, inspired by existing measures of other freedoms and rights, we decided to look at academic freedom in its negative form, meaning the freedom from interference. Strictly speaking, we are not measuring academic freedom, but rather the degree to which there are infringements of academic freedom.
What exactly are we looking at? While there is no clear definition, there are a number of elements that are generally accepted as being part of or closely linked to academic freedom.
- The freedom to research and teach is one of them. Are scholars free to determine their own research agenda and their teaching curricula? That is, of course, within the limits of the quality standards set by the academic community itself.
- The exchange with other academics in the research process, uncensored access to research material, and the publication of findings both within academia and for the wider public, are other key ingredients of academic freedom.
- Many studies also emphasize the aspect of institutional autonomy, which can be regarded as a necessary precondition of academic freedom: Are universities de facto exercising autonomy over their internal governance, including budgeting, hiring, student admissions, and so on?
- A further precondition of academic freedom that we found important to include is what we call ‘campus integrity’. What we mean by that is the absence of a climate of intimidation through securitization, targeted physical threats or oppressive surveillance on campus. We have seen that such practices are widespread in many countries and can have stark effects on academic activities.
So, without committing ourselves to a particular definition of academic freedom, we have set out to measure the extent to which these different elements are restricted or not.
The Data Challenge
The second measurement challenge relates to the fact that no data source exists on these issues that cover countries across the world.
There are several methods that have been applied to gather data in some geographical areas – mostly in Europe. These include self-reporting mechanisms like the EUA’s Autonomy Scorecard, surveys among academics or students, or legal analyses of countries’ protection of academic freedom. However, if we were to widen the scope of these studies to a global level, or even a wider European level, we would encounter a number of problems.
In repressive countries, the discrepancies between the de jure and the de facto situation are often substantial, so that a purely legal analysis risks capturing a very misleading picture. Just because a constitution stipulates that academic freedom is protected doesn’t mean that scholars aren’t intimidated in practice. Surveys among academics and students can give a good impression of their situation and opinions in an open country context. But as researchers who have worked extensively on repressive countries, we know that manipulation and self-censorship would very likely distort both the selection of participants and the survey results. Furthermore, conducting surveys in repressive countries always raises serious ethical questions, as even a survey request might put people at unforeseeable risk. Lastly, self-reporting can only meaningfully be applied to relatively factual questions, and even these can easily be manipulated if done in bad faith.
A fourth method that has been applied globally is the collection of so-called events data. That means that incidents of repressive events against scholars or students are documented, as done by Scholars at Risk in their Academic Freedom Monitor. However, from the literature on conflict data and my own experience with human rights case data, we know that events data have critical limitations. First of all, such data focus exclusively on events of repression and cannot pay attention to institutional restrictions that are already in place. For example, the politically biased dismissal of a faculty member would be captured, but not a systematic government control over hiring. Secondly, at the international level, we only learn of a fraction of the cases that are really occurring around the world – and it is a heavily biased selection on top of that. We are more likely to hear about a professor receiving death threats in Western Europe than about one whose class was suspended in South-East Asia. Events data typically capture only the tip of the iceberg. For these reasons, they are also unfit to paint a representative and comprehensive picture of global restrictions on academic freedom.
This is not to say that all of these methods are useless in a global context, quite to the contrary. Legal analyses, for example, can be extremely useful in furthering our understanding of the de jure situation of universities across the world. And events data, even if incomplete and selective, can be of great illustrative value, and indicative of academic communities that are particularly under attack.
For a global measurement design, we needed a different approach. After much deliberation – and the consideration of some other possible methods – we found that the most promising approach was to rely on expert assessments.
Expert surveys have been used for many years in political science endeavors, in particular in the context of democracy measurements. In its most condensed form it can deliver a numerical country score that indicates how well the country is doing. This is achieved by asking experts to rate the degree of compliance or repression on a predefined scale. One of the clear advantages of assessments by country experts is that they are able to integrate in-depth analyses, to contextualize de jure protections with observations of the de facto situation, and to include data from other methods like survey results or events data wherever they are available.
However, it is important to emphasize that we believe that there is not one single method that can easily do the trick. Expert assessments have their own methodological shortcomings. An important challenge is the dependency on the expertise and integrity of all the experts involved. A second problem is to ensure the comparability of the data, as different experts might interpret measurement standards differently: some people may see the glass as half full when others see it as half empty. So, the data become messy and problematic if similar situations are rated differently. Lastly, expert assessments, especially if done well, are relatively expensive and logistically demanding. I will come back to these issues below.
Comparing the Apparently Incomparable
The third challenge is the one that I started with: how to devise a comparative measurement between situations that seem so utterly incomparable – between the Khmer Rouge’s “year zero” approach and Bayer’s funding of graduate programs. Perhaps the Khmer Rouge example is historically unique and unlikely to be repeated; but closures and purges of universities to subordinate them to political will seem to be a relatively common phenomenon, in particular after radical regime changes. So how do we address the challenge of doing justice to such widely differing situations?
First, I want to make clear that in comparative measurements, there is an inherent tension between the level of comparability and the level of complexity that can be achieved. In other words: We need to simplify in order to compare. This is always unsatisfactory, because simplifying means losing information, losing variation, losing context. So comparative measurements are a lot about striking the right balance between information loss and comparability. This is particularly true when it comes to quantitative measurements in the form of scores or rankings. Their distinct strength is the ease of comparison between countries. However, without the necessary level of complexity, these scores are relatively meaningless and do not actually bring us closer to understanding the underlying issue.
We approach this dilemma in two ways:
Firstly, it is very helpful to understand academic freedom as a composite concept that consists of several distinct elements like the ones I mentioned earlier. Different country cases show us that restrictions on academic freedom not only vary widely in severity of infringements. Repressive actors also employ a very diverse mix of methods of interference. Scholars might have the freedom to determine what they want to research, but they could be strongly limited in what they can or cannot communicate to the public. Universities might enjoy complete autonomy from the state, while their scholars are influenced by corporate money on what they teach their students. So, on a basic level, by breaking the measurement of academic freedom down into different elements and measuring them separately from each other, we can depict some of the complexity of restrictions on the academic sector.
In addition, when asking experts to assess these different aspects, they not only determine whether the situation is good or bad, but they evaluate the given situation using a scale that has several levels: ranging from very severe restrictions to the absence of restrictions. Taken together, the different dimensions and scaling levels thus allow us to capture a relatively broad variety of scenarios.
However, as handy as quantitative scores might be when comparing countries, there is a lot of important information and variance that simply cannot be captured by this type of method. It will only give us a general idea of the degree to which academic freedom is respected in a given country, in a particular year.
For this reason, we do not rely exclusively on the quantitative approach in our pilot project, but will complement it by qualitative case studies. Here, too, we are developing a research protocol that seeks to establish some level of comparability between the studies. But through the use of a narrative approach, these studies will of course allow for a much greater degree of complexity, detail and historic context. In the long run, ideally, we would have historic and recent country-year scores at a global level as well as a growing number of case studies on individual countries.
These two methods are mutually beneficial. For example, the quantitative scores will allow researchers to detect recurring or unusual patterns, which can then be further explored in case studies. As for the practical use of these tools: those who represent universities or ministries which are committed to the cause of academic freedom could use this quantitative tool to identify countries with problematic track records of infringements. If such an institution seeks to collaborate with partners in one of these countries, it could refer to the case study for further detail, to help assess risks, inform their strategy, and develop appropriate measures for cooperation agreements.
Dealing With Within-Country Variations of Academic Infringements
Furthermore, there can be a lot of variation in the degree and type of infringements on academic freedom within a country itself. Be it between different institutions: private or public, rural or urban, historically critical of the government or not; or between subject areas: natural sciences or social sciences, established fields of study or newer ones. These within-country variations represent the fourth key challenge of measuring academic freedom on a global scale.
The variation between subject areas is indeed a crucial part of academic freedom infringements. Looking at examples of authoritarian governments: typically the social sciences are under stricter control by the state. In contrast, natural sciences are more easily exposed to the influence of corporate money. We firmly believe it is important to assess the integrity of the academic community as a whole, and that it would be dangerous to excuse or relativize the infringements on some subjects by the freedom of others. At the same time, the quality of restrictions on the academic sector as a whole is different depending on whether only some or all disciplines are targeted. For these reasons, we decided to include the scope of interference across disciplines as part of our measurement.
Measuring the variation on a subnational or university level, on the other hand, is unfortunately not realistic within the scope of our pilot project, especially if we want to reach global coverage. University excellence rankings do focus on the institutional level, and for good reason. But at the same time, this generally means that only high-ranking universities draw attention, resulting in the fact that many countries are even completely off the map. This should not happen with a measurement of academic freedom. At this stage of the methodological design, we therefore focus on the country level and ask experts to generalize across universities. By doing so, we can at least find out what the prevailing practices and restrictions are. I should add, though, that the qualitative case studies allow us to account for variations at the subnational level in more detail.
From Design to Data: Implementing the Measurement
The question of what is realistically possible brings me to the fifth and last challenge of designing a global measurement: Namely, the data collection has to be practical and feasible, both within the context of our pilot project, and for the measurement to continue in the future.
I mentioned earlier that expert assessments are expensive and logistically demanding. At the Global Public Policy Institute, where our project is based, we have the methodological and substantial expertise to design the questions for an expert survey; and we have a network of committed experts and practitioners who are advising us in this process. But what we do not have is the necessary infrastructure to carry out a global expert survey in a sound and credible way.
For this reason, we decided to collaborate with V‑Dem. V‑Dem, or Varieties of Democracy, is a well-established democracy measurement project based at the University of Gothenburg in Sweden. At V‑Dem, they have the necessary infrastructure for and experience in carrying out such a survey. This includes a broad network of experts across the world. It also includes a statistical approach that helps to cross-verify ratings and to reduce biases in the results. What is more, all their data, including instructions given to experts, are transparent and publicly accessible and usable.
We are currently at the stage of finalizing our partnership with V‑Dem, including the exact formulation of the survey questions on academic freedom which will be integrated in their next cycle.
If all goes well, we will have global data on five questions relating to academic freedom by spring 2020. The ratings will be available on a country-year basis reaching back to the early 20th century. On this basis, we – and others – will be able to develop a comprehensive picture of the state of academic freedom across the world. And we will be able to explore some of the more complex research questions around academic freedom which were outlined earlier.
In addition, as previously mentioned, we are preparing a methodology to carry out complementary case studies. This methodology will be tested and refined with a handful of pilot countries, and the results presented within a year from now.
Beyond the Data
This brings me to my last point: the synergies that I see between our measurement project and current political efforts around the issue of academic freedom, in particular the Bologna monitoring endeavor. In their Paris Communiqué last year, the ministers of the European Higher Education Area (EHEA) made a strong commitment to protect and promote academic freedom in their higher education spaces. This commitment and the envisaged monitoring effort is a very timely endeavor. It rightly puts the issue of academic freedom on the political agenda – even if it focuses on the European Higher Education Area, it will have a signaling effect not only to countries in this area, but also to the rest of the world.
The success of university excellence rankings shows that the higher education sector is very reputation-sensitive. I strongly believe that collectively we can contribute to a shift in reputation criteria; a shift to a situation where academic freedom constitutes a necessary building block of universities’ reputation. This can only be done by systematically monitoring restrictions on academic freedom, by exposing unacceptable practices, and by increasing the overall knowledge around the protection of academic freedom in the world.
Our quantitative data, as well as our case study methodology, will be released within one year from now. This means that it will be available within the timeframe when the Bologna Follow-up Group is devising a possible monitoring mechanism. I am convinced that the data we collect on European countries could very usefully feed into this monitoring effort.
I want to emphasize that the current Bologna process is a unique opportunity to create a monitoring structure that is embedded in an existing institutional – and, of course, also political – process. The key advantage of this embeddedness is that we can set up an incentive structure for countries and universities to better protect academic freedom. I believe, however, that the data collection itself should not be placed under the control of EHEA member states. Rather, it should be carried out by academic institutions that can ensure that academic standards are upheld in the process.
There is not one ideal method to evaluate academic freedom and there are a number of limitations to our measurements. We excluded possible data sources due to the methodological problems attached to them at a global, and already at a wider European level. But we also deliberately decided to focus on a type of data that can bring something new to the table: New data that can be used in combination with all other types of data wherever available. New data that strike a reasonable compromise between levels of abstraction and worldwide coverage. New data that allow us to look at restrictions of academic freedom as a global phenomenon that requires global action.
I personally believe that establishing a global comparative measurement of academic freedom will be a great and very important achievement. But data alone will not improve academic freedom. A global measurement tool is only one step, and the new data will tell us just how much work there remains to be done.
…
This text is adapted from a keynote speech given on June 25, 2019 at the conference Bologna Process Beyond 2020: Fundamental values of the EHEA.
Related
