Daniel Gunnarsson and U-G Nilsson:
When the latest URANK was presented, it was with great anticipation that we looked at in what place our own library had come. And whoops! That was not a stimulating read… We had slipped down from 5th place to the 21st! Sure, they had changed assessment values, but the newer one were supposed to suit us better, that’s what we thought anyway.
Without trying to defend ourselves, one could take a closer look at the figures. What is it that they measure? Do they really measure what they are supposed to measure? Why are they measuring this in particular, and what is the purpose of URANK? Without going into too much detail, we just want to comment on URANK from a librarian’s point of view, however limited this might seem.
Without quality aspects
Firstly, what seems to be the main focus of interest in the assessment of libraries seems to be input (i.e. means) only, not attained quality, not even the scope of the activities of the library organization! They could choose to focus more on other aspects of the organization, for example, loans, downloads of full text material, number of library visitors, and all this in proportion to the number of full-time students, or to obtained means.

Incomparable values
The measured input is based on data that is not at all comparable. The first variable measures means obtained from the mother organization. Some libraries have study counselors, welfare officers and others, who are included in their budget, whereas some do not. Libraries divided into several branches demand increased resources for this kind of activities. Since staff constitutes a large amount of the budget, university libraries, which are usually smaller organizations, will show greater differences when compared with each other. The second variable in the assessment concerns the number of students? Not full-time students or whole-year achievements. This means that a university with only programme students and no courses gets better ranking, since acquisitions of resources are based on fewer individuals.
Alternative measurements
We then made some tests by carrying out some alternative measurements with considerations taken to the commentaries above, and compared our library with those ranked higher on URANK’s list. And certainly, we are doing very well in such comparisons when you look at how well we’ve managed to administer the means by loans carried out, downloads, visits at the library, in proportion to full-time students, and adjusted for means not normally within the scope of ordinary library activities.

Improved data
The research library statistics that we produce and make public via Swedish Statistics (SCB) is used, and will be used, for different kinds of purposes. As professionals, we know that it’s not comparable between different libraries. We are aware of the fact that we are measuring different things. We even know that we may measure them differently from year to year. Accordingly, we have a responsibility to stop this inaccurate production of statistics until we can guarantee reliable data.
Reliability and validity
It’s more or less a fact that ranking lists are here to stay. However, one could demand more from the methods that are used. Since we’re teaching our students scientific methods, including reliability and validity and a critical attitude, we, as academics, should perhaps be a bit careful not to produce and embed inaccuracies which will then be considered “true”. We know that URANK writes a great deal about the existing uncertainties and flat-out inaccuracies, but if they are so widespread, we can’t see why they’re not trying to find more comparable variables. If not, it would perhaps be better not to present such a deficient ranking list.