Assessing the assessors of research

There is no reason to pick on this press release from the University of St Andrews beyond the fact that it is a peg to talk about the 2008 UK Research Assessment Exercise. St Andrews is just one of many universities that has alighted on the RAE as a way of promoting themselves. You will find similar tales from Edinburgh, Cambridge and doubtless many other places where the mortar board isn’t strange headgear.

The press release from of St Andrews tells us that the university “has emerged as 14th in the UK for the quality of all its research across Science and the Arts and has significantly expanded its complement of world class research.” But visit the RAE’s own web site and you will find no league table. For that you will have to consult the various media outlets that have taken the time to put the RAE’s data into their spreadsheets.

Times Higher Education has done its usual thorough job on the numbers. (The readers’ comments there are particularly entertaining.) You can even retrieve the magazine’s spreadsheets to make your own league tables.

By the way, the THE’s league table puts St Andrews 14th equal, alongside Durham, Sheffield,  Southampton, Leeds and Bristol.

The press release from Cambridge shows just what the RAE does put out. “The University of Cambridge submitted 2,040 researchers to the RAE. 31.7 per cent of submissions were in the 4* category (world-leading) while 39.2 per cent were in the 3* (internationally excellent).”

By coincidence, perhaps, Cambridge appears at number two in the THE’s tables. But the institution above it, the Institute of Cancer Research, is a specialist organisation with just 97 people submitted for “two units of assessment” against 2040 in 50 “UoAs” for Cambridge. So the Fenland mob could have made all sorts of grand claims.

Does this mean anything for the wider world of R&D? The RAE has changed over the years so that it does not, as it did in earlier versions, penalise researchers who work with business. Indeed, you are now supposed to get brownie points for such activities. But it is hard to tell if any company in search of academics would turn to the RAE as a source of wisdom.

One place to look for clues is in the tables of subjects that the THE has assembled. So, for example, Imperial College heads the table for “Civil engineering” while Cambridge is number one in “General engineering and mineral & mining engineering”.

Cambridge is also ahead of the rest when it comes to “Computer science and informatics,” an area where there is plenty of business interest. Imperial College comes second in this subject. Than again, so do Southampton and Edinburgh.

Perhaps there should be an exercise  to assess the user friendliness of universities as seen through the eyes of R&D people in business. If personal comments, or rather gossip over a glass of wine, are anything to go by, there would not be a direct mapping on to the RAE’s tables. Indeed, the higher a university is in that league, the lower it could be in an assessment of “user friendliness”.