They are always controversial, but one of the most common complaints about university rankings is that they use surveys to rate the reputation of institutions. Such measures are based on style over substance, the critics say, and do little to recognise up-and-coming universities, or those without a famous name.
But carried out with proper care, reputation surveys can form an important part of a serious ranking, helping to provide a valuable tool for students, academics and policymakers alike.
A university’s reputation for excellence in teaching and research is a top priority for students who will invest tens of thousands of dollars in their degree. Reputation may be subjective, but students need to know their degree certificate has a brand that carries respect in the global job market.
And if you ask the right people the right questions, a reputation survey can provide valuable information.
For its 2011-12 World University Rankings, due out this Autumn, Times Higher Education magazine will draw on the views of more than 17,500 academics from 137 countries who took part in the survey carried out by our rankings data provider Thomson Reuters.
Each respondent was invited to take part because they were statistically representative of both their country, based on United Nations data, and of their academic discipline, no volunteers were allowed. Some 28% of respondents were from Europe and 18% were from Asia and the Middle East. Views from the Indian subcontinent will be properly balanced against those in the traditional superpowers of the West.
The survey asks respondents to judge institutions in their specialist fields, based on direct, personal experience. They are asked, for example, where they would send a highly talented undergraduate student for the best postgraduate teaching.
A reputation survey should be one that students can trust. But reputation is just part of the picture. Despite the faith placed in it, the survey is used in just two of at least 13 separate performance indicators that are used to build the THE World University Rankings. (The writer is also deputy editor of Times Higher Education magazine in the UK)