On this page:
The Melbourne Institute of Applied Economic and Social Research
In November 2004, the Melbourne Institute of Applied Economic and Social Research [6] released a unique report [7] ranking Australian universities in terms of their international competitiveness and standing, as gauged by quantitative data and subjective data taken from leaders in both domestic and international higher education. The Melbourne Institute, in designing its study, has attempted to respond to a number of common criticisms leveled against ranking initiatives around the world. Most notably it has introduced an interesting methodological element designed to respond to the arbitrary nature by which quality indicators are normally weighted (see below).
The report — known as the Melbourne Institute Index — also seeks to provide suggestions as to what factors contribute to improving an institution’s international standing. The ultimate goal of the report and its rankings is to help Australian universities understand where they stand in relation to one another, and in relation to their international counterparts, so that efforts can be focused on improving their status and ability to compete more effectively for the best students and scholars worldwide. A second report was issued in November 2005.
Methodology
The results of the Institute’s survey are based on a mix of subjective opinions and quantitative data. The subjective findings are derived from two separate surveys, one sent out to deans at 39 Australian universities, and the other to 172 chief executive officers (vice-chancellors, presidents, rectors) from the world’s ‘best’ overseas universities (as identified by the Shanghai Jiaotong rankings). Quantitative data is derived from five different indicators: quality/international standing of staff, quality of graduate programs, quality of undergraduate entry, quality of undergraduate programs, resource levels.
The aim of the surveys is twofold: to introduce an element of peer review to the overall ranking and to establish the weighting that should be applied to the six overall indicators (with the survey results acting as the sixth indicator).
University CEOs from around the world were asked to rate each Australian university on a scale of 1 to 5 as compared to institutions in their continent. Australian deans (and New Zealand vice-chancellors) were asked to rate institutions in comparison to U.S. universities. The scales were calibrated using the results of the Shanghai study (for example, top 50 US = top 25 Europe= top 5 Asia = top 80 world.) Replies were received from 40 CEOs and 80 deans. Australian deans and New Zealand vice-chancellors were also asked to rank the top 10 Australian universities in order using as the criteria “international standing”.
Weighting
In response to a common criticism of university rankings — that criteria weightings are arbitrarily chosen by the researchers — all respondents were also asked what weights they would attribute to the six indicators when measuring international standing. The average weights suggested by foreign CEOs and Australian deans were very closely correlated and are as follows:
- 40 %: quality/international standing of staff
- 16 %: quality of graduate programs
- 11 %: quality of undergraduate entry
- 14 %: quality of undergraduate programs
- 11 %: resource levels
- 8%: opinions of educationists
Scores for the five categories of quantitative data, which account for 92 percent of the overall rank, were established using previously published data. The international standing of academic staff was determined according to the volume and quality of their research output using data on publications, citations of those publications, peer recognition, and research income as tabulated by the Institute for Scientific Information [8] over a ten-year period from 1994 to 2004. The quality of graduate programs was assessed by tabulating doctoral completion and progression rates in addition to student review of their doctoral programs. At the undergraduate level, universities were assessed according to student tertiary entry scores, staff-student ratios, progression rates, continuation into higher studies, and student evaluations. Finally, resource levels were judged on a per student basis and a per staff member basis.
Results
Overall Ranking (2004) | |||
Rank | University | Group | Weighted Score |
1 | Australian National University | Go8 | 100 |
1 | University of Melbourne | Go8 | 100 |
3 | University of Sydney | Go8 | 95 |
4 | University of Queensland | Go8 | 87 |
5 | University of New South Wales | Go8 | 85 |
6 | Monash University | Go8 | 76 |
6 | University of Western Australia | Go8 | 76 |
8 | University of Adelaide | Go8 | 70 |
9 | Flinders University of South Australia | IRUA | 56 |
10 | La Trobe University | IRUA | 55 |
11 | Macquarie University | IRUA | 54 |
12 | University of Tasmania | – | 53 |
13 | University of Newcastle | IRUA | 52 |
14 | Murdoch University | IRUA | 51 |
15 | University of Wollongong | – | 50 |
16 | Curtin University of Technology | ATN | 49 |
16 | Griffith University | IRUA | 49 |
16 | Queensland University of Technology | ATN | 49 |
19 | Deakin University | – | 47 |
19 | University of New England | – | 47 |
19 | University of Technology, Sydney | ATN | 47 |
22 | James Cook University | – | 46 |
22 | Swinburne University of Technology | – | 46 |
24 | University of South Australia | ATN | 44 |
25 | RMIT University | ATN | 43 |
26 | University of Canberra | NGU | 42 |
27 | Charles Darwin University | – | 41 |
27 | Edith Cowan University | NGU | 41 |
27 | Victoria University | NGU | 41 |
30 | Charles Sturt University | – | 39 |
30 | Southern Cross University | NGU | 39 |
30 | University of Western Sydney | NGU | 39 |
33 | University of Ballarat | NGU | 38 |
34 | Australian Catholic University | NGU | 37 |
34 | Central Queensland University | NGU | 37 |
36 | University of Southern Queensland | NGU | 36 |
37 | University of Notre Dame, Australia | – | 32 |
37 | University of the Sunshine Coast | NGU | 32 |
Source: The Melbourne Institute of Applied Economic and Social Research
Key: Go8: Group of Eight Universities; IRUA: International Research Universities Australia; ATN: Australian Technological Network; NGU: New Generation Universities.
Results of the 2005 report are available from: http://melbourneinstitute.com/publications/reports/MelbIndex.pdf [7]
Other Resources
Australian Universities Network
Australian Universities Network [9] provides a number of different rankings. Other than the Melbourne Institute Index, the tables are largely drawn from international rankings such as the Times Higher Education Supplement World University Rankings and a number of different international business school rankings.
Student Outcome Indicators for the Learning and Teaching Performance Fund
Also included is a league table ranking teaching performance as published in the Australian newspaper on August 12, 2005. The data is drawn from the Department of Education, Science and Training’s “Student Outcome Indicators for the Learning and Teaching Performance Fund,” which was commissioned to reward best-performing teaching universities with supplemental funding from the federal budget.
Hobsons Guides
Hobsons Guides publishes an annual ratings guide to Australian universities called the Good Universities Guide. Hobsons rates universities across a number of different indicators on a five-star scale. The findings are available to paid subscribers, although the methodology is outlined to non-subscribers. Most of the data is drawn from publicly available Department of Education, Science and Technology indicators.