Performance-Based Research Funding (PBRF)
Traditionally, government research funding at postsecondary institutions has been distributed based on enrollments of equivalent full-time students. This system is currently being phased out in favor of performance-based funding, similar to Britain’s Research Assessment Exercise, through an evaluation and funding system known as Performance-Based Research Funding (PBRF), which is administered by the Tertiary Education Commission [1] (TEC). By rewarding ‘high-quality’ research institutions with ‘top-up’ funding, the TEC hopes to improve the quality of academic research in New Zealand.
As defined by the TEC,
… the primary goal of the PBRF is to ensure that excellent research in the tertiary education sector is encouraged and rewarded. This entails assessing the research performance of tertiary education organisations and then funding them on the basis of their performance.
The institutional assessment is based on three criteria:
- The quality of academics’ research outputs – 60 percent of the fund.
- The number of research degrees completed per institution – 25 percent of the fund.
- The external research income of each tertiary education institution – 15 percent of fund.
The major element, the Quality Evaluation, is held periodically. The first was completed in 2003 and the second, a partial round, is being held this year (2006). The next full round is scheduled for 2012. In 2003, just half of 45 eligible institutions participated in the assessment.
Methodology
The PBRF rankings are based on the Quality Evaluation, which consists of two main elements. The first is a review of portfolios, containing publications and other research outputs, put together by PBRF-eligible staff. The portfolios receive a passing grade from ‘A’ to ‘C’, or a failing grade of ‘R’. Of the 8,013 portfolios submitted, 5,771 received a grade of ‘C’ or better and went on to be reviewed in the second stage, peer review.
In this second stage, portfolios were assessed by panels of experts in 12 subject-area groupings, which in total comprised 165 academic experts. Each portfolio was scored on a scale of 0-7 according to three criteria: research output, peer esteem, and contribution to the research environment. Each criterion was weighted 70/15/15 respectively and awarded a grade from ‘A’ to ‘R’. Just 5.7 percent of the 8,013 faculty to have their research assessed received an ‘A,’ and almost two-fifths were awarded an ‘R’. Subject-area quality scores were used to rank subject areas and institutions.
Results
| Evaluating Research Excellence: the 2003 Assessment | ||
| Tertiary Education Institution | Quality Score | FTE Staff Assessed |
| University of Auckland | 3.96 | 1,411.80 |
| University of Canterbury | 3.83 | 590.1 |
| Victoria University of Wellington | 3.39 | 579.3 |
| University of Otago | 3.23 | 1.174.9 |
| University of Waikato | 2.98 | 536.3 |
| Lincoln University | 2.56 | 195.3 |
| Massey University | 2.11 | 1.225.8 |
| Carey Baptist College | 1.16 | 8.6 |
| Anamata | 1 | 2 |
| Bible College of New Zealand | 0.83 | 17.9 |
| Auckland University of Technology | 0.77 | 567.7 |
| Unitec Institute of Technology | 0.71 | 345.8 |
| Auckland College of Education | 0.39 | 174.2 |
| Whitecliffe College of Arts and Design | 0.36 | 16.29 |
| Te Whare Wananga o Te Pihopatanga o Aotearoa | 0.35 | 11.5 |
| Waikato Institute of Technology | 0.32 | 108 |
| Te Wanange o Aotearoa | 0.32 | 67.2 |
| Dunedin College of Education | 0.27 | 66.4 |
| AIS St. Helens | 0.22 | 18.2 |
| Christchurch College of Education | 0.2 | 170.4 |
| Wellington College of Education | 0.03 | 109.7 |
| Bethlehem Institute of Education | 0 | 17.2 |
| Average | 2.59 | – |
Source: Tertiary Education Commission [2]
All quality scores are out of a maximum total score of 10. The overall scores seem very low, ranging from just below 40 percent to zero. It is pointed out in the report, however, that it is unreasonable to expect an institution to achieve anywhere near 10, as all PBRF-eligible staff at the institution would have to receive an ‘A’ quality category for this to happen.