Trent Joins Other Canadian Universities and withdraws from Maclean's

August 30, 2006

After careful consideration, Trent University has decided to join with other universities from across Canada and withdraw from the Maclean’s annual ranking survey. In explaining her decision, President Patterson emphasized, “Trent will continue to discuss with Maclean’s how their survey can be improved and remains committed to transparency and providing new and improved benchmarking data to parents, students and the public.” To see the full correspondence sent to Maclean’s and concerns about the survey methodology scroll down.

 

Letter to Maclean's Magazine

August 30, 2006

Mr. Tony Keller
Managing Editor, Special Projects
Maclean’s Magazine
One Mount Pleasant Rd.
Toronto, ON M4A 2Y5

Dear Mr. Keller:

After considerable discussion and consideration, Trent University has decided not to participate in the 2006 Maclean’s survey that results in a ranking edition of your magazine. There are two primary reasons for our decision to withdraw. First, as has been pointed out by many members of the sector over a number of years, the methodology used to produce a league table, ranking institutions within three major categories, does not hold up to analytical or statistical scrutiny. Secondly, as numerous competitor primarily undergraduate institutions in Ontario have indicated they will not be submitting the data requested, the value we receive from being able to benchmark ourselves using some of the more meaningful data elements has eroded. More recently, I understand that additional universities across Canada in our category have also withdrawn.

We would welcome the opportunity to participate in a dialogue with our colleagues and Maclean’s on an alternative to the current league tables in order to strengthen the value of the information you provide to prospective students and their families which support their decision making. It would also be important to review the considerable commitment and cost to institutions of submitting data as it is currently requested by Maclean’s.

The methodology used would need to address several of the concerns that have been expressed by the university community. Attached we have reiterated just a few examples of where methodology and league rankings based on your approach raise serious concerns for our institution. We hope you find them useful.

We believe it important to provide useful and accurate information to the public about our institution and indeed Trent will continue to do so by sharing both information and benchmarking data obtained through various other survey instruments on our website and in numerous annual publications.

I have gone on at some length, Tony, to help you and your colleagues understand just some of the challenges we find in the approach taken to the Maclean’s annual ranking. Let me reiterate what I have said to you on the telephone in earlier discussions. Trent University would remain open to participation in future surveys that Maclean’s will conduct once our concerns around your methodological issues in your rankings have been addressed and the workload involved in submitting data as you currently request it is reviewed. I encourage you to enter an open dialogue with colleague universities to address issues that would lead to an outcome mutually acceptable to all parties.

In the meantime, this fall we will be providing stakeholders interested in learning more about Trent University, additional information about our institution’s performance. Along with data sets developed in concert with Ontario universities, the public will have indicators from which to understand Trent’s performance and that of other institutions in this province. It is our belief that this approach will better inform students, their parents and the public about why they should consider joining and supporting our University community.

Yours truly,

Bonnie M. Patterson
President and Vice Chancellor

 

Example Concerns Regarding Maclean’s Methodology and Approach to Rankings

1. In the student body section of the ranking you rate the quality of new intakes by assuming all primarily undergraduate Universities have a homogenous distribution of intakes in programs of study. For example, in the 2004-05 academic year, a comparator primarily undergraduate institution had more than 30% of its new intakes in professional based programs, while Trent University had only 12% of its new intakes enrolled in those same professional based disciplines as defined by your magazine. As a result of enrolment limitations in these professional programs, universities are able to raise their admission standards and thus increase the quality of their intakes with entering grade point averages above 75%. This makes institutional benchmarking on grade point averages misleading to prospective students trying to assess the academic value and reputation of each University based on overall student composition.

2. Issues exist around the methodology used to assess University finances and expenditures per weighted full-time equivalent enrolment. As you are already aware, Trent University and other Ontario Universities have had to accommodate the double cohort in this province, thus reducing our operating expenses per student and adversely impacting our rankings due to economy of operation. Meanwhile, institutions outside of Ontario did not experience the same surge in enrolment that Trent and all other Ontario Universities have experienced over the past three academic years, thus their rankings have not been impacted adversely.

3. Methodology concerns centre around evaluating the quality of faculty research. Your ranking on faculty research performance only counts research awards from the federal government’s granting council agencies. As you are already aware, Research Infosource Inc. ranked Trent University for the third year in a row as number one in the Primarily Undergraduate research category. Their rankings are based on a variety of factors – half of the points awarded are based on financial indicators such as total sponsored research income, research income per full-time faculty position and research income per full-time graduate student. The remaining points were awarded based on the number of publications per full-time faculty member. Your methodology at best captures less than half of the research activity on campus. Using additional factors such as externally funded research from the private sector, the provincial government and graduate students allows a more accurate perspective on the research performance of our university.

4. A final concern is about the weighting system used in your methodology to generate Trent’s ranking as a tool to measure the overall experience that Trent University offers its students. By assigning weights in the various categories used in your methodology your magazine has developed an assessment tool to rank our University against others as an output measure of the student learning experience. However, should one view the results from the 2006 National Survey of Student Engagement (NSSE), one would find that students attending Trent think highly of their learning experiences. Data provides evidence of a strong supporting campus environment designed to help students achieve their ultimate goal of attaining a University degree. For example, from this instrument we have learned that our students are more engaged in their learning experiences than those at Ontario universities.