Cover

Ranking Canadian universities

The Maclean’s ranking takes a measure of the undergraduate experience at Canada’s public universities.

ANN DOWSETT JOHNSTON,MARY DWYER November 18 2002
Cover

Ranking Canadian universities

The Maclean’s ranking takes a measure of the undergraduate experience at Canada’s public universities.

ANN DOWSETT JOHNSTON,MARY DWYER November 18 2002

Ranking Canadian universities

Cover

UNIVERSITY RANKINGS | Methodology

The Maclean’s ranking takes a measure of the undergraduate experience at Canada’s public universities. ANN DOWSETT JOHNSTON and MARY DWYER explain the methodology behind the annual survey.

Maclean’s places each university in one of the following three categories:

How the process unfolds

It has now been 11 years since Maclean’s launched its first survey of Canadian universities, a top-to-bottom ranking of 46 schools. That 1991 ranking was controversial, to say the least: while it sold out in a matter of days, the university community reacted with howls of outrage. According to many, the methodology was flawed. In 1992, editors at Maclean’s began a five-month process, travelling across the country to meet with university officials and educational experts. The challenge? To design a more sophisticated institution-wide ranking, a prototype to build on for years to come. What emerged were three rankings, comparing universities in peer categories. Thanks to the enormous involvement of the university community at large, the new rankings offered a much clearer window into the ivory tower—a window that has become increasingly important ever since, as the landscape has shifted.

The Maclean’s ranking is now a year-round exercise, and we remain grateful for the input and wise counsel of members of the university community. One of the hallmarks of the exercise has been continued dialogue with our partners, the universities themselves. Together, we have improved the exercise on an regular basis. This year, Maclean’s made a significant adjustment to the classsize indicators, one that better captures the reality of the student experience.

How is the ranking done? In June, we circulate a 17-page questionnaire to the universities. At the same time, more than 7,500 reputational surveys are sent out to a broader community across the country, and information on student and faculty awards is collected from the administering agencies. Throughout the fall, editors spend several weeks collating and checking the data. Consulting statisticians from McDougall Scientific Ltd. perform the final calculations and produced the ranking.

How we place the universities in peer groups

Using such factors as research funding, diversity of offerings and the range of PhD programs to define peer groupings,

PRIMARILY UNDERGRADUATE

Universities largely focused on undergraduate education, with relatively few graduate programs.

COMPREHENSIVE

Universities with a significant amount of research activity and a wide range of programs at the undergraduate and graduate levels, including professional degrees.

MEDICAL DOCTORAL

Universities with a broad range of PhD programs and research, as well as medical schools.

In reporting to Maclean’s, universities include all federated and affiliated institutions. The magazine does not rank schools with fewer than 1,000 full-time students or those with a strictly religious or specialized mission.

How we evaluate

The universities in the three categories are treated as separate but equal. Maclean’s ranks the schools on a range of factors in six broad groupings (weightings are in parentheses). In total, Primarily Undergraduate universities are ranked on 21 performance mea-

Digging in at the library at the University of New Brunswick

sures, Comprehensive universities on 22 and Medical-Doctoral universities on 23—resulting in slightly different weightings for some performance measures.

STUDENT BODY (21% to 22% of

final score)

Students are enriched by the input of their peers. For that reason, Maclean’s collects the incoming students’ average high-school grades (11%), and the proportion of those with averages of 75 per cent or higher (3%).

This count includes only those students whose secondary-school averages or CEGEP scores served as the basis of admission. Mature students, for example, are excluded. As well, it should be noted that certain universities, in the spirit of accessibility, accept students with lower grades.

As a measure of drawing power, we count the proportion of out-of-province students in the first-year undergraduate class (1.5%). The percentage of international students in the first-year undergraduate class (0.5%) is measured as well, acknowledging the growing initiative to attract students from abroad and the benefits such diversity brings to the classroom. The percentage of international students at the graduate level (1%) is also measured for Comprehensive and Medical-Doctoral universities.

The student-body section also includes graduation rates (2%): the percentage of full-time undergraduate students in their second year who go on to graduate from

the institution within one year of the expected time period. In addition, Maclean’s j collects data on the success of the student body at winning national academic awards (3%) over the past five years.

CLASSES (17% to 18%)

The rankings embrace the entire distribution of class sizes at the firstand second-year ¡ levels (7.5% for Primarily Undergraduate : universities, 7% for the other two categories), 1 as well as the thirdand fourth-year levels ! (75% for the Primarily Undergraduate category, 7% for the others). Class-size ranges j are as follows: 1 to 25; 26 to 50; 51 to 100; 101 to 250; 251 to 500; 501 and higher.

This year, Macleaïi’s made a significant change to the methodology employed. Previously, the number of classes in each classsize range was counted. Now, in order to better reflect the students’ experience in the classroom, we are counting the number of students in classes in each class-size range, ;

Macleans also ranks universities on the percentage of first-year classes taught by tenured and tenure-track professors (3%), a measure of how much access new students have to top faculty.

FACULTY (17%)

The rankings assess the calibre of faculty by calculating the percentage of those with PhDs (3%), and the number who win national awards (3%). In addition, the magazine measures the success of eligible faculty in securing grants from each of the three major federal granting agencies: Social Sciences and Humanities Research Council of Canada, Natural Sciences and Engineering Research Council of Canada, and the Canadian Institutes of Health Research, as well as the Canada Council. Maclean’s takes into account both the number and the dollar value received last year. Social sciences and humanities grants plus Canada Council grants

(5.5%) and medical/science grants (5.5%) were tallied as separate indicators.

FINANCES (12%)

This section examines the amount of money available for current expenses per weighted full-time-equivalent student (3.3%), as well as the percentage of the budget spent on student services (4.3%) and scholarships and bursaries (4.3%). When presenting their general operating budget, institutions deducted any funds used to pay off debt.

LIBRARY (12%)

This section assesses the breadth and currency of the university’s collection. Schools received points for the number of volumes and volume equivalents per number of fulltime-equivalent students (4% for Primarily Undergraduate and Comprehensive, 3% for Medical-Doctoral). An additional indicator, measuring total holdings, regardless of student numbers, was used in the MedicalDoctoral category (1%) to acknowledge the importance of extensive on-campus collections in those universities. As well, Maclean’s measured the percentage of a university’s operating budget that was allocated to library services (4%) and the percentage of the actual library budget that was spent on updating the collection (4%).

In acknowledging a shift from the traditional library model to an access model, Maclean’s captures spending on electronic resources in both the library expenses and acquisitions measurements.

REPUTATION (20%)

This section reflects a university’s reputation with its own graduates, as well as within the community at large. For its reputational survey (15%), Maclean’s sent surveys to 7,528 individuals across the country. Respondents rated the universities in three categories: Highest Quality, Most

Reputational Response Rate

THIS YEAR, Maclean ’s solicited the opinion of 7,528 individuals across the country. They included high-school guidance counsellors from every province and territory, university officials at each ranked institution, the heads of a wide variety of national and regional organizations, plus CEOs and recruiters at corporations large and small. As in recent years, Maclean’s expanded the list of corporate representation, with particular attention to regional balance. The reputational survey is both regional and national in character, dividing the country into four key areas: the western provinces, Ontario, Quebec and the Atlantic provinces. All respondents completed a national survey; university officials and guidance counsellors also completed regional surveys.

Innovative and Leaders of Tomorrow. Best Overall represents the sum of the scores.

When looking at alumni support, institutions received points for the number— rather than the value—of gifts to the university over the past five years (5%). lifl

Weightings

Maclean’s ranks universities on up to 23 performance measures with varying

weights assigned to each.

STUDENT BODY 21% to 22% Average Entering Grade_11% Proportion with 75% or Higher_3% Proportion Who Graduate_2% Out of Province (1st Year)_1.5% International (1st Year)_0.5% _International (Graduate)*_1% Student Awards 3% CLASSES_17% to 18% Class Sizes: 1st and 2nd Year Level 7 to 7.5% Class Sizes: 3rd and 4th Year Level 7 to 7.5% Classes Taught by Tenured Faculty 3% FACULTY_17% Faculty with PhDs_3% Awards per Full-time Faculty_3% Social Sciences and Humanities Grants 5.5% Medical/Science Grants_5.5% FINANCES_12% _Operating Budget_3.3% Scholarships & Bursaries_4.3% Student Services 4.3% LIBRARY_12% _Total Library Holdingst_1% Holdings per Student_3 to 4% Acquisitions_4% Expenses_4% REPUTATION_20% Alumni Support_5% Reputational Survey_15% ♦Comprehensive and Medical-Doctoral categories only fMedical-Doctoral category only GROUP RESPONSE RATE University Officials_44.4% Guidance Counsellors_11% CEOs_11.1% Corporate Recruiters_9.6% Heads of Organizations_8.5% TOTAL 13.5%