UNIVERSITY RANKINGS

OUR 16TH ANNUAL RANKINGS

We evaluated 47 Canadian universities, by the numbers and under the microscope

MARY DWYER November 13 2006
UNIVERSITY RANKINGS

OUR 16TH ANNUAL RANKINGS

We evaluated 47 Canadian universities, by the numbers and under the microscope

MARY DWYER November 13 2006

OUR 16TH ANNUAL RANKINGS

UNIVERSITY RANKINGS

We evaluated 47 Canadian universities, by the numbers and under the microscope

MARY DWYER

Nowinits l6th year, the annual Maclean’s rankings assess Canadian universities on a diverse range of factors from class sizes and average entering grades to spending, library volumes and faculty success in obtaining national research grants. The focus is on the undergraduate experience, and the intent is to offer an overview of the quality of instruction and services available to students at public universities across the country.

Maclean’s places universities in one of three categories, recognizing the difference in levels of research funding, the diversity of offerings, and the range of graduate and professional programs. Primarily Undergraduate universities are largely focused on undergraduate education, with relatively few graduate programs. Those in the Comprehensive category have a significant amount of research activity and a wide range of pro-

grams at the undergraduate and graduate levels, including professional degrees. Medical-Doctoral universities offer a broad range of Ph.D. programs and research. In addition, all universities in this category have medical schools, which sets them apart in terms of the size of research grants.

In each category, Maclean’s ranks the institutions on a range of factors—or performance indicators—in six broad areas (weightings are in parentheses). In total, Primarily Undergraduate universities are ranked on 22 performance measures, Comprehensive universities on 23 and Medical-Doctoral universities on 24. In reporting to Maclean’s, universities include data from all federated and affiliated institutions. The magazine does not rank schools with fewer than 1,000 fulltime students or those that are restrictive due to a religious or specialized mission.

The ranking process begins in the spring when a detailed questionnaire is circulated to the universities. At the same time, Maclean ’s collects information on dozens of student and faculty awards from 45 administering agencies. Thousands of reputational surveys

are also sent to high school principals and guidance counsellors, university officials, heads of organizations, CEOs and corporate recruiters across the country, asking for their views on quality and innovation at Canadian universities.

This year, the rankings were marked with controversy as 26 universities refused to make public performance indicators that they had released, as a matter of course, for the past 15 years. On Aug. 14—two weeks before the deadline for submitting completed surveys— 11 universities informed Maclean’s that they would not do so, citing concerns about the ranking methodology and some of the measures used, and claiming that over the years these concerns had “gone largely uñaddressed.” Fifteen other universities shortly followed suit. The rankings have always been contentious, however, since they place universities under the microscope. During the mid-1990s, a number of universities boycotted the exercise.

From its inception, Maclean’s has consulted

with academic experts about the design, composition and methodology of the rankings. After the inaugural rankings were released in 1991,Maclean’s responded to criticism from the university community by overhauling the survey. For several months in 1992, editors met with educational experts and

representatives, including

university presidents, institutional analysts and such organizations as the Association of Universities and Colleges of Canada and its regional counterparts, asking them for input on what factors to consider when assessing how well a university is doing its job. New indicators were developed and the universities were divided into the three peer-grouped categories. Over the next several years, Maclean’s continued to respond to concerns and suggestions from the post-secondary sector by adding three new indicators-total library holdings, international first-year students and retention rate—as well as significantly revising the class size indicators. Guidelines were refined, definitions were expanded, and the reputational survey increased the range and number of groups canvassed while boosting regional representation. Maclean’s continues the mandate it established 16 years ago: to provide basic, essential information in a comprehensive package to help students choose the university that best suits their needs.

In calculating the rankings this year, universities that completed a questionnaire were

processed with data being collated and checked in the usual manner. For universities that refused to complete a questionnaire,Maclean’s obtained public data wherever possible for the most recently available fiscal year. For the two research grant indicators, data for the fiscal year 2005-2006 was received directly from the three major federal granting agencies: the Social Sciences and Humanities Research Council of Canada, the Natural Sciences and Engineering Research Council of Canada, and the Canadian Institutes of Health Research. Statistics Canada data was used for the five financial indicators—operating budget, spending on student services, scholarships and bursaries, library expenses and acquisitions. The Canadian Association of Research Libraries and its regional counterparts provided figures used for the library holdings indicators. All financial and library figures are for fiscal year 2004-2005. As in previous years, Maclean’s conducted the reputational survey, and collected student and faculty awards for all universities. Any outstanding data for non-participating universities was taken from last year’s questionnaires.

The 26 universities that did not complete a ranking questionnaire this year are: University of Alberta, Brandon University, University of British Columbia, Brock University, University of Calgary, Carleton University, Concordia University, Dalhousie University, Lakehead University, Laurentian University, University of Lethbridge, the University of Manitoba, McMaster University, Université de Moncton, Université de Montréal, University of New Brunswick, Nipissing University,

University of Ottawa, Queen’s University, Ryerson University, Simon Fraser University, University of Toronto, Trent University, the University of Western Ontario, University of Windsor and York University.

STUDENT BODY (22% to 23% of final score) Students are enriched by the calibre of their peers. For that reason, Maclean’s collects incoming students’ average high school grades (ll%), and the proportion of those

with averages of 75 per cent or higher (2%). This count includes only those students whose secondary school averages or CÉGEP scores served as the basis of admission. No conversion formula is applied to incoming grade averages to adjust for provincial differences or varying admission policies, although CÉGEP grades are converted from an R score to a percentage grade. As well, it should be noted that certain universities, to enhance accessibility, accept students with lower grades. As a measure of drawing power, we count the proportion of out-of-province students in the first-year undergraduate class (l.5%). The percentage of international students in the first-year undergraduate class (0.5%) is measured as well, acknowledging the benefits such diversity brings to the classroom. The percentage of international students at the graduate level (l%) is also measured for Comprehensive and Medical-Doctoral universities.

WEIGHTINGS OF INDICATORS

Maclean’s ranks universities on 22, 23 or 24 performance measures, according to peer grouping, and then allocates the appropriate weights to those measures.

STUDENT BODY 22% TO 23%

Average Entering Grade 11%

Proportion with 75% or Higher 2%

Student Retention 2%

Proportion Who Graduate 2%

Out of Province (1st Year) 1.5%

International (1st Year) 0.5%

International (Graduate)* 1%

Student Awards 3%

CLASSES

17% TO 18%

Class Sizes: 1st

and 2nd Year Level 7% to 7.5%

Class Sizes: 3rd

and 4th Year Level 7% to 7.5%

Classes Taught by

Tenured Faculty 3%

FACULTY

Faculty with Ph.D.’s 3%

Awards per Full-time Faculty 3%

Social Sciences and

Humanities Grants 5.5%

Medical/Science Grants 5.5%

FINANCES

Operating Budget 3.3%

Scholarships & Bursaries 4.3%

Student Services 4.3%

LIBRARY

Total Library Holdings* 1%

Holdings per Student 3% to 4%

Acquisitions 4%

Expenses 4%

REPUTATION

Alumni Support 3%

Reputational Survey 16%

•Comprehensive and Medical-Doctoral categories only +Medical-Doctoral category only

In taking stock of retention rates (2%), Maclean’s asks for the percentage of first-year students who return in second year, either full-time or part-time. While many factors can affect a student’s choice not to returnpersonal considerations, or a decision to transfer to a program unavailable at their home uni-

versity-student retention, on the whole, reflects a university’s success in keeping its students on course. Maclean’s also measures graduation rates (2%): the percentage of fulltime undergraduates in their second year who go on to graduate from the institution within one year of the expected time period.

In addition, Maclean’s collects data on the success of the student body at winning national academic awards (3%) over a five-year period. The list of roughly 40 fellowships and prizes includes such prestigious awards as the Rhodes Scholarships, the Fulbright Awards, as well as scholarships from professional associations and the three federal granting agencies.

CLASSES (17% to 18%) The rankings take stock of the entire distribution of class sizes at the firstand second-year levels (75% for Primarily Undergraduate universities, 7% for the other two categories), as well as the thirdand fourth-year levels (75% for the Primarily

Undergraduate category, 7% for the others). Maclean’s measures the percentage of students in classes in each of the following classsize ranges: 1 to 25; 26 to 50; 51 to 100; 101 to 250; 251 to 500; 501 and higher.

Maclean’s also ranks universities on the percentage of first-year classes taught by tenured and tenure-track professors (3%).

FACULTY (17%) In assessing the calibre of faculty, Maclean’s calculates the percentage offaculty with Ph.D.’s (3%), and the number who win national awards, including the distinguished Killam, Molson and Steacie prizes, the Royal Society of Canada awards, the 3M Teaching Fellowships and almost 40 others (3%). In addition, the magazine measures the success of eligible faculty in securing research grants from each of the three major federal granting agencies: Social Sciences and Humanities Research Council of Canada, Natural Sciences and Engineering Research Council of Canada, and the Canadian Institutes of Health Research. Maclean’s takes into account both the number and the dollar value received last year. Social sciences and humanities grants (5.5%) and medical/science grants (5.5%) were tallied as separate indicators.

FINANCES (12%) This section examines the amount of money available for current expenses per weighted full-time-equivalent student (3.3%), as well as the percentage of the budget spent on student services (4-3%) and scholarships and bursaries (4.3%). Funds used to pay off debt are excluded from the analysis.

LIBRARY (12%) This section assesses the breadth and currency of the collection. Universities received points for the number of volumes and volume equivalents per number of full-time-equivalent students (4% for Primarily Undergraduate and Comprehensive, 3% for Medical-Doctoral). The total holdings measurement was used in the Medical-Doctoral category (l%), acknowledging the impor-

tance of extensive on-campus collections in those universities.

As well, Maclean’s measured the percentage of a university’s operating budget that was allocated to library services (4%) and the percentage of the library budget spent on updating the collection (4%). In acknowledging a shift from the traditional library model—books on shelves—to an electronic access model, Maclean’s captures spending on electronic resources in both the library expenses and acquisitions measurements.

REPUTATION (19%) This section reflects a university’s reputation in the community

at large, as well as with its own graduates. For the reputational survey (16%), respondents rated the universities in three categories: Highest Quality, Most Innovative and Leaders of Tomorrow. Best Overall represents the sum of the scores.

When looking at alumni support, institutions received points for the percentage of alumni giving gifts—rather than the total value of gifts—over the past five years (3%). M

NEW ON THE WEB Customize your own ranking using the new Personalized University Ranking Tool at www.madeans.ca/university. Choose your own weights and criteria from our exclusive database of 24 indicators.