Cover

HOW GRADS GRADE THEIR SCHOOLS

ANN DOWSETT JOHNSTON November 15 2004
Cover

HOW GRADS GRADE THEIR SCHOOLS

ANN DOWSETT JOHNSTON November 15 2004

RANKING CANADIAN UNIVERSITIES

Cover UNIVERSITY RANKINGS

Methodology

ANN DOWSETT JOHNSTON and MARY DWYER explain the methodology behind the annual Maclean's survey

The Maclean’s rankings offer students and parents a rare view into the ivory tower—and a unique opportunity to make informed decisions on the comparative strengths of public universities across Canada. On behalf of those readers, we ask a number of key questions related to the undergraduate experience. Where are the brightest first-year students? Where will you find the smallest classes? Where are you most likely to have access to tenured faculty? Where will you find the richest library resources? Which university has made the largest commitment to student services or scholarships and bursaries? Which university has the best reputation for quality and innovation? We ask all this, and much more.

How the ranking is done

In June, we circulated a detailed questionnaire to the universities. At the same time, we sent out thousands of reputational surveys across the country, and collected information on student and faculty awards from the administering agencies. Throughout the fall, editors spent several weeks collating and checking the data. Consulting statisticians from McDougall Scientific Ltd. performed the final calculations.

How we place the universities in peer groups

Using such factors as research funding, diversity of offerings and the range of Ph.D. programs to define peer groupings, Maclean’s places each university in one of three categories. Primarily Undergraduate universities are largely focused on undergraduate education, with relatively few graduate programs. Those in the Comprehensive category have a significant amount of research activity and a wide range of pro-

grams at the undergraduate and graduate levels, including professional degrees. MedicalDoctoral universities offer a broad range of Ph.D. programs and research, as well as medical schools.

In reporting to Maclean’s, universities include all federated and affiliated institutions. The magazine does not rank schools with fewer than 1,000 full-time students or those with a strictly religious or specialized mission.

How we evaluate

The universities in the three categories are treated as separate but equal. Maclean’s ranks the schools on a range of factors in six broad groupings (weightings are in parentheses). In total, Primarily Undergraduate universities are ranked on 22 performance measures, Comprehensive universities on 23 and Medical-Doctoral universities on 24—resulting in slightly different weightings for some performance measures.

STUDENT BODY

(22% to 23% of final score) Students are enriched by the calibre of their peers. For that reason, Maclean’s collects the incoming students’ average high-school grades (11%), and the proportion of those with averages of 75 per cent or higher (2%). This count includes only those students whose secondary-school averages or CEGEP scores served as the basis of admission. With the exception of CÉGEP scores, no conversion formula is applied to incoming grade averages to adjust for provincial differences or varying admission policies.

As a measure of drawing power, we count the proportion of outof-province students in the firstyear undergraduate class (1.5%). The percentage of international students in the first-year undergraduate class (0.5%) is measured as well, acknowledging the benefits such diversity brings to the classroom. The percentage of international students at the graduate level (1%) is also measured for Comprehensive and Medical-Doctoral universities.

The student-body section also includes graduation rates (2%): the percentage of fulltime undergraduate students in their second year who go on to graduate from the institution within one year of the expected time period. In addition,Macleans collects data

on the success of the student body at winning national academic awards (3%) over the past five years.

Last year, Maclean’s introduced a new indicator measuring the percentage of first-year students who return in second year, either full-time or part-time (2%). While many factors can affect a student’s choice not to return-personal considerations, or a decision to transfer to a professional program unavailable at their home university—student retention, on the whole, reflects a university’s success in keeping its students on course.

CLASSES (17% to 18%)

The rankings embrace the entire distribution of class sizes at the firstand second-year levels (7.5% for Primarily Undergraduate universities, 7% for the other two categories), as well as the thirdand fourth-year levels (7.5% for the Primarily Undergraduate category, 7% for the others). Maclean’s measures the percentage of students in classes in each of the following class-size ranges: 1 to 25; 26 to 50; 51 to 100; 101 to 250; 251 to 500; 501 and higher.

Maclean’s also ranks universities on the percentage of first-year classes taught by tenured and tenure-track professors (3%), a measure of how much access new students have to top faculty.

FACULTY (17%)

The rankings assess the calibre of faculty by calculating the percentage of those with Ph.D.s (3%), and the number who win national awards (3%). In addition, the magazine measures the success of eligible faculty in securing grants from each of the three major federal granting agencies: Social Sciences and Humanities Research Council of Canada, Natural Sciences and Engineering Research Council of Canada, and the Canadian Institutes of Health Research, as well as the Canada Council. Maclean’s takes into ac-

count both the number and the dollar value received last year. Social sciences and humanities grants plus Canada Council grants (5.5%) and medical/science grants (5.5%) were tallied as separate indicators.

FINANCES (12%)

This section examines the amount of money available for current expenses per weighted full-time-equivalent student (3.3%), as well as the percentage of the budget spent on student services (4.3%) and scholarships and bursaries (4.3%). When presenting their general operating budget, institutions deducted any funds used to pay off debt.

LIBRARY (12%)

This section assesses the breadth and currency of the collection. Universities received points for the number of volumes and volume equivalents per number of full-time-equivalent students (4% for Primarily Undergraduate and Comprehensive, 3% for MedicalDoctoral). The total holdings measurement was used in the Medical-Doctoral category (1%), acknowledging the importance of extensive on-campus collections in those universities. As well, Maclean’s measured the percentage of a university’s operating budget that was allocated to library services (4%) and the percentage of the library budget spent on updating the collection (4%). In acknowledging a shift from the traditional library model to an access model, Maclean’s captures spending on electronic resources in both the library expenses and acquisitions measurements.

REPUTATION (19%)

This section reflects a university’s reputation with its own graduates, as well as within the community at large. For the reputational survey (16%), respondents rated the universities in three categories: Highest Quality, Most Innovative and Leaders

of Tomorrow. Best Overall represents the sum of the scores.

When looking at alumni support, institutions received points for the number—rather than the value—of gifts to the university over the past five years (3%). 171

WEIGHTINGS

Maclean’s ranks universities on up to 24 performance measures with varying

weights assigned to each.

STUDENT BODY_22% to 23%

Average Entering Grade_11%

Proportion with 75% or Higher_2%

Student Retention_2%

Proportion Who Graduate_2%

Out of Province (1st Year)_1.5%

International (1st Year)_0.5%

International (Graduate)*_1%

Student Awards 3%

CLASSES_17% to 18%

Class Sizes: 1st and 2nd Year Level 7% to 7.5% Class Sizes: 3rd and 4th Year Level 7% to 7.5% Classes Taught by Tenured Faculty 3%

FACULTY_ 17%

Faculty with Ph.D.s_3%

Awards per Full-time Faculty_3%

Social Sciences and Humanities Grants 5.5% Medical/Science Grants 5.5%

FINANCES_12%

Operating Budget_3.3%

Scholarships & Bursaries_4.3%

Student Services 4.3%

LIBRARY_12%

Total Library Holdings!_1%

Holdings per Student_3% to 4%

Acquisitions_4%

Expenses 4%

REPUTATION_19%

Alumni Support_3%

Reputational Survey 16%

’Comprehensive and Medical-Doctoral categories only tMedical-Doctoral category only

REPUTATIONAL RESPONSE RATE

THIS YEAR, Maclean’s solicited the opinion of 11,454 individuals across the country. They included university officials at each ranked institution, high school principals and guidance counsellors from every province and territory, the heads of a wide variety of national and regional organizations, plus CEOs and recruiters at corporations large and small. The reputational survey is both regional and national, dividing the country into four key areas: the western provinces, Ontario, Quebec and the Atlantic provinces. All respondents completed a national survey; university officials, principals and guidance counsellors also completed regional ones.

GROUP RESPONSE

RATE

University Officials_41.4%

High-school Principals_10.7%

Guidance Counsellors_8.9%

CEOs__8%

Corporate Recruiters_9.3%

Heads of Organizations_10%

TOTAL 12.6%