UNIVERSITY RANKINGS

Students grade their universities

Many top schools get less than stellar marks from their own students

SANDY FARRAN November 13 2006
UNIVERSITY RANKINGS

Students grade their universities

Many top schools get less than stellar marks from their own students

SANDY FARRAN November 13 2006

Students grade their universities

UNIVERSITY RANKINGS

Many top schools get less than stellar marks from their own students

SANDY FARRAN

If you’re wondering whether you’ll enjoy dining at a certain restaurant, you may want to consider the opinion of people who have already eaten there. If you’re planning on buying a new car, it pays to know what current owners have to say about the quality and reliability of different models. And when considering a university,

you’d be well advised to talk to people with first-hand experience: students. Most parents and prospective students do that by talking to a few friends or acquaintances. The sample size is small; the information is anecdotal. There is a more scientific way.

On the pages that follow are the results of three national surveys. Two are surveys of university students; the other is a poll of recent graduates. In total, more than 54,000 people responded. They were asked some basic questions about their university experience: are you satisfied with the quality of

teaching at your university? How would you rate the overall educational experience? Would you recommend this university to a friend or relative? Are you satisfied with your decision to attend?

The results are often surprising. Broadly speaking, they show that students at small, primarily undergraduate universities are often more satisfied and more engaged than their peers on larger, often urban, campuses. They show a number of less prestigious universities whose students are quite happy with the education they received, and many high-prestige institutions—including some leaders on the annnûMaclean’s university rankings—whose students have markedly less positive things to say about their experience. The results also show many large Canadian institutions underperforming their American peers.

However, though many large institutions do poorly on measures of student satisfaction, some do extremely well, raising questions as to why they outperformed, and what they are doing differently than their peers. Being relatively large is not necessarily a barrier to student satisfaction and engagement, at least according to students at midto large-sized institutions such as Queen’s, Guelph, Waterloo and McMaster.

The surveys do show that students on the whole are more satisfied than not with their university educations. For all institutions, the two possible positive responses—such as “excellent” and “good” or “agree strongly” and “agree”—were chosen by survey-takers more often than the negative responses.

THE SURVEYS: WHAT THEY ARE, AND HOW THEY WERE DONE

You will find results from three surveys on the following pages: the National Survey of Student Engagement (NSSE), the Canadian Undergraduate Survey Consortium (CUSC), and the Maclean’s University Graduate Survey. Both the NSSE and CUSC, which were commissioned by the universities, ask more than 100 questions about specific aspects of the undergraduate experience—inside the classroom and beyond—designed to provide universities with data to help them assess programs and services. On the accompanying pages are the responses to several of those questions.

Launched in 1994, CUSC is coordinated through the University of Manitoba’s department of housing and student life. In 2005,28 universities took part, sending surveys to a random sample of approximately 1,000 undergrads each. A total of 12,783 students responded.

The U.S.-based NSSE began as a pilot project in 1999 and is distributed to firstand senior-year students. In 2004,11 Canadian universities participated for the first time with 14,267 students completing the sur-

vey. Two of those universities, Carleton and York, as well as eight other Canadian institutions, took part in the 2005 NSSE. That year, the average response rate was 43 per cent with 12,593 students responding.

The NSSE charts published on the accompanying pages list 18 Canadian universities that participated in either the 2004 or 2005 surveys. For each institution, we display results from the most recent survey in which they took part. Institutions that did not disclose their NSSE and/or CUSC results are listed in the tables, and identified as “Refused to make this information public.”

Participants for the Maclean’s survey were randomly selected from those who graduated in 2002, 2003 and 2004The 23 participating universities contacted selected grads by email and/or surface mail, inviting them to participate. Smaller universities, with fewer than 1,333 grads per graduating year, surveyed all graduates. Larger institutions could elect to survey all grads or implement a sampling plan—developed by the Maclean’s project statistician—to survey

a minimum of 4,000 grads.

The survey was conducted online and asked graduates eight questions about the quality of their student experience and the impact that their university had on their lives. The website was active from April 24 to June 1. To ensure that only those who had been chosen could take part, each individual was assigned a unique PIN number. These PINs allowed Maclean’s to identify grads by university, graduation year and program, while guarding their anonymity.

The Maclean’s survey achieved a 21 per cent response rate; 14,697 graduates from 23 universities took part. The results, when presented for all universities, are accurate within 0.8 per cent, 19 times out of 20. Individual institutional accuracy varies from plus or minus 2.7 per cent to plus or minus 6.8 per cent. For each university, the survey data were weighted by graduating year and program, to minimize non-response bias.

Maclean’s commissioned an independent ethical review by ethica Clinical Research Inc. of Montreal. It gave the survey its unconditional approval.

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Hundreds of American universities, and a growing number of Canadian institutions, participate in this annual undergraduate survey. Listed here are 18 Canadian universities that took part in 2004 or 2005. For each of these, we show only their most recent results. For comparison, we have included the 2004 results from the AAUDE, a group of leading U.S. public universities.

HOW WOULD YOU EVALUATE YOUR ENTIRE EDUCATIONAL EXPERIENCE AT THIS INSTITUTION?

FIRST-YEAR STUDENTS EXCELLENT (%) ^GOOD (%) Queen’s 55 S 38 Western 45i 42 Guelph 441 47 Waterloo 43i 43 McMaster 411 45 U.S. peer group* 37 51 McGII 34 50 Acadia 291 57 Toronto 291 45 Ryerson 281 50 Carleton 241 55 Alberta 231 55 Calgary 201 57 York 20? 56 UBC 198 52 Ottawa 19? 59 Regina 16 64 Windsor 151 57 New Brunswick Refused to make this information public

SENIOR-YEAR STUDENTS Queen’s 48 42 Guelph 45 44 McMaster 38? 48 Waterloo 47 U.S. peer group* 50 Acadia 331 53 McGill 32 I 49 Western 32 52 Alberta 251 56 Toronto 24« 48 Ryerson 22 52 UBC 20; 53 Carleton 20? 59 Regina 20 58 York 201 56 Windsor 19 57 Calgary 161 53 Ottawa 131 54 New Brunswick Refused to make this information public

IF YOU COULD START OVER AGAIN, WOULD YOU GO TO THE SAME INSTITUTION YOU ARE NOW ATTENDING?

FIRST-YEAR STUDENTS DEFINITELY YES (%) PROBABLY YES (%) Queen’s 64 27 Guelph 58? 34 Waterloo 531 35 McGill 52' 34 McMaster 521 37 Western 511 37 U.S. peer group* 50 38 Ryerson 42 42 Acadia 411 42 Toronto 41 41 Alberta 38 51 UBC 36 50 Ottawa 35; 48 Carleton 33 49 York 31 49 Calgary 29 : 52 Regina 29 57 Windsor 26 ; 48 New Brunswick Refused to make this information public

SENIOR-YEAR STUDENTS Queen's 57 31 Guelph 54 34 McGill 47 38 McMaster 46 39 Waterloo 46 36 U.S. peer group* 46 38 Western 431 38 Acadia 39 39 Alberta 34 48 UBC 32. 47 Carleton 47 Ryerson 46 Toronto 30 40 Regina 29 49 York 271 46 Windsor 2446 Ottawa 221 45 Calgary 20 46 New Brunswick Refused to make this information public

*U.S. peer institutions are members of the AAUDE, a group of major American public universities. The results are for 2004.

The telling difference is between universities that received a high number of “excellent” or “agree strongly” responses, and those where few students were willing to offer the strongest level of support.

All three surveys are scientific studies, with

the individuals who took part randomly chosen, and representative of the underlying population of students or grads. (For more on the methodology, see the sidebar “The surveys: what they are and how they were done”) Two of the studies—the National Survey of Student Engagement (NSSE) and the Canadian Undergraduate Survey Consortium (CUSC)—were conducted by and for the universities. The results published on

these pages have traditionally been for university administrators’ eyes only. For many schools, this information has never before been made public. The third study, the Maclean’s University Graduate Survey, was conducted on behalf of Maclean’s, with the cooperation of 23 universities.

The survey results point to two long-standing sources of concern among some observers of Canada’s universities: a sense that undergraduate teaching is not sufficiently valued at Canada’s big research universities, and the challenge taxpayers and students have in assessing individual university performance, because of a dearth of information.

CANADIAN UNDERGRADUATE SURVEY CONSORTIUM (CUSC)

Universities participating in CUSC sent an extensive questionnaire to a random sampling of 1,000 undergraduates, asking for feedback on everything from academics to support services to campus safety. The CUSC results show a strong relationship between school size and satisfaction—with students at smaller schools generally more satisfied than their large school peers.

HOW SATISFIED ARE YOU WITH THE OVERALL QUALITY OF THE EDUCATION YOU HAVE RECEIVED AT THIS UNIVERSITY?

■ VERY SATISFIED (%) |SATISFIED (%) N ¡ pissing 55 Wilfrid Laurier 60 Winnipeg 60 2 6 UNBC 25^^———61 Mount Saint Vincent 2464 Carleton 66 Simon Fraser 2364 Dalhousie 64 Lethbridge 68 Victoria UNB (Fredericton) 67 Ryerson 66 Toronto (Scarborough) 62 Alberta 70 UBC 17—^^^^^^—— 65 Lakehead 70 Regina 69 Saint Mary’s 70 Windsor 68 Calgary 62 Manitoba 72 Montréal 67 Ottawa 72 Saskatchewan 12 74 York 12iWÜ¡j¡^ 68 Concordia Refused to make this information public

B AGREE STRONGLY (%) | AGREE(%) Wilfrid Laurier 44 Nipissing 44 Montréal 54 UNBC 54 Winnipeg 55 55 Victoria Carleton Mount Saint Vincent 57 Lethbridge 65 Dalhousie 58 UNB (Fredericton) 63 Ryerson 29HmHBBBIB 57 Ottawa 61 Saint Mary’s 2862 Alberta 66 Lakehead 62 Simon Fraser 58 UBC 25M^M^MMBBB|||^^B 58 Toronto (Scarborough) 24 55 Regina 67 Manitoba 22WÊÊÊÊÊÊÊ68 Saskatchewan 70 2 2 Windsor 21 64 Calgary 56 Concordia Refused to make this information public

I AM SATISFIED WITH MY DECISION TO ATTEND THIS UNIVERSITY.

When asked which university services were most in need of improvement, 41 per cent of students taking part in the 2005 CUSC survey named “emphasis on teaching excellence” as a top priority. Students at large research universities were the most likely to have called for a greater emphasis on teaching, with 48 per cent of them calling it a priority, compared to just 32 per cent at small, largely undergraduate institutions. And when CUSC asked them to respond to the following statement—“Generally I am satisfied with the quality of teaching I have received”—only 11 per cent of students at large research universities answered “agree strongly.”

A similar big school/small school split was in evidence when CUSC asked students to respond to the statement, “I am satisfied with my decision to attend this university.”

At small undergraduate universities, 39 per cent agreed strongly, compared to only 25 per cent at large research universities taking part in CUSC. And nearly twice as

many students at large schools disagreed or strongly disagreed with the statement. When asked about their “satisfaction with overall quality of education,” there was, again, a split. Among students at small undergraduate universities taking part in CUSC, 28 per cent indicated the highest level of satisfaction-double the percentage at larger research universities. Similarly, the percentage saying that they were dissatisfied or very dissatisfied with their overall education was almost twice as high among big school students.

NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE)

Many Canadian universities taking part in NSSE requested that an additional 10 questions be put to their undergraduates. Two questions, featured below, asked students to evaluate the quality of teaching in their first-year and senior-year courses. No comparison to peer American universities is offered, as their surveys did not include these questions.

OVERALL, HOW WOULD YOU RATE THE QUALITY OF INSTRUCTION IN THE FIRSTAND SECOND-YEAR COURSES YOU’VE TAKEN AT THIS UNIVERSITY?

FIRST-YEAR STUDENTS EXCELLENT (%) HGOOD(%) Queen’s 26 60 Waterloo 25 "54 Western 20 ~~5Ï McMaster 18 60 Toronto 18 "52 Acadia 16 ~64 McGill_16 "59 Carleton 15 60 Ryerson 14 "53 York 12 "55 Alberta 10 ~57 Regina 10 Ü7 Windsor 10 ~56 Ottawa 9 ~53 UBC 8 ~47 New Brunswick Refused to make this information public

SENIOR-YEAR STUDENTS Waterloo 151 54 McGill 41 Acadia 60 Queen’s 54 Toronto 39 McMaster 10 I 48 York 49 Carleton 51 Ryerson 44 Western 49 Windsor 45 Alberta 40 UBC 37 Ottawa 44 Regina 46 New Brunswick Refused to make this information public Note: Guelph and Calgary did not include this question in their survey.

SENIOR-YEAR STUDENTS EXCELLENT (%) ■ GOOD(%) Queen’s 45 45 Acadia 41 45 Waterloo 3746 McMaster 36 48 Alberta 311 50 McGil 311 51 Toronto 311 49 Western 311 47 Carleton 30 52 Regina 26 54 York 26 53 UBC 24 55 Ottawa 211 53 Ryerson 18 50 Windsor 14 48 New Brunswick Refused to make this information public Note: Guelph and Calgary did not include this question in their survey.

READING THE CHARTS

OVERALL, HOW WOULD YOU RATE THE QUALITY OF INSTRUCTION IN THE THIRDAND FOURTH-YEAR COURSES YOU’VE TAKEN AT THIS UNIVERSITY?

; Universities are listed in descending order, according to the perj centage of survey participants who choose the highest level of satÎ isfaction when responding, for example, “excellent.”

The NSSE and CUSC surveys include hundreds of questions; we ; have published seven (four from NSSE and three from CUSC) that ; are the most broad and summative of student attitudes. The ; Maclean’s University Graduate Survey asked participants eight I questions; for reasons of space, only three are included here. For ; a full listing of the remaining Maclean’s University Graduate Suri vey charts, as well as data from our 2004 grad survey, please visit ! our website at www.macieans.ca/university.

Through provincial access to information legislation, Maclean’s ! has requested NSSE and CUSC data from universities that have rej fused to make this information public. Maclean’s will publish this ; data when it is released.

Last spring, as part of our first annual University Student Issue, Maclean’s asked all of Canada’s ranked universities to take part in the Maclean’s university graduate survey. Twenty-three universities agreed. The magazine also asked all universities that had taken

part in NSSE and CUSC to make that data public. Some agreed to do so. Most refused.

To get NSSE and CUSC data from universities that declined to release it, Maclean’s filed 22 freedom of information requests in six provinces. Provincial freedom of information legislation covers governments and public bodies in most provinces; it is designed to foster transparency and accountability, by making information available to the public. Some universities responded favourably to our freedom of information requests, by immediately posting the results on their websites. In other cases, the request

was denied by the university, leading Maclean’s to formally appeal to the respective provincial information commission.

Just like a court case, a freedom of information request can take months to work its way through the system. As a result, the University Student Issue went to press last June without NSSE and/or CUSC results from 12 universities: Concordia, Dalhousie, Lakehead, Manitoba, Montréal, Mount Saint Vincent, New Brunswick, Saint Mary’s, Saskatchewan, Toronto (Scarborough campus only), Windsor and York. In the University Student issue, Maclean’s promised that we would pursue our freedom of information requests, and publish complete information on all missing schools in an upcoming report: and now, here it is.

CANADIAN UNDERGRADUATE SURVEY CONSORTIUM (CUSC)

In the 2005 CUSC questionnaire, 12,783 undergraduates took the opportunity to state their opinion on important aspects of the learning environment, including quality of teaching.

GENERALLY, I AM SATISFIED WITH THE QUALITY OF TEACHING I HAVE RECEIVED.

I AGREE STRONGLY (%) JAGREE (%) Nipissing 30MBIi^H—^BB^BBil^—B— 63 Winnipeg 24: TTWTTBB-^^HB-B^B 67 UNBC 23^ 63 Wilfrid Laurier 22HM^^^HHm 71 Mount Saint Vincent 211^1B^^^H—69 Brock 20 67 Victoria 18BBB^^^^^^^^^^^^^B Dalhousie 171 ■BBB—BHI^^^I 67 Lethbridge 17 ^^^^^^^BBBBB^H 74 Montréal 17 70 Carleton 16 | 69 Saint Mary’s Lakehead 15BBHBI^^^^^^^^^BI 67 Ryerson 14 ^^^^^^^BBl^^B 63 Simon Fraser 14BBHBBB^^^^^^^^I 63 Toronto(Scarborough) 14 ¡¡^^^■■BH 64 Alberta 12BI^^BBI^^^^^^^^^B 70 Manitoba 12ÜHI^^^BBI^^^^^^^^B 70 UNB (Fredericton) 12BB^^^^^^BBBI^^^B 72 Regina 121'T^I^^^^^^BBI^^^^^^l 74 UBC 11 ■•61 Ottawa 10 69 Windsor lOHI^^^^BB^^^^^BBi 67 Calgary 9 I 65 Saskatchewan 9 70 Concordia Refused to make this information public

PRIMARILY UNDERGRADUATE if VERY GOOD (%) I GOOD (%) Cape Breton 81 BBBBBB—1^— 16 St. Francis Xavier 79 B——^Bi^^B 18 Mount Allison 76 —■ 22 St. Thomas 75 BBi^BB—BB—23 Nipissing 74 ÍB—NNHHB—— 22 Bishop’s 73 BBB—B—BB—26 Acadia 70 BH^BBIB28 Winnipeg 70 BBBBB^H28 Wilfrid Laurier 66 ——51 UNBC 59 BBBB— ■ UPEI 56 ■—HT Lakehead 55 —BH^B 39 Mount Saint Vincent 52 a—BBI—^B 42 Moncton 42 BÍ—BBBB— 50

COMPREHENSIVE_ Guelph 76—BB^^^^BBBBI^BB^^^^B 22 Waterloo 64^BBBBBB^B 32 Memorial 53BHBBBB^^^B^^^^^^^^B 43 New Brunswick 511^BH 43 Windsor 43BBBBBBBH^^^^^^^^^B 48

MEDICAL DOCTORAL Sherbrooke 68 I 31 Western 59 34 Saskatchewan 52 43 Laval 44 "TT

MACLEAN’S UNIVERSITY GRADUATE SURVEY

Twenty-three universities took part in the 2006 Maclean’s University Graduate Survey, but 24 declined, including six in the Comprehensive category and 11 in the Medical Doctoral.

HOW WOULD YOU RATE YOUR ENTIRE EDUCATIONAL EXPERIENCE AT THIS UNIVERSITY?

Forty-two of Canada’s 47 ranked universities are covered. Three of the missing institutions did not take part in any of the surveys: Trent, Brandon and Laurentian. Two others—Concordia and the University of New Brunswick—did take part, but have so far refused to make their information public. As we went to press, our freedom of information request for Concordia is still making its way through the legal process. Maclean’s was unable to file a freedom of information request in New Brunswick, as it is one of the few provinces whose information and privacy legislation does not cover universities.

The questions published by Maclean’s are just one aspect of what the questionnaires ask about the undergraduate experienceinside the classroom and beyond. The three surveys are similar, but not perfectly comparable: they polled different groups, at different times, at different schools. Even the surveys’ purposes are slightly different.

MACLEAN’S UNIVERSITY GRADUATE SURVEY

Maclean’s survey results are grouped according to categories from Maclean’s annual university rankings. Medical-Doctoral institutions offer undergrad education and significant graduate programs and research, including medical schools; Comprehensives offer a range of undergrad and grad programs; Primarily Undergraduate institutions focus on undergrad education.

WOULD YOU RECOMMEND THIS UNIVERSITY TO A FRIEND OR RELATIVE INTERESTED IN A SIMILAR COURSE OF STUDIES?

PRIMARILY UNDERGRADUATE ; DEFINITELY YES (%) I PROBABLY YES (%) Nipissing St. Francis Xavier 10 Cape Breton TF St. Thomas 16 Winnipeg "TF Bishop’s "TF Wilfrid Laurier "TF Mount Allison "TF Acadia "TF UNBC "FT UPEI "FF Lakehead "FF Mount Saint Vincent "FF Moncton FF

COMPREHENSIVE Guelph 14 Waterloo 17 Memorial 27 New Brunswick FF Windsor FF

MEDICAL DOCTORAL Sherbrooke 16 Saskatchewan 23 Western FF Laval FF

PRIMARILY UNDERGRADUATE DEFINITELY YES (%) PROBABLY YES (%) St. Francis Xavier Bishop’s 13 Cape Breton TF Mount Allison 15 Nipissing FF Acadia 17 Wilfrid Laurier FF St. Thomas FF Winnipeg FF Lakehead FF UNBC FF UPEI 731 FF Moncton "FF Mount Saint Vincent "FF

COMPREHENSIVE Guelph 14 Waterloo TF Memorial 20 New Brunswick "FF Windsor "FF

MEDICAL DOCTORAL Sherbrooke 17 Saskatchewan TF Western TF Laval FF

THINKING BACK ON YOUR TIME AT UNIVERSITY, DO YOU FEEL THAT THIS EXPERIENCE WAS OF SIGNIFICANT BENEFIT TO YOUR LIFE TODAY?

Most of the questions asked on the NSSE survey attempt to find out how students are spending their time and how “engaged” they are with their schools, their professors and their peers. It is largely about asking students what they did—not how they felt about it. CUSC similarly looks at detailed aspects of the student experience, thought it does ask students to not merely say what they did, but to assess how satisfied they were, and where they would like to see improvement. The Maclean’s survey is a shorter survey, asking grads to evaluate their university by answering a series of broad, summative questions.

The surveys are not exactly the same, but there are several questions on NSSE, CUSC and the Maclean’s survey that are broadly similar, asking survey-takers to assess their university experience. Those questions, and their answers, are presented here. M