UNIVERSITY RANKINGS

STUDENTS GRADE THEIR UNIVERSITIES

70,000 students took part in three national surveys. Their responses suggest Canada’s universities still have some homework to do.

SANDY FARRAN,TONY KELLER November 19 2007
UNIVERSITY RANKINGS

STUDENTS GRADE THEIR UNIVERSITIES

70,000 students took part in three national surveys. Their responses suggest Canada’s universities still have some homework to do.

SANDY FARRAN,TONY KELLER November 19 2007

STUDENTS GRADE THEIR UNIVERSITIES

UNIVERSITY RANKINGS

70,000 students took part in three national surveys. Their responses suggest Canada’s universities still have some homework to do.

SANDY FARRAN

TONY KELLER

In 2004, the Ontario government commissioned former premier Bob Rae to prepare a report on post-secondary education. Raising concerns about the quality of undergraduate education, Rae called on Canada’s largest province to establish benchmark data on “key aspects of higher education,” and for “evaluating and publicly reporting on quality and system performance.” To this end, he

recommended that all Ontario universities participate every two years in a long-standing American student survey: the National Survey of Student Engagement, or NSSE. “I am a great believer that if you can’t measure you really can’t make change happen,” Rae told Maclean’s.

One way of measuring universities is to ask the opinion of their customers: the students. Are they satisfied with their education? Their

professors? Their choice of university? To try to answer those questions, we present the results of three national student surveys: the NSSE, a Canadian survey known as the Canadian Undergraduate Survey Consortium (CUSC), and a survey conducted by Maclean’s to cover off the small number of universities that took part in neither NSSE nor CUSC.

More than 70,000 Canadian students participated in these three national surveys, across nearly every university campus.

NSSE and CUSC each ask dozens of questions about specific aspects of the undergraduate experience—inside the classroom and beyond. Both surveys were commis-

sioned by the institutions themselves. NSSE, an American survey that 28 major Canadian universities took part in in 2006, focuses on student engagement. Most of the questions asked on NSSE are an attempt to find out how students are spending their time and

how “engaged” they are with their schools, their professors and their peers. It is mostly about asking students what they did, not how they felt about it. NSSE does include a few satisfaction questions, however, which you will see featured on pages 84 and 86. The

NSSE surveyed undergraduate students in first year and fourth year.

The CUSC survey also looks at detailed aspects of the undergraduate student experience, but takes a slightly different approach. Unlike NSSE, it includes many questions asking students to assess how satisfied they are, and where they would like to see improvements. The 2006 CUSC survey was conducted among a sample of 1,000 graduating-year students at each of the participating universities.

There are nine Canadian universities that did not take part in the 2006 CUSC or NSSE surveys. To provide readers with feedback from their students, Maclean’s earlier this year asked those universities to invite their students to take part in a Web-based survey based on the CUSC, using CUSC methodology and CUSC questions. Eight of the nine universities agreed.

Of the 47 universities appearing in the annual Maclean’s ranking of universities each fall, only Université de Moncton did not take part in any of the three surveys. It is not listed in any of the charts. York took part in both

THE SURVEYS: WHAT THEY ARE, AND HOW THEY WERE DONE

You will find results from three surveys on the following pages: the National Survey of Student Engagement (NSSE), the Canadian Undergraduate Survey Consortium (CUSC), and the Maclean's University Student Survey. The NSSE and CUSC surveys, which were commissioned by the universities, ask more than 100 questions about specific aspects of the undergraduate experience— inside the classroom and beyond—designed to provide universities with data to help them assess programs and services. On the accompanying pages are the responses to several key questions.

Launched in 1994, CUSC is coordinated through the University of Manitoba's department of housing and student life. In 2006,

25 universities took part, sending surveys to a random sample of approximately 1,000 senior-year undergrads at each university.

A total of 10,464 students responded.

The U.S.-based NSSE began as a pilot project in 1999 and is distributed to firstand senior-year students. In 2004, 11 Canadian universities participated for the first time with 14,267 students completing the survey.

Last year, that number had grown to approximately 60,000 students at 31 Canadian institutions taking part.

Nine institutions ranked in the annual Maclean's University Rankings issue did not participate in either the 2006 CUSC or NSSE surveys. To provide student feedback from these institutions, Maclean's asked them to take part in a short survey using questions drawn directly from the CUSC questionnaire, addressing such issues as the quality of teaching and the overall educational experience.

Eight of the nine universities agreed (Université de Moncton declined). CUSC wording was followed and CUSC methodology was also employed: participants in the Maclean's University Student Survey were randomly selected from students in their final year. Universities contacted selected students by email, inviting them to participate. Large universities contacted 1,000 students; smaller universities, with fewer than 1,000 students in their graduating year, surveyed the entire cohort.

The survey was conducted online by Angus Reid Strategies, and was active from Feb. 14 to March 12. To ensure that only those who had been chosen could take part, each individual was assigned a unique PIN. These PINs allowed Maclean's to identify students by university while guarding their anonymity.

The Maclean's survey achieved a 43 per cent response rate, with 2,683 students from

eight universities taking part. The results, when presented for all universities, are accurate within 1.52 per cent, 19 times out of 20. Individual institutional accuracy varies from plus or minus 3.06 per cent to plus or minus 8.88 per cent.

The Maclean's survey asked only eight questions from a much longer CUSC survey. As the Maclean's survey questions were not asked within the context of the larger CUSC survey, the inherent question ordering and placement bias may have been different in both surveys. This may have had an impact upon the comparability of the Maclean's results to the CUSC results, since measuring satisfaction across an extensive battery of specific questions can result in lower satisfaction scores than when asking fewer, more generalized questions. We have therefore chosen to present the CUSC and Maclean's surveys separately.

READING THE CHARTS

The charts published on the accompanying pages list the 28 Canadian universities that Maclean's ranks annually that participated in the 2006 NSSE survey as well as the 23 ranked universities that took part in the 2006 CUSC. For most charts, universities are listed in descending order, according to the percentage of survey participants who choose the highest level of satisfaction when responding, for example, "excellent."

When displaying the NSSE benchmark charts, universities are listed according to the benchmark scores associated with their senior-year students.

The NSSE and CUSC surveys include more than 150 questions; we have published eight (two from NSSE and six from CUSC) that are the most broad and summative of student attitudes. The Maclean's University Student Survey is modelled on eight of the broadest CUSC questions. In addition, we present five NSSE benchmark comparisons created by NSSE to compare performance in five key areas across all universities—American and Canadian—taking part in the 2006 survey. For a listing of data from past NSSE, CUSC and Maclean's surveys, please visit www.macleans.ca/oncampus and click on "Rankings."

National Survey of Student Engagement (NSSE)

The NSSE survey asks undergraduates dozens of detailed questions—as well as some broader ones—to assess how engaged they are with their schools, their profs and their peers. Hundreds of American universities, and a growing number of Canadian institutions, participate in the annual survey. Listed here are 28 ranked Canadian universities that took part in 2006.

How would you evaluate your entire educational experience at this institution?

FIRST-YEAR STUDENTS I Excellent(%) SGood(%) Queen's 53 38 Western 47 41 Nipissing 45 45 Guelph 43 47 Waterloo 41 44 Trent 40 45 Wilfrid Laurier 40 :~ 49 St. Thomas 38 51 McMaster 37 48 McGill 33 48 NSSE 2006* 32 52 Victoria 31 55 Lakehead 29 52 Alberta 28 53 Brock 28 53 Carleton 28 53 Laval 27 57 Ryerson 27 52 UNB (Fredericton) 26 60 UPEI 26 57 UBC 24 50 Laurentian 24 51 Ottawa 24 53 Dalhousie 23 54 Saskatchewan 23 60 Toronto 22 48 York 20 55 Concordia 19 58 VVindsor 16 55

SENIOR-YEAR STUDENTS • Excellent(%) Good(%) Trent 45 46 Guelph 43 46 UPEI 41 50 Queen's 41 45 St. Thomas 41 49 Western 38 47 Brock 37 40 Nipissing 36 53 Waterloo 36 47 VViIfrid Laurier 36 52 NSSE 2006* 35 50 McMaster 34 47 Victoria 29 57 McGill 26 52 New Brunswick 26 57 Ryerson 24 51 Alberta 23 56 Vindsor 23 53 Carleton 22 57 Concordia 21 55 Saskatchewan 21 60 roronto 21 48 Jalhousie 20 55 _akehead 20 52 _aurentian 20 52 _aval 20 59 JBC 18 52 `ork 18 53 Dttawa 12 55

NSSE and CUSC, but for many months refused to make public those results. After Maclean’s filed and pursued a request under the provincial access to information law, York released its data just prior to this issue going to press. So what do the surveys tell us?

Overall, students at smaller, undergraduate-focused universities say that they are generally more satisfied than students at larger, research-oriented universities. There are exceptions to this trend, with larger research powerhouses such as Queen’s, Guelph, Western and Waterloo getting high marks from their students. But on the whole, small schools tend to do better than larger institutions.

When the CUSC survey asked students, “has your experience at this university exceeded, met or fallen short of your expectations?” a substantial majority at all universities said that their expectations had been exceeded or met. However, at a surprising number of universities—all larger universities such as Calgary, Simon Fraser, Ottawa,

Montréal, UBC, Dalhousie and the University of Toronto Scarborough—around a quarter of graduating-year students say that their university experience had fallen short of their expectations.

Similarly, on the NSSE, in which both firstand final-year students took part, two broad satisfaction questions also elicited high overall positive responses, and not just at smaller undergraduate universities. When asked to “evaluate their entire educational experience,” a majority of students answered either “excellent” or “good.” However, while more than a third of students at many universities were willing to describe their educational experience as excellent, fewer than one in five senior-year students at Dalhousie, Lakehead, Laurentian, Laval, UBC and Ottawa were willing to give their education top marks.

When asked, “if you could start over would you go to the same university?” the majority of students at all universities answered “definitely yes” or “probably yes.” Once again,

the trend favoured smaller universities, but some larger universities also did well. For example, fourth-year students at a number of universities with a wide range of master’s and doctoral programs gave their schools grades above the NSSE average.

But by the time they reach fourth year, fewer students at most universities were willing to say that, “definitely yes,” they would choose the same university. For example, Ottawa’s score went down from 35 per cent to 17 per cent. Even top performers Queen’s and Western both declined between first year and fourth year from 60 per cent to 45 per cent. It seems that students are, for whatever reason, generally less likely to recommend their school after completing their course of studies there. Notable exceptions are Brock, UPEI and Trent.

When asked on the CUSC survey about the quality of teaching at their university, at no institution did a majority of students choose “agree strongly.” Students at smaller

National Survey of Student Engagement (NSSE)

NSSE surveys are distributed to undergraduate students in their first and final years. In general, senior students are more critical when evaluating their university experience. While the majority of students would choose to return to their alma mater, the number drops—in some cases sharply—for students in their final year as compared to freshmen.

If you could start over again, would you go to the same institution you are now attending?

FIRST-YEAR STUDENTS U Definitely Yes(%) Probably Yes(%) Queen's 60 31 Western 60 30 Guelph 54 35 Nipissing 54 35 Waterloo 53 36 Wilfrid Laurier 53 36 McGill 52 37 Laval 50 42 St. Thomas 49 37 Trent 47 36 McMaster 45 43 Victoria 45 43 UBC 43 43 Alberta 42 46 NSSE 2006* 42 41 Laurentian 41 39 Ryerson 41 44 Saskatchewan 40 49 Carleton 39 46 Lakehead 39 41 UNB (Fredericton) 39 47 Brock 38 45 UPEI 38 46 Concordia 37 48 Ottawa 35 46 Dalhousie 34 45 Toronto 34 43 York 32 51 Windsor 26 49

SENIOR-YEAR STUDENTS • Definitely Yes (%) Probably Yes (%) 3uelph 51 34 rent 51 37 ;t. Thomas 50 37 JPEI 46 41 Vilfrjd Laurier 46 36 5rock 45 38 ueen's 45 38 Vestern 45 38 AcG1II 44 38 IaterIoo 44 37 Jipissing 43 41 AcMaster 41 39 iSSE 2006* 41 40 Iictoria 39 47 aval 38 47 \Iberta 34 50 ;askatchewan 34 49 :oncordia 33 48 yerson 33 44 JBC 32 45 JNB (Fredericton) 31 49 :arleton 30 46 aurentian 30 41 oronto 29 39 akehead 28 44 tlindsor 28 45 `ork 24 45 )alhousie 21 49 )ttawa 17 50

campuses were once again most likely to indicate the highest level of satisfaction. And students at a number of other larger, arguably more prestigious universities—such as UBC, Saskatchewan, Calgary, Ottawa, Simon Fraser and U of T Scarborough—were least likely to agree strongly that they were satisfied with the quality of their university’s teaching.

Though university students clearly have complaints about specific aspects of their learning experience, most report that they are “satisfied” or even “very satisfied” with their university experience. On the CUSC survey, 89 per cent of students indicated that they were “satisfied” or “very satisfied” with their “overall educational experience,” while 82 per cent said their university had “met” or “exceeded” their expectations. Another 89 per cent also said they were “satisfied” or “very satisfied” with their “decision to attend

Canadian Undergraduate Survey Consortium (CUSC)

The CUSC survey targets a different student population each year: all undergraduates, first-year students and graduating students. The 2006 survey, including 23 ranked universities, covers students graduating this spring. Participating universities sent an extensive questionnaire to a random sampling of 1,000 students, asking for feedback on everything from academics to support services.

How satisfied are you with the overall quality of the education you have received at this university?

65 71 73 72 71 70 70 73 66 75 75 71 76 75 72 69 71 • Very satisfied (%) Satisfied (%) Nipissing 41 53 Winnipeg 35 59 Mount Saint Vincent 27 66 Lethbridge 26 65 Wilfrid Laurier 25 69 Victoria 21 72 Montréal 20 UNBC 20 Regina 20 UNB (Fredericton) 18 ______ Ryerson 18 Concordia 17~ Daihousie 17 Saskatchewan 16. UBC 15.• Carleton 15 Saint Mary's 15 Ottawa 14 Manitoba 13 Simon Fraser 13 Toronto (Scarborough)1 3 Calgary 12 York 12

How satisfied are you with your decision to attend this university?

U Very satisfied (%) Satisfied (%) Nipissing 54 39 Winnipeg 53 42 Victoria 41 54 Wilfrid Laurier 40 51 Lethbridge 36 55 UNBC 36 55 Mount Saint Vincent 34 54 Ryerson 32 58 Concordia 29 62 Regina 29 63 Saint Mary's 29 58 Dalhousie 27 57 UBC 26 58 Carleton 26 62 Saskatchewan 26 63 UNB (Fredericton) 25 64 Toronto (Scarborough)23 60 Simon Fraser 22 65 York 21 61 Manitoba 20 71 Calgary 18 61 Montréal 18 67 Ottawa 18 _____ 64

Maclean’s University Student Survey

Maclean's conducted its own survey among the following universities asking the identical question as CUSC.

Mount Allison 66 31 St. Francis Xavier 57 40 Acadia 54 42 Cape Breton 50 46 Bishop's 47 48 Brandon 47 43 Sherbrooke 47 46 Memorial 44 51

Maclean’s University Student Survey Maclean's conducted its own survey among the following universities asking the identical question as CUSC.

St. Francis Xavier 73 23 Sherbrooke 72 24 Brandon 63 30 Mount Allison 63 30 Acadia 61 32 Bishop's 61 32 Cape Breton 55 38 Memorial 52 43

their university.” In the Maclean’s survey, more than 90 per cent of students said they were “satisfied” or “very satisfied” with the “overall quality of education” and with “their decision to attend their university.”

Ontario schools taking part in the NSSE also presented their students with a few Ontario-only questions. When asked which areas inside the classroom were most in need of improvement, Ontario first-year students chose “improving the quality of teaching assistants.” Areas outside the classroom they said were most in need of improvement were “increasing contact with professors” and “working toward a better social environment.” Areas most cited by upper-year students as

Canadian Undergraduate Survey Consortium (CUSC)

Quality of teaching and the overall learning environment are of vital importance to the undergraduate. Most students responded positively to questions assessing faculty, with those at smaller schools being the most satisfied, particularly when asked about professors being reasonably accessible outside the classroom and being encouraged by their profs to participate in class.

My academic learning experiences at this university have been intellectually stimulating.

I Agree strongly (%) Agree (%) Nipissing 59 Winnipeg M 59 Mount Saint Vincent WKK: 61 Lethbridge 62 UNBC 63 Victoria 62 Wilfrid Laurier Ml 64 Montréal 62 Regina 66 Dalhousie 65 Manitoba 68 UNB (Fredericton) 65 UBC 64 Carleton 68 Toronto (Scarborough)23 64 Concordia 22 HMMBMMM 64 Saskatchewan 22 71 Calgary 21 69 Saint Mary's 21 66 York 21 64 Simon Fraser 20 71 Ryerson 19 68 Ottawa 17 64

Generally, 1 am satisfied with the quality of teaching I have received.

lAgree strongly (%) Agree (%) Nipissing 46MMHMHMHMBMI 50 Lethbridge 36MHHBHMM 53 Mount Saint Vincent 34MHMHHMM 60 UNBC 29MMMHÍ^HÉM^MMi 63 Wilfrid Laurier 29MHHHHB1V * I 64 Regina 27 ' 64 Victoria 27 MMMMBMMNMMMNHNNMI 65 Saint Mary's 26MMMM1 64 Dalhousie 62 Montréal 24MMHÍ^^^H'3 60 Concordia 23MMMM1 58 UNB (Fredericton) 23MHMM 68 Ryerson 23MMMM1■NMMMHNMMMI 61 Manitoba 22 ■■■HÜNMNMK^Y 65 Carleton 21 68 UBC 19MMMI HMMIMilliliS....... 64 Saskatchewan 19HI^M 71 Calgary 18 H^HMMMMMMMMMHMi 65 York .......... 64 Ottawa 17 IÍHMMMMNÉÍNMHHMMI 59 Simon Fraser 17I^^HHNMNMMNMMMM^ - 71 Toronto (Scarborough) 16MIM 71

Maclean’s University Student Survey

Maclean's conducted its own survey among the following universities, asking the identical question as CUSC.

Mount Allison 34 Sherbrooke 36 St. Francis Xavier 44 Acadia 47 Bishop's 47 Cape Breton 49 Brandon 55 Memorial 58

Maclean’s University Student Survey

Maclean's conducted its own survey among the following universities, asking the identical question as CUSC.

Mount Allison 64 34 St. Francis Xavier 52 46 Acadia 51 47 Cape Breton 46 52 Bishop's 45 52 Sherbrooke 41 ! 50 Brandon 28 » 65 Memorial 27 11 67

needing improvement were “the quality of course instruction by professors” and “providing students more opportunities to undertake research with faculty.”

So what do all of these results mean?

One university whose students give it among the lowest satisfaction ratings on the CUSC survey is the University of Manitoba. Across town, students at the smaller, undergradute-focused University of Winnipeg gave their school among the highest satisfaction ratings. Tessa Vanderhart, a third-year political science student at Manitoba, calls these results “not unexpected. I think student satisfaction is higher at Winnipeg because everyone knows each other there.” As to the question of the validity of the results, “I think the perceptions people have [of the two univer-

sities] are true, so people generally know what they are getting themselves into.”

Is a university with higher student satisfaction a better university? “I think one should take student survey responses seriously,” Carl Wieman, Nobel Prize winner in physics and director of the Carl Wieman Science Education Initiative at UBC, told Maclean’s earlier this year. “But one should not automatically assume that the best policies are to follow everything students say they would prefer.”

For example, argues Wieman, the finding that students at smaller universities tend to have higher satisfaction levels than those at larger universities may not tell us anything about the quality of education. “I know everybody likes something sort of more personal-

ized,” says Wieman. “And that does happen better at small places, and so people are happier. Whether the actual education they get is better is quite a different question, and frankly that’s not something students are necessarily in a position to evaluate, at least while they’re going to school.”

For NSSE participants, the key results are not the answers to the satisfaction questions, but the school’s performance on five performance benchmarks, measuring “engagement.” NSSE assumes that engagement is a correlate of quality, or a measure that indicates that more and better learning is likely to be taking place.

Each university participating in the NSSE receives a benchmark report comparing scores from firstand fourth-year students on key

Canadian Undergraduate Survey Consortium (CUSC)

While students weren't shy about expressing criticism when answering detailed questions about university facilities, faculty and extracurricular activities, on the whole most showed satisfaction with their time at university. Students at small universities were somewhat more satisfied than their large-school peers. Overall, 86 per cent would recommend their university to others.

Has your experience at this university exceeded, met or fallen short of your expectations?

I Exceeded (%) Met (%) Nipissing 361 52 Winnipeg 281 62 Wilfrid Laurier 271 59 Lethbridge 261 61 Mount Saint Vincent 251 58 UNBC 251 63 Concordia 201 64 Victoria 201 70 Ryerson 19| 63 Saint Mary's 19| 65 Carleton 181 62 Regina 181 69 Dalhousie 151MB 61 UNB (Fredericton) 15BBBI 71 Saskatchewan 151 66 UBC 141 63 York 14 63 Ottawa 111 62 Simon Fraser 111 68 Toronto (Scarborough) 11 BB 63 Calgary 10BB 64 Manitoba 10 74 Montréal 9 66

Would you recommend this university to others?

B Yes (%) Hi No (%) Winnipeg 95 BBBBBBBBBBBBBBBBBBBBB 3 Victoria 93MBBBBBBBBBBB^^BBBBBB Nipissing 92 BBBBBBBBBBBBBBBBBBBBB' Wilfrid Laurier 92BBBBBBBBBBBBBBBBBBBB 7 Lethbridge 90 BBBBBBBBBBBBBBBBBBBBB I 10 90 BBBBBBBBBBBBBBBBBBBi UNBC 89HBBBBBBBflflBBBflBflBBBB 9 Saskatchewan 8 9 ■BBBBBBBBBBBBBBBBBBBi 9 Concordia 88BBBBBBBBBBBBBBBBBBBI 12 UNB (Fredericton) 88BBBBBBBBBBBBBBBBBBB 11 Ryerson 8 8 BBBBBBBBBBBBBBBBBBBI 12 Manitoba 87 BBBBBBBBBBBBBBBBBBBB Carleton 86 BBBBBBHBBBBBBBBBBBRsCfl 14 Saint Mary's 86BBBBBBBBBBBBBBBBBB Simon Fraser 86BBBBBBBBBBBBBBBBBBB 13 Montréal 85 BBBBBBBBBBBBBBBBBBBI 14 Mount Saint Vincent 85BBBBBBBBBBBBBBBBBBBB 15 UBC 84BBBBBBBBBÍBÍBBÍÍBBBÍÍBiM York 80BBBBBBBBBBBBBBBBBBB Ottawa 79BBBBBBBBBBBBBBBBBB Toronto (Scarborough)78BBBBBBBBBBBBBBBBBBI 21 Dalhousie 77 BBBBBBBBBBBBBBBB 20 Calgary 73BBBBBBBBBBBBBBBBB 26

Maclean’s University Student Survey

Maclean's conducted its own survey among the following universities asking the identical question as CUSC.

universities asking the identical question as CUSC. St. Francis Xavier 58I 34 Acadia 531 38 Mount Allison 531 38 Bishop's 511 39 Sherbrooke 46I 42 Brandon 40I 52 Cape Breton 391 52 Memorial 331 58

Maclean’s University Student Survey

Maclean's conducted its own survey among the following universities asking the identical question as CUSC.

Sherbrooke 95 BBBBBBBBBBBBBBBBBBBBMS 5 Memorial 94 BBBBBBBBBBBBBBBBBBB 7 St. Francis Xavier 93 BBBBBBBBBBBBBBBBBBB' 7 Cape Breton 92 BBBBBBBBBBBBBBBBBBBBB 8 91 BBBBBBBBBBBBBBBBBBBi Bishop's 91 flBBBBBBBBBBBBBBBBBBB Mount Allison 91 BBBBBBBBBBBBBBBBBBBi 9 Brandon 86 BBBBBBBBBBBBBBBBBBBI B

questions with those of other participating institutions, including all of their Canadian and American peers. The key questions are then grouped together in five broad benchmark categories, each with an overall benchmark score: level of academic challenge faced by students; amount of active and collaborative learning; quality of student-faculty interaction; availability of enriching educational experiences; and supportiveness of the campus environment.

So what do the benchmarks tell us about the undergraduate learning experience at Canadian universities? For starters, Canadian universities at both the firstand finalyear level compare quite well to their American peers in the benchmark areas “level of academic challenge” and “supportive cam-

pus environment.” The academic challenge measure is made up of scores on questions in such areas as “number of assigned textbooks,” “number of written papers,” and “coursework that emphasizes analysis of the basic elements of an idea.”

Surprisingly, however, all Canadian universities participating in the NSSE scored below the NSSE benchmark average on the “student-faculty interaction” benchmark, and were also behind their American peers on “active and collaborative learning.”

Why? In 2004, a handful of Canadian universities participated in NSSE for the first time, and showed the same low scores. It may come down to nothing more than resources: Canadian universities have fewer. “I was surprised,” says Chris Conway, director of

institutional research and planning at Queen’s University. Given the resource disparity, he expected his institution and others to be behind their American peers, but not to that extent. “That was our first hard empirical evidence that showed resourcing matters on those apsects of learning that are directly related to student-faculty interaction.” Canadian universities do poorly on these benchmarks, relative to their American peers, because the Canadian institutions, with smaller per student budgets, have fewer professors for each student.

A growing number of Canadian universities are taking part in NSSE, and Bob Rae is not the only one pushing a greater reliance on its findings. “We want to be really good and give the best that we can to our students

NSSE 2006 BENCHMARK COMPARISONS

National Survey of Student Engagement (NSSE)

NSSE created benchmarks to compare performance in five key areas across all universities—American and Canadian—taking part in the 2006 NSSE survey. Level of Academic Challenge assesses the intellectual and creative demands on students, measuring such things as the number of assigned readings, written papers and reports, as well as coursework that emphasizes judgment and transforming information into more complex interpretations. Student-Faculty Interaction gauges professors as mentors, measuring how often students meet with faculty to discuss career plans or ideas outside the classroom, or work with them on research projects or other activities outside of course requirements.

Level of Academic Challenge

First-year results: Senior-year results: Trent (51.7/58.5) — St. Thomas (51.8/58.4) Brock (50/57.2) — Ryerson (52.1/56.5) — McMaster (52.4/56.2) Lakehead (50.9/55.9)Queen's (53.4/55.9) — Nipissing (49.9/55.8)UPEI (47.2/55.7) — NSSE 2006* (51.5/55.6) Wilfrid Laurier (52/55.6)Carleton (50.2/55.5) — Victoria (49.6/55.4)Guelph (48.2/55.3) — Laurentian (50.8/55.1) — York (49.7/54.9) — Dalhousie (48.8/54.8) UNB (Fredericton) (48.6/54.8) Waterloo (52.6/54.5)McGill (51/54.4) — Toronto (50.1/54.2)Ottawa (49.5/54.1)Concordia (49/54)Saskatchewan (47.4/53.6) Western (48.8/53.6)Laval(49.9/53.5)— Windsor (46.8/53.5)UBC (49.9/52.8)Alberta (49.3/52) — BENCHMARK SCORE 30 40 50 60

Student-Faculty Interaction

First-year results: Senior-year results: -NSSE 2006* (30.4/39.7) -•UPEI (24.8/38.8) — St. Thomas (26.8/37.9)Brock (22.3/37.3) — Trent (25.6/37.3) — Laurentian (23.1/36)Nipissing (26.7/36)UNB (Fredericton) (23.1/33.4) Ryerson (25.1/33.3)Dalhousie (23.4/32.8)Guelph (18.9/32.8)Queen's (21.8/32.6)Wilfrid Laurier (22.7/32.5) Saskatchewan (20.4/32) — McMaster (23.1/31.8) — Windsor (22.1/31.4) — Western (22.5/31.1)Concordia (21.7/30.8)Lakehead (24.2/30.7) — Victoria (21.8/30.7)Carleton (23.1/30.2)McGill (20.1/30.2) — York (22.4/29.8) — Toronto (19.4/29.1)Alberta (20.4/29) Laval (18.9/28.9) — Waterloo (21.1/28.6)UBC (20/27.4) — Ottawa (19.5/27.3)BENCHMARK SCORE

*NSSE 2006 represents results from 557 Canadian and American universities

and that’s what drives us,” says Harvey Weingarten, president of the University of Calgary. “It drives our behaviour, it drives our resource allocation, it’s why we do things like NSSE; that’s why we listen to what NSSE has to tell us.”

Many universities prefer NSSE to CUSC, and are less comfortable with the kind of student satisfaction questions that CUSC asks. Despite Queen’s high standing on satisfaction surveys, Conway maintains that satisfaction scores aren’t always useful, because they are highly dependent on the expectations that students have going in to a university. As a result, he says, questions of satisfaction may not be comparable among universities: “If high expectations are more or less met, then students respond accordingly. Or if they have low expectations and

NSSE 2006 BENCHMARK COMPARISONS

National Survey of Student Engagement (NSSE)

Active and Collaborative Learning assesses involvement and teamwork, measuring how often students work with classmates, make class presentations, or participate in community-based projects. Enriching Educational Experience recognizes that diversity and complimentary learning opportunities enhance academic programs. This includes internships and co-ops, community service, study abroad, as well as a campus environment that promotes contact among students from different backgrounds.

Active and Collaborative Learning

First-year results: Senior-year results: Brock (35.9/52.5)-•-» Ryerson (40/51.2)-•-•— NSSE 2006* (40.2/49.2)-»-•— UPEI (36/49.1)-•-•— Trent (35/48.3)-♦-♦UNB (Fredericton) (33.1/46.8) •-•St. Thomas (36.1/46.7)-•-•Wilfrid Laurier (37.7/46.6)-•-•Nipissing (38.3/46.2)-•-•Guelph (34.1/45.3)-•-♦Lakehead (38.5/45.2)-•-•McMaster (38.6/44.9)-•-•— Laval (37.9/44.4)-•-•— Laurentian (31.3/43.8)-•-•York (34.5/43.7)-•-•Queen's (36.1/43.6)-•-•Saskatchewan (31.3/43.3) -•-•Concordia (34.8/43.2)-•-•Dalhousie (35.1/42.9)-•-•Carleton (35.2/42.8)-•-•— Victoria (32.5/42.8)-♦-•Windsor (32.1/42.6)-•-•Alberta (33.7/42.1)-♦-•Ottawa (31.1/41.7)-•-•McGill (34.6/41.2)-♦-♦Western (32.3/40.5)-•-•UBC (34.2/39.7)-•-•Waterloo (33.7/38.9)-♦-•Toronto (29.7/35.6)-•-•BENCHMARKSCORE 30 40 50 60

Enriching Educational Experience

First-year results: Senior-year results: -NSSE 2006* (26.3/38.9)Queen's (27.5/38.9)Waterloo (26.8/37.4)Guelph (24.7/36.9)McGill (26.8/36.7)McMaster (25.6/36.1)Brock (23.3/35.8)•Ryerson (25.3/35.6)-•— Trent (26.6/35.5)-• Wilfrid Laurier (25.4/34.1)•-• UBC (25.3/33.9)•-• Alberta (25/33.7) — Dalhousie (23.3/33.2)Western (26.3/33.2) — Carleton (24.3/32.7) — Victoria (24.1/32.7) — UNB (Fredericton) (21.7/32.5) Windsor (22.9/32.3) — Laurentian (22.9/31.9)St. Thomas (24.2/31.9) — Lakehead (24/31.6)Laval (21.2/31.6)Ottawa (23.4/31.5)UPEI (22.4/31.3) — Toronto (22.9/31.2)York (23.2/30.4) — Saskatchewan (20.2/30.3)Concordia (22.8/30)Nipissing (24.6/29)BENCHMARK SCORE 10 20 30 40

*NSSE 2006 represents results from 557 Canadian and American universities

nothing happens to subvert that, then students also give relatively consistent satisfaction scores.” Conway doesn’t reject satisfaction questions, and says that they are “roughly correlated, but not particularly well, with NSSE engagement scores.” But he doesn’t view satisfaction scores as being as useful as the more “empirical” measures of NSSE. “Satisfaction readings have to be taken with a grain of salt,” says Conway.

For example, a student at a primarily residential university, with all the social life and student interaction that implies, might be more satisfied than someone at a commuter school. And, given that they attend a small, residential school, NSSE results might also indicate that they are more engaged. Conway cautions against making too much of student satisfaction data. “On average, Canadian students are reasonably satisfied and that’s

NSSE 2006 BENCHMARK COMPARISONS

National Survey of Student Engagement (NSSE)

Supportive Campus Environment recognizes that students perform better at schools that cultivate positive working and social relationships between students, faculty and administrative staff, as well as providing support for academic and non-academic endeavours.

Supportive Campus Environment

First-year results: - Senior-year results: -Trent (59.2/60.2) — • • UPEI (58.2/59.9)• • Nipissing (63.5/59.8) — St. Thomas (58/57.9)Brock (56.3/57.3) — Guelph (60.6/56.9)• • Queen's (60.7/55.8) NSSE 2006* (58.3/55.6) — • • Wilfrid Laurier (59.8/55.3) — McMaster (58.3/53.4)UNB (Fredericton) (53.9/53) •• Victoria (56/52.8)Laval (57/52.7)Western (58.6/52.5)Laurentian (53.7/51.8)Windsor (51.7/51.8) — Concordia (52.5/51.5)-••Saskatchewan (51.8/51)-• •Lakehead (56.2/50.8)-•-• Ryerson (55.6/50.3)-•-•Carleton (56.5/50.2)-•-• Waterloo (57.5/49.5) — —•-• Dalhousie (50.9/49.1)-••Alberta (53.4/48.6)-•-•York (51.1/46.8)McGill (50.9/45.6) — UBC (50.8/44.9)•Toronto (51.6/44.8)—• Ottawa (52/44.6)-• BENCHMARK SCORE 30 40 50 60

*NSSE 2006 represents results from 557 Canadian and American universities.

a good story. I wouldn’t make hay out of what appear to be minor differences.”

To help improve the student experience, and in response to the findings of these surveys, universities are developing innovative programs and support services to help improve the student experience. In 2005, the University of Calgary established the Quality Money fund. The student government, in consultation with the student community, can direct money to projects that improve the student learning experience. Last year, the student union directed $1.37 million to establish a variety of projects such as an undergrad research program in health and wellness, a class-size reduction program, and the establishment of a Teaching Excellence Awards program. UBC hired Nobel laureate Wieman to study

and reform the teaching of science at the undergraduate level. And many universities are introducing new ways to deliver first-year programs that give students a chance to experience small seminar-type settings that are more often associated with upper-year courses. These small learning groups are particularly important at large universities where first-year students often sit in large lecture halls with hundreds of others, rarely getting a chance to ask a question or discuss ideas with the professor or fellow students. In fact, 17 per cent of Canadian first-year students told the NSSE that they have never asked a question in class, compared to just three per cent of their American peers. All Canadian universities are trying to address this problem. For example, the University of Toronto has

First Year Learning Communities in the faculty of arts and science, bringing together groups of 24 students in the same sections of first-year courses in a regularly scheduled meeting facilitated by an upper-year peer mentor. The meetings include social, developmental and academic programming. In addition, a staff and faculty adviser attend the meetings.

Even though universities rely heavily on surveys to improve the quality of the undergraduate learning experience, many universities are still reluctant to make this information public. Where universities declined to provide this data, Maclean’s filed access requests through provincial Freedom of Information and Protection of Privacy legislation. Several universities released data as a result of the filings. M